One of humanity’s most remarkable cultural achievements is the division of explanatory labor. When my daughter came down with a fever, I solved the problem by consulting a pediatrician, who shared a likely diagnosis of the flu. Unlike the doctor, I wasn’t able to tell my daughter much about the mechanism by which the flu leads to fever, or exactly why it was so important for her to keep drinking fluids. But the explanation I acquired—roughly “fever because flu”—was what was needed to treat her.
Society runs on this efficient type of information sharing. Just as coordinating specialized skills can streamline the fabrication of products—be it pins or cars or newspapers—coordinating specialized knowledge can facilitate the answering of questions. You can drive a car even if you don’t have the expertise to manufacture one yourself, and I benefited from the pediatrician’s explanation even though his understanding far exceeded my own.
We can thank this divide-and-conquer method for many advancements in medicine and science, engineering and commerce; in fact, it has helped drive the accumulation of human knowledge across communities and over time. But it also leaves us susceptible to illusions of understanding: cases in which we mistake the understanding of others for our own.
Yes, broadly speaking we decide if we don’t know better, but it sounds right, then it is probably right.
AI excels at sounding right no matter what.
But not new, if you take a look at most business successes, you’ll definitely find confidence, but often not competence. “Confidence men” have been a thing for a reason, and business leaders are usually con-men.
AI is good at being a con-man.
Limited Liability Model.
Human doctor earns trust though qualification and training period, has accountability to professional regulator, likely feels remorse for mistakes (even for misfortune) and, ultimately, can be sued or criminally prosecuted if they really fuck someone over.
Once an LLM has all that ‘skin in the game’ , i might consider trusting it with something important. TBH the remorse thing is the hardest to prove but the most important to build trust (IMO).
Limited Liability Machine
That’s giving LLMs far too little credit: They can worsen all the cognitive traps simultaneously.
The truth is that the developers are the problem. They put their bias into the feedback and the result is we don’t get a truth. Even though they programmed it to give you the right answer, it’s not the same as the factual true story
The perfect example is the results you get when you ask for results about animals and farming. Most animal protein is coming from industrial farms but getting that result from the AI is ultra difficult
Even though they programmed it to give you the right answer, it’s not the same as the factual true story
I’m not sure you know how it works ngl. Or this is just phrased weirdly.
I’m well aware of how it works and how it’s tinkered to give you result that are expected. Just because it’s an expected response doesn’t mean it’s the fact/true answer
Have you never been given a bad response and you point it out and the system is like oh yeah so sorry your right
