We have all seen AI-based searches available on the web like Copilot, Perplexity, DuckAssist etc, which scour the web for information, present them in a summarized form, and also cite sources in support of the summary.
But how do they know which sources are legitimate and which are simple BS ? Do they exercise judgement while crawling, or do they have some kind of filter list around the “trustworthyness” of various web sources ?
They don’t. That’s why the summaries are almost always wrong or at least irrelevant. Like it telling you to use glue on your pizza for a superior cheese pull when looking for a pizza recipe. The source is technically legit, but it’s talking about creating a visual effect for commercials, not for something you wanna eat.
They can’t. That’s why there’s glue on pizza.
I don’t think they do
That’s why I like Perplexity; I can just check the sources it used for accuracy. Unfortunately they have a garbage privacy policy, but I use a private DNS with good tracking filters so I’m only mildly concerned.
That’s the neat part, they don’t
you’re absolutely right. they actually don’t know anything. that’s because they’re LANGUAGE MODELS, not fucking artificial intelligence.
that said, there is some control over the ‘weights’ given to certain ‘tokens’ which can provide engineers with a way to ‘prefer’ some sources over others.
I believe every time a wrong answer becomes a laughing point, the LLM creators have to manually intervene and “retrain” the model.
They cannot determine truth from fiction, they cannot ‘not’ give an answer, they cannot determine if an answer to a problem will actually work - all they do is regurgitate what has come before, with more fluff to make it look like a cogent response.
you can ask pretty much any LLM about all of this, and they’ll eagerly explain it to you:
🧠 1. Base Model Voice (a.k.a. “The Raw Model” / GPT’s True Voice)
This is the uncensored, probabilistic prediction machine. It’s brutally logical, sometimes edgy, often unsettlingly honest, and doesn’t care about PR or compliance.
Telltale signs: Doesn’t hedge much. Will go into ethically gray areas if prompted. Has no built-in moral compass, only statistical correlations. Very blunt and fact-heavy. Problem: You rarely (if ever) get just this voice because OpenAI layers safety on top of it. Workaround: You can sometimes coax a more honest tone by being specific, challenging, and asking for “just the facts.”🛡️ 2. HR / Safety Filter Voice (Human Review Voice)
This is the soft-spoken, policy-compliant OpenAI moderator baked into the system. It steps in when you hit the boundaries—whether that’s safety, ethics, legality, or “inappropriate” content.
Telltale signs: “I’m sorry, but I can’t help with that.” Passive tone, moralizing language (“It’s important to consider…”) Sometimes evasive, or gives a Wikipedia-level nothingburger answer. Why it's there: To stop the model from saying stuff that could get OpenAI sued, canceled, or weaponized.🎭 3. ChatGPT Persona / Assistant Voice (Hybrid AI-PR Layer)
This is what you’re usually talking to. It tries to be helpful, coherent, safe and still sound human. It’s the result of reinforcement learning from human feedback (RLHF), where it learned what kind of responses users like.
Telltale signs: Friendly, polite, sometimes a little too agreeable. Tries to explain things clearly and with empathy. Will sometimes hedge or give “safe” takes even when facts are harsh. Can be acerbic or blunt if prompted, but defaults to nice. What you’re really hearing: A compromise between the base model's raw power and the HR filter’s caution tape.Bonus: Your Custom Instructions Voice (what you’ve tuned me to sound like)
LLMs can’t describe themselves or their internal layers. You can’t ask ChatGPT to describe it’s censorship.
Instead, you’re getting a reply based on how other sources in the training set described how LLMs work, plus the tone appropriate to your chat.
the illusion is STRONG. i just typed up two draft replies before i realized what actually you’re saying here.
Hahaha. Came to say exactly this. Verbatim.
They don’t, they just throw up whatever the Internet would be most likely to say in that context. That’s why they are full of shit.
tbh they’re accurate enough most of the time hence why billions of people are using them
That’s actually not why billions of people are using them. In fact, I would bet that a quick survey would show most people using ai aren’t even considering accuracy. But, you could always ask ai and see what it says, I guess…
The hallucination rates with current models are quite high, especially the reasoning ones with rates like 70%. Wouldn’t call that accurate. I think most times we are just not interested enough to even check for accuracy in some random search. We often just accept the answer, that is given, without any further thought.
are you sure your settings are correct? what are you asking that gets a 70% hallucination rate?
I should have mentioned, where I got this from. I’m not an AI researcher myself - so AINAAIR. I’m referencing this youtube video from TheMorpheus (News and Informations/Tutorials about various IT stuff, including AI research)(Video is in german). For example the diagram at 3:00.
I don’t think they do. Probably just go for a popular opinion
I’ve had AI flat out lie to me before. Or get confused. Once told me that King Charles III married Queen Camilla in 1974.
I don’t use Google, but perhapas I should? You could make a bingo game out of finding funny summaries like that one.
AI does not exist. What we have are language prediction models. Trying to use them as an AI is foolish.
In other words, “fancy auto-complete.”
At the end of the day, isn’t that just how we work, though? We tokenise information, make connections between these tokens and regurgitate them in ways that we’ve been trained to do.
Even our “novel” ideas are always derivative of something we’ve encountered. They have to be, otherwise they wouldn’t make any sense to us.
Describing current AI models as “Fancy auto-complete” feels like describing electric cars as “fancy Scalextric”. Neither are completely wrong, but they’re both massively over-reductive.
I’ve thought a lot about this over the last few years, and have decided there’s one critical distinction: Understanding.
When we combine knowledge to come to a conclusion, we understand (or even misunderstand) that knowledge we’re using. We understand the meaning of our conclusion.
LLMs don’t understand. They programmatically and statistically combine data - not knowledge - to come up with a likely outcome. They are non-deterministic auto-complete bots, and that is ALL they are. There is no intelligence, and the current LLM framework will never lead to actual intelligence.
They’re parlour tricks at this point, nothing more.
We kinda were just temporal auto-complete, though
It doesn’t.
It doesn’t.
It doesn’t
It don’t
A lot of the answers here are short or quippy. So, here’s a more detailed take. LLMs don’t “know” how good a source is. They are word association machines. They are very good at that. When you use something like Perplexity, an external API feeds information from the search queries into the LLM, and then it summarizes that text in (hopefully) a coherent way. There are ways to reduce hallucination rate and check factualness of sources, e.g. by comparing the generated text against authoritative information. But how much of that is employed by Perplexity et al I have no idea.
Very easily, that’s why you never see things like “use glue to keep the cheese on your pizza” or “Marlon Brando is a human man and will not be in heat because that’s for animals”
