It doesn’t mean automatically wrong. It’s just bullshitting. It makes up something that fits a pattern. Depending on the question, the pattern may be right more often than not.
If you ask ChatGPT, “hey is the McDonald’s near my house open at 6pm?” It doesn’t know anything about where you are or if there’s a McDonald’s or what its hours are. It will likely hallucinate that sure, it’s open at 6pm. But when it does so is it “right” in a meaningful way?
Yup. IMHO, I think "bullshitting" is a much better word than hallucinating and/or getting it right!
Much like real life bullshitters, it is inclined to say something truthful-sounding, but doesn't actually have a strong reliability towards truth per se.
Bullshitting implies intent to deceive. As far as we know, an LLM honestly "believes" (as if you need another rabbit hole) what it says. Delusion, perhaps?
Delusion implies a degree of consistency, though. LLMs can be on point one minute and completely off the rails the next even when prompted with the same prompt. Hallucination fits better here as it speaks to the real-time "perception" (there's another one for you).
An LLM is not a brain, though, so no matter which analogy you choose, it will come with some flaws. Regardless, "hallucination" has moved past analogy territory and now has its own LLM-specific usage with reasonably wide acceptance so the analogy angle is now moot anyway.
Not in the formal sense. The philosopher Harry Frankfurt famously distinguished bullshitting from lying because a liar knows the truth and is trying to hide it where a bullshitter is simply trying to sound convincing and may or may not be telling the truth (and may not even know themselves if they are)
In the current formal sense. It may be true the formal sense in 1986 was different. Words do evolve in meaning over time, but since we're talking about right now...
You are right that lying and bullshitting are different. A lie is a false statement with intent. Bullshit is nonsense with intent. A false statement and nonsense may share some similarities, but are ultimately different.
Perhaps nonsense is the word we should be applying to LLMs, but often what they say isn't nonsense, even if only by accident, so that doesn't exactly work either. Regardless, it doesn't matter now. As before, "hallucination" has moved beyond analogy and now has its own LLM-specific usage.
You can ask it „Is the McDonalds nearby open? I live in Brumbledon, Ohio.“
It will do a search, and then confidently state that the McDonalds in Brumbledlon, Ohio is in fact open 24/7.
I guess it doesn’t matter that no such town exists.
That's fair. I'm honestly stunned how little work there's been to incorporate search, in a real way, into products. perplexity, ChatGPT and bing to some extent, that's it.
If you ask ChatGPT, “hey is the McDonald’s near my house open at 6pm?” It doesn’t know anything about where you are or if there’s a McDonald’s or what its hours are. It will likely hallucinate that sure, it’s open at 6pm. But when it does so is it “right” in a meaningful way?