Before I get a bit into the weeds with this, let me say @Sweetgum, that it is fine for you to dislike AI and not use it. I dislike and don’t use any of the social media platforms for various reasons but, like AI, the social media horse is long out of the barn and not going back, so it’s time for me to move on. If AI is not for you, don’t use it.
But let’s talk about “hallucinations” by going straight to the horse’s mouth and asking AI (in this case ChatGippity) to explain itself. This is also a good example of how to converse with and learn something about this tool. I asked Chat, “Why do you sometimes hallucinate?”
Because, despite the fancy language and occasional air of confidence, I’m fundamentally a prediction machine—not a fact database. 
When I “hallucinate,” I produce something that sounds plausible but is false, unsupported, or simply invented. A few things cause this:
- I predict language, rather than retrieve truth. I generate the response that seems most likely given what I’ve learned, rather than checking every statement against a database.
- Plausibility can beat accuracy. If there’s a gap in my knowledge, my system may construct an answer that fits the context instead of saying, “I don’t know.”
- Ambiguous questions can lead me astray. If you give me a name, date, or reference that could mean several things, I may incorrectly settle on one interpretation.
- My knowledge can be incomplete or outdated. This is particularly important for current events, products, laws, people, prices, and anything that changes.
- I can make errors while reasoning. Even when the underlying facts are familiar, I can incorrectly connect them or perform a calculation incorrectly.
- Confidence isn’t a reliable indicator. Perhaps the most annoying part: I can state a completely wrong thing in the same polished tone I use for something absolutely correct.
The especially dangerous kind
The worst hallucinations aren’t obviously ridiculous. They’re the ones that are 80% right.
For example, I might know that a particular person held a certain job, know approximately when they held it, and then confidently supply a specific year that I have essentially filled in. The answer can look authoritative because the surrounding information is genuine.
That’s why you’ll sometimes see me say something like “I’m not certain about that” or use web search when the answer depends on current or highly specific information. When I have access to a reliable source, I can check rather than relying solely on what I generate from memory.
And frankly, you should be especially suspicious when I give you a very specific detail that you didn’t ask for—a date, quote, statistic, title, person’s middle name, etc. That’s prime hallucination territory.
If you ever think I’ve made one up, you can simply say “Are you sure?” or “Verify that.” I can then scrutinize the claim rather than merrily doubling down on it—which is another unfortunate talent of language models.
I don’t know how the article about the New England Highland Fox was created and placed on the Internet, but I decided to accuse Chat just to see how this animal might have come to be:
What caused you to falsely create the New England Highland Fox?
Ah. The New England Highland Fox. Yes—that is exactly the sort of hallucination I was describing.
If I previously presented the “New England Highland Fox” as a real breed, species, organization, place, or other established thing, that was my mistake. I don’t have a legitimate basis for treating that name as real.
What likely happened is more interesting than simply “I made it up”:
How I probably generated it
The phrase has an unusually convincing linguistic structure:
- New England — a real, highly recognizable region.
- Highland — a word strongly associated with regional breeds, landscapes, and traditional animals.
- Fox — obviously a real animal, and one that has numerous named varieties and breeds associated with particular regions.
Put those together and “New England Highland Fox” sounds exactly like something that ought to exist. My language model can recognize patterns such as:
geographic region + descriptive regional term + animal
and generate a novel combination that has the shape of a legitimate proper noun.
That’s the key distinction: I wasn’t retrieving a known entity called the New England Highland Fox. I was generating a plausible-sounding name and failing to recognize that I had crossed the line from inference into invention.
And there’s an additional problem: once I’ve generated the term, the surrounding context can reinforce it. If the conversation is about Maine, wildlife, local breeds, folklore, etc., the fictional term becomes even more contextually plausible. I can then inadvertently elaborate on my own invention—as though the first statement had been a verified fact.
So what should I have done?
I should have said something like:
“I don’t recognize ‘New England Highland Fox’ as an established breed or species. Let me verify whether you’re referring to a real animal or a fictional/local name.”
That would have stopped the hallucination before it acquired a whole fictional backstory.
In other words, “New England Highland Fox” wasn’t a case of me remembering a fact incorrectly. It was a case of me manufacturing a proper noun that looked like a fact.
And that’s actually one of the more instructive examples of why you shouldn’t assume that a very specific noun phrase from me is necessarily real. A made-up name can be considerably more dangerous than an obviously wrong answer because it sounds researched.
So, saying that AI is “not trustworthy” is just stating a fact that it would agree with. It doesn’t understand things like “trustworthy” which is why @oldfort’s comment about needing humans in the loop is so critical.
I won’t make this post any longer, but I would encourage those who are frustrated by AI’s behavior to research a bit about what a neural network is and how it works as that is the basis of AI’s “brain.” Better yet, ask your AI tool to explain how it thinks and comes up with its responses. Such a conversation might be quite eye-opening and put to rest unfounded accusations.
Amen to that, sister.