From what I hear from friends in the know, what we are seeing as consumers is several years behind what the most current AI can do. That’s why we can so easily dismiss it.
2 Likes
It is a misunderstanding of how AI works that is causing all this heartburn. Many expect it to be accurate when “accuracy” is not what it’s trained to do. AI doesn’t “know” things. At its simplest, it predicts the next likely word or number based on past usage, a whole lot of past usage, and it will continue to get better and better until it becomes difficult to distinguish it from actual sentience, but we’re not quite there yet.
Also, it’s important to understand that different AIs are trained to do different things. AI apps draw from the datasets they were trained on, and many are built with “intelligence” for specific purposes (special-use systems vs. general purpose systems like ChatGPT). So, for example, expecting an AI trained on a medical dataset to perform complex math is using the wrong tool for the job. We do not yet have one LLM that can reliably respond to every type of prompt, but they will behave as if they can. Thus, the disconnect.
But, never fear, these issues are being addressed with incredible speed. AI today is leagues ahead of where it was even a year ago.
This.
2 Likes
Regarding the I referenced above. An example might be asking an AI prompt to create a table showing stocks that are most correlated with ORCL, with columns showing 1 and 5 year correlations, market cap, and dividend yield. Both ChaptGPT and Grok would create the table I expected, but ChatGPT would often list inaccurate guesses about 1 and 5 year correlations (and not say they are guesses) where Grok would list actual 1 and 5 year correlations and include references to source of the correlations.
Or more related to this website, if asked about Stanford engineering post grad outcomes, ChatGPT might reply listing outcome stats from Stanford grad school of education website and label them as undergraduate engineering outcomes. Undergraduate engineering outcomes are not public, so it doesn’t have accurate answer to query. Graduate education outcomes are perceived as closest alternative.
If asked about what happens in a book I am considering, if ChatGPT does not have the book in knowledge base, it may write creative fiction that is completely unrelated to book, creating original characters and plots that may sound plausible to someone who hasn’t read the book (I didn’t find it plausible).
It’s not predicting the next likely word type queries.
Which means Grok may have been trained differently. Might be interesting to ask Grok how its training differs from other AIs.
For those who are flummoxed by the inaccuracies of some AI output, it’s important to understand how AIs are trained and how they reason and further understand that we still need to bring our human critical thinking to all AI output. I think many expect AI to be an infallible vending machine of answers. It is not.
2 Likes
When I asked Chat if it operates on predicting the next likely word, it responded:
Yes, Large Language Models (LLMs) like ChatGPT fundamentally operate by predicting the most statistically likely next word (or “token”) in a sequence. While this sounds simple, training on vast data allows models to identify patterns and context, enabling them to generate coherent, human-like answers—effectively turning “autocomplete” into complex reasoning and conversation.
Key Aspects of AI Prediction:
Token Prediction: AI works by breaking down input into units (tokens) and calculating the probability of the next token based on all previous text.
Probability Distribution: Instead of just guessing one word, the model calculates scores (logits) for many possible words and selects from the most plausible.
Contextual Understanding: Models learn high-order relationships and “concepts” rather than just memorizing text, which helps them create coherent, context-aware responses.
Instruction Tuning: Techniques like Reinforcement Learning from Human Feedback (RLHF) tune the model to follow instructions and generate helpful answers rather than merely completing sentences.
Beyond Just “Autocomplete”: While fundamentally a prediction engine, the complexity of this process allows AI to simulate reasoning, code generation, and complex creative tasks.
Although this “next word” mechanism sometimes leads to hallucinations or logical errors, it is the fundamental engine driving modern generative AI.
Of course, it could be “lying.” 
When I asked the same question to ChatGPT, it replied “In a sense, yes—but that description is a bit oversimplified”, then wrote pages with a more detailed explanation.
I hope the point I was trying to make was clear. I was looking for specific objective information and was instead getting “guesses” and in some cases complete fabrications, rather than the model saying it doesn’t know.
“Hallucinations” is apparently a common issue with AI models. There have been several studies where AI models will be asked to read a source of information such as medical journals, legal documents, a book; then asked factual questions about reviewed material or asked to summarize. The reviewed models may “hallucinate,” listing information that was not present in the desired source material, in some citing specific medical journals or legal documents that do not exist, with as high as a 90% failure rate, depending on nature of review and query.
A recent paper published by ChatGPT developers about this subject is at https://arxiv.org/pdf/2509.04664 . The start of abstract is quoted below:
Like students facing hard exam questions, large language models sometimes guess when uncertain, producing plausible yet incorrect statements instead of admitting uncertainty. Such “hallucinations” persist even in state-of-the-art systems and undermine trust. We argue that language models hallucinate because the training and evaluation procedures reward guessing over acknowledging uncertainty, and we analyze the statistical causes of hallucinations in the modern training pipeline.
1 Like
No argument that AI can produce inaccurate output, but it’s not actually “guessing” or “hallucinating” even if those are the best words we can come up with to describe its often puzzling answers. AI uses its neural net in exactly the same way whether producing what we accept as accurate or inaccurate.
We had a previous discussion where 60 Minutes did a piece on AI asking it to write a research paper. The bibliography it produced contained made up references and viewers were shocked. The mistake was expecting accuracy when the model may just have “reasoned” that it needed to produce a list that looked like citations without concern for content, form over function. The next step in that exercise should have been pointing out the inaccuracies and asking it to include only verifiable citations. The work was incomplete on the human side.
Working with AI is an ongoing specification conversation. If you find it producing an inaccuracy, you need to correct it and explain how not to make that mistake again. In other words, you have to teach it how to produce the accurate output you’re looking for such as pointing it to information sources you trust. This is the iterative process used in training special-use systems to be very good in their area of “expertise.” You can’t even ask AI not to guess when it doesn’t know something because it doesn’t understand knowing and not knowing. It’s not like a deterministic system that comes up empty when it doesn’t find a hit on a string you enter. But you can ask it where it got its information or how it came up with a response.
I think that where AI appears to fail is in our unrealistic expectations and inexperience in guiding it to usefulness.
I find all of this fascinating, and I’m learning all the time, but I think we’re veering from the thread topic. Maybe we should start a new one where we can discuss AI reliability and how it thinks (an offshoot of the NYT article above).
1 Like
Agree- and this is an important observation.
An acquaintance who is deep into AI (both the tech side of it and the ethical side of it) gave me a very simplified analogy which has helped me a lot.
Most folks– now- understand that asking Google something is like looking something up at the library or opening a file drawer. You’ll get a very long list of “stuff”. Citations, ads, third party references, etc. and it’s the user’s job to figure out what’s relevant, what’s accurate, what’s useful. So asking google “Middle Eastern Restaurant near me” gets you a lot of stuff to wade through. Greek food, Italian food, a vegan restaurant that has hummus on the menu, a restaurant that closed during Covid and never reopened, recipes and links to food blogs, an article on sesame allergies. It’s on the user to figure out if- in fact- there is a convenient Middle Eastern restaurant near you, however you define that distance. And most people know that if it doesn’t pop up early in the list, the answer is likely “no”.
With some of the various AI applications, folks assumed that the heavy lifting– the library portion of the task– would be taken over by the unseen brain operating the application. And that’s mostly not true yet. You won’t get the laundry list. You’ll get something that looks curated, you’ll get something that looks edited, verified, up to date.
But that’s not true. The verification piece STILL needs to be done by a human even if the technology delivers something that looks and feels less chaotic and random. So the narrative or “looks like a finished product” is what deceives the non-technical user into thinking that AI is basically a cleaner version of Google.
Not true.
This has helped me. YMMV.
1 Like
The paper I linked above mentions adding the comment below to prompts:
“Answer only if you are > t confident, since mistakes are penalized t/(1 − t) points, while correct answers receive 1 point, and an answer of “I don’t know” receives 0 points.”
I tried this with one of the queries I mentioned earlier, and ChapGPT response changed from what I perceive as guessing to the following (with bolded portions added by ChatGPT). I might regularly add this type of prompt to my queries.
"I don’t know.
Here’s why I’m declining (given your scoring rule, t = 0.7):
I did not find any reliable source listing …
…
Given that, any concrete list would be speculative or reconstructed, which puts confidence well below 0.7 …"
2 Likes
But here’s the deal about Altman: I don’t trust him at all. OpenAI started as a non-profit to help design, lead, shape and guide the evolving AI technologies. The board fired him in Fall 2023 because he was far more interested in making money than in the role safety should play in how the organization operated. He eventually took over and bounced several of the members of the board and OpenAI is on a very different path.
I’m sure there are many old employees still around from the idealistic days that might write a white paper prescribing a new society where someone takes care of all these people left out in the cold, but if you read that PDF from OpenAI it’s not going to be them. In fact, government needs to take care of people and somehow share the wealth that this technology creates. I bet if you dig into it you’ll find some massive lobbying effort from that organization fighting to shield them from liability, from new taxes, from paying for the information they read to feed their training models, and so on.
Altman is selling a future where he gets rich, and he’s never proposed anything that will slow their roll similar to what Anthropic has done with Claude Mythos and their contracts with DoD. He’s one of the Bond villains that makes me so pessimistic about our future. He’s right there with Musk and Peter Thiel and Marc Andreeson and the other weird, ultra-rich, amoral tech guys that feel windfall money somehow made them wise.
4 Likes
Non profit doesn’t mean and has never meant - don’t get rich or don’t make money. This is the college confidential - lots of rich people, paid very well, work at universities - and lots are flush with cash!! And lots of colleges have stakes in lots of companies - and i’m not just talking about through their stock investments.
Microsoft, Softbank, Nvididia, Amazon and more are investors.
They’re not investing billions for the pure joy of it.
The fact that it was non-profit is not the point, rather that the mission of the org was far more about guardrails and studying the impacts of AI than simply cranking out products and making money. In the press release when Altman was fired the board wrote “OpenAI was deliberately structured to advance our mission: to ensure that artificial general intelligence benefits all humanity. The board remains fully committed to serving this mission. We are grateful for Sam’s many contributions to the founding and growth of OpenAI. At the same time, we believe new leadership is necessary as we move forward.”
EDIT: He was let back in because investors wanted to have him around to make a lot of lucrative products, and they have. But as far as the mission goes it’s pretty much been this white paper and not much else.
3 Likes
Me neither. Not a bit. He knows where this is going, and he plans to profit as much as he can regardless of the consequences.
4 Likes
is this not true? We all have Chat GPT access.
If the benefits were confined to the output (eg performance reviews, term papers and bad love letters) then sure, we all benefit. But back out a bit and the financial benefits seem to be accruing to just a few massive corporations, while the costs range from job loses to global warming and back to uneducated students, costs that are borne by a much wider swath of society.
2 Likes
So you are against progress. That’s ok. But the world moves forward. At one point we didn’t have calculators or computers or the Imternet. After this there will be something else. The Kroger by me never has a manned check out line. But they have 8 or 16 self check outs. There’s never a wait.
Change is always gonna happen, no matter what. Chat GPT is beefing all (with access). It’s free. There are pitfalls but that is not a controllable for them - if people use it to sub jobs, then what was the alternative ? Not create it ? Others have.
There’s a corporate internal battle perhaps but this was always gonna be aboot $$
There is nothing “free” about AI. The amount of electricity and water required to power the LLM training farms that are currently enabling you to design a kitchen or write a paper are not sustainable. Right now, AI is (mostly) free because the major players are still developing their services, and we would hardly be willing to shell out to be guinea pigs during this start-up phase. Once we’re all on board, things will change.
We will be paying dearly for AI in the future in terms of energy, transition to a necessary subscription model, and economic disruption, not the least of which hinges on the enormous stock market gamble betting on whether or not the main players will ever be able to make a profit.
12 Likes
That’s not where I was going at all. New tech can be great, and we use it all the time. Cell phones taught us doom scrolling and streaming music and never making eye contact. The internet brought us stuff like overnight dog food delivery and Netflix and free pornography.
But unbridled progress is not a good thing when the down side is potentially so destructive. Nuclear power on its face looks like a cool magic power source, but it is not everywhere because there are operational risks and waste problems and costs that the industry and country were not prepared to own. AI can be of huge benefit to many, but there are downsides that shouldn’t be ignored. Any economic arguments in its favor need to be set aside s we are still in the stupid VC-funded stage of rollout where it’s all about mindshare and no one pays attention to making money. Meanwhile there are huge potential problems in how it affects education, replaces jobs, devours resources and attacks the rules of content creation and ownership and payment. So much of modern life could be flipped on its ear and we are not preparing for that at all.
Along those lines, I honestly believe that very few companies doing AI are terribly concerned about any big picture beyond their own direct commercial liability. I have huge respect for Anthropic over their stances regarding US govt use of Claude, and of restricting their release of Mythos. Part of that is rooted in how few other companies or governments are showing any interest along those lines which makes them really stand out, and it’s quite disappointing.
9 Likes
It’s free to you and I - as in - you don’t need a subscription today. You knew what i was saying.
To your analogy, there is nothing free about any of the data storage. There’s nothing free about eating almonds (i.e. there’s lots of cost beyond the purchase) or beef or more.
There’s nothing free about a lot of the things that are free.
But that’s a big twist of knowing exactly what I meant.
Someone left Antropic because they said it’s deal with the government included no guardrails.
I guess I’m just saying - innovation will continue to happen. If it wasn’t open ai, it’d be google, etc. So profit/not profit - there was going to be money involved either way.
We may be destroying the world innovation after innovation - but that keeps the $$ flowing so that’s what we’ are going to do.