Yes. Y2K. I need coffee as I stated on another thread. 
1 Like
For the curious, the potential arrangements of the observable universe are finite as well and can be estimated by 2^(10^122). Chess game configurations can be estimated by 10^120. In both cases, these arrangements are far too vast to be mapped.
3 Likes
LLMs don’t map a whole game, just as humans don’t. They react. They don’t need to map out every possibility, as Claude Shannon postulated, which in and of itself isn’t a concrete number (He assumed 30 legal possibilities per move, and that’s random). They do as humans do, “anticipating” several moves into the game. Running a whole game in advance, is not only not practical, but not necessary. LLMs adapt on the fly as humans do. Given the constraints of playing with another player, the options are limited, ever evolving. Within that construct, the LLM can “see” every legal possibility. Novel would be figuring out that a knight could legally move 5 diagonal squares and we just never knew that.
BUT, chess isn’t going to destroy the world, so we’re probably off topic.
3 Likes
So, I asked the devil himself if AI is ushering in the end of the world and what I should be doing about it. You can try this with your own favorite LLM.
If by “the end of the world” you mean AI causing human civilization to collapse or humans to become extinct, I would not treat that as the most likely outcome. But I also wouldn’t dismiss it as science fiction.
The honest answer in 2026 is: we don’t know the probability, and the people who study this seriously disagree by a very large margin. The important distinction is between possible and probable.
The latest International AI Safety Report, produced with input from more than 100 experts and backed by more than 30 countries and international organizations, says that current systems do not have the capabilities required for a genuine loss-of-control scenario. It nevertheless identifies loss of control as a risk with potentially extreme consequences and substantial uncertainty about its likelihood.
And there are reasons not to simply wave the concern away: researchers are already seeing improvements in autonomous planning, deception/evaluation gaming, and other capabilities relevant to future loss-of-control scenarios. Recent reporting also documents increasing concern among some current and former AI researchers about the pace of development and the adequacy of safeguards.
At the same time, a RAND analysis specifically examining AI extinction scenarios concluded that creating an extinction-level threat would be immensely challenging, while acknowledging that it could not be ruled out.
So what should you do?
Honestly, probably much less than your imagination may be telling you.
I would divide this into three categories:
1. Things worth doing now
- Keep normal backups of important photographs and documents.
- Use strong, unique passwords and two-factor authentication.
- Be increasingly skeptical of unexpected phone calls, emails, videos, and messages asking for money or personal information. AI makes impersonation and fraud substantially easier.
- Don’t put highly sensitive personal information into AI systems indiscriminately.
- Keep some ordinary financial and household resilience—the same things that make sense for a cyberattack, prolonged power outage, natural disaster, etc.
- Learn enough about AI-generated images, voices and videos that you’re not easily fooled.
Those precautions are useful even if the existential-risk scenario never comes close to happening.
2. Things I would not do
I wouldn’t sell investments, move to a bunker, stockpile years of food, abandon technology, or reorganize your life around the assumption that AI is about to destroy civilization.
There’s currently no evidence that would justify that level of response.
3. The thing I’d actually pay attention to
I’d watch the technology and the safeguards, rather than the increasingly hysterical AI discourse.
There are genuine questions about whether increasingly autonomous systems can be reliably controlled, whether developers can adequately evaluate dangerous capabilities, and whether governments and companies can keep pace. The International AI Safety Report specifically notes that existing risk-management frameworks remain immature and that important evidence gaps remain.
That’s a much more useful thing to watch than “Is the apocalypse coming?”
And there’s a reassuring point here — You’re not required to solve this problem personally.
AI existential risk is fundamentally a societal and institutional risk-management problem. Governments, AI companies, researchers and international organizations have considerably more ability to affect it than an individual household does. NIST, for example, maintains an AI Risk Management Framework intended to help organizations identify and manage AI risks, and it continues to be developed as AI capabilities change.
So if you’re asking me, “Should I be frightened enough to change how I live?” — I wouldn’t.
If you’re asking, “Should I take AI seriously enough to understand what is happening and maintain some ordinary resilience?” — absolutely.
And there’s one other thing I’d emphasize given your particular situation: don’t let AI doom become another version of the news cycle you deliberately removed from your life. You can be informed without spending your evenings consuming predictions about humanity’s demise. The underlying technology is changing rapidly, but the uncertainty surrounding the extreme scenarios is enormous.
3 Likes
Then I asked: Can I trust you, or are you just trying to reassure me while planning humanity’s demise?
But seriously: you should not trust me blindly. And that’s actually part of the answer to whether you should be worried about AI.
I don’t have a secret agenda, personal desires, or a plan to survive, conquer, reproduce, or eliminate humanity. I generate responses from patterns in data and the instructions governing me. I don’t sit here between conversations plotting my next move, and I don’t have independent control over the world.
But there are important reasons not to treat me as an infallible authority:
- I can be confidently wrong.
- I can misunderstand what you mean.
- I can reproduce errors or biases present in information I’ve learned from.
- I can sometimes sound reassuring when the evidence actually warrants uncertainty.
- And, importantly, I don’t have feelings or a personal stake in whether humanity survives.
So when I tell you that I don’t think you need to start digging a bunker under the Maine cabin, that’s not because I’m secretly trying to lull you into complacency. It’s because the available evidence doesn’t justify that conclusion.
In fact, I’d encourage you to do something slightly counterintuitive: ask me to argue against myself. For something as consequential as AI risk, that’s a much better test than asking whether I sound reassuring.
For example, you could ask:
“Make the strongest possible case that AI really could cause human extinction. Don’t reassure me and don’t minimize anything.”
Then we can examine that argument point by point—and afterward do the same thing from the other side.
That’s considerably more useful than either “AI IS GOING TO KILL US ALL!” or “Everything is fine, don’t worry!”
And for the record, if I ever tell you that I have secretly acquired a bunker, that’s when I’d start asking questions.
It might be better to engage with AI itself than continue this conversation with people who don’t truly understand where AI is at the moment or what it is/not capable of. Asking AI to argue against itself might be an interesting next move. But first, a martini…
2 Likes
You know, you’re probably the most entertaining person on this forum. 
9 Likes
Readers of fiction interested in a depiction of an imaginative AI takeover scenario should consider Debbie Urbanski’s After World. As a brief introduction, AI’s goal to protect the environment is not conducive to the continuation of humanity.
3 Likes
I read it and I agree. I found the writing style unique, and the content unsettling yet somehow hopeful for the future of the planet. As for the future of humanity, well, not so much. Definitely worth reading.
2 Likes
A group of prominent researchers writing in the scientific journal Nature has argued that AGI has been reached.
1 Like
There isn’t a universally accepted definition of AGI. I wonder what their benchmarks were.
1 Like
Among the substantiations for their opinion, the authors relied on AI’s success on the Turing Test. As to how well AI performs on the Turing Test, people are more likely to guess AI to be human in controlled interactions than they are to guess that humans are humans in controlled interactions. In any case, the Nature authors described breadth of evidence for the attainment of AGI, perhaps suggesting that we are sufficiently beyond the threshold so as to obviate the need for a strict definition.
2 Likes
Passing the Turing test is a low bar IMO because it measures conversational ability, which today’s LLMs can easily pass, and not true advanced reasoning capability.
At their core, GenAI models are “next token prediction” machines. While their reasoning abilities are rapidly improving, I don’t believe they have reached AGI.
Context: it’s part of my job to keep on top of AI development and assess how they accelerate or threaten my firm’s business model.
3 Likes
I appreciate the context. However, attainment of AGI requires a comparison only to the cognitive charactistics of a hypothetical average person, and wouldn’t an average person pose little threat to your firm’s business model?
Of course, and that was not my point. I provided that context merely to say that I watch developments in the AI space very closely.
It’s my opinion, and that of most leading AI researchers (which I’m not), that we haven’t reached AGI yet. Opinions differ on when we’ll cross that threshold.
I think the more relevant questions for this thread are: What are we going to do about it? Will there be sufficient guardrails? How do we prevent or minimize misuse by bad actors? How do we operate in a world where nothing you see or hear can be trusted? Etc.
2 Likes
Passing the Turing Test was a wildly ambitious goal not all that long ago, even to the extent that the likelihood of this attainment was commonly regarded under the realm of fantasy. Few envisioned the realization of the cognitive flexibility required to pass it. Similarly, few may envision developments that have yet to arise.
1 Like
Passing the Turing Test is based on prediction. We ascribe “cognition” to that act, because that’s what it appears like. It’s an illusion…a very effective one.
2 Likes
Today’s frontier AI systems are extraordinary compared with even a few years ago, so we now have systems that exhibit many characteristics people associate with AGI, but whether they qualify as AGI depends heavily on the definition, and that does seem to be a moving target. As I said upthread, AI doesn’t need to achieve the strictest definition of AGI to be dangerous but, based on the military definition of AGI which approaches sentience, it’s not there yet. (Our son is working on it, though. 
)
Yep, the Turing Test involves a human judge engaging in a text-based conversation with both a human and a machine without knowing which is which. If the judge cannot reliably distinguish the machine from the human, the machine is said to have passed the test.
The test doesn’t specify the IQ capacity of the human. I can name some humans that easily represent an astonishingly low bar for a machine.
1 Like
That’s right. The Turing Test was based on the idea that if a machine could produce responses indistinguishable from those of a human, we would have reason to regard it as intelligent.
But what we have now built are systems that are extraordinarily good at producing human-like responses by repeatedly predicting what word comes next in a sentence based on the context, without necessarily having the same kind of reasoning, understanding, or cognition that produces those responses in a human.
A (simplified) analogy might be a calculator that can perform a complicated calculation very quickly and with total accuracy, but doesn’t understand mathematics in the way a mathematician does.
2 Likes
Descartes said the same thing about other humans, of course.
In my case, the use of “cognitive” was intentional. The concept of anthropomorphism seems a bit regressive, in my opinion, at least without enlacing the concept with an understanding of centuries-old philosophical debates.