I always thought the prepper dudes were a little bit nuts, and maybe they were hoping for the end of the world so they could live life with their multiple wives in a culture that rewards machismo.
But. Maybe I was wrong about them, and they are the smart ones, being prepared. When you read about all these AI CEO’s talking about the possibility of world ending scenarios while building their massive compounds to hide out in, it is concerning. I’m starting to feel like we’re living in a double feature movie of “Idiocracy” and “Don’t Look Up”.
I know people have felt terrified and threatened throughout eternity, sometimes justified and sometimes not, but this feels different, like it’s totally out of anyone’s control at this point.
Are you doing anything different now? Building a compound? Getting a “Go Bag” ready? Stockpiling supplies? Giving up? I feel like I’ve spent the last 40 years saving for retirement and trying to stay healthy, but if Lucifer’s Hammer drops, that means nothing.
That would definitely be a good option if I had a wine cellar. During Covid times, I’d look around at people’s carts piled up with toilet paper, while mine was piled up with red wine and coffee. Gotta have your priorities!
TBH, if the proverbial s**t hits the fan, I don’t want to be one of the few left. All I can envision is a bleak world, and I’m too old to want to live in a nuclear winter. I don’t mean to be glib, though … I do find it all scary beyond belief. At the same time, this is all just so completely out of my control … and as a control freak, I just can’t dwell on it or I’ll go nuts. I don’t drink a lot these days, but if it all goes haywire, I’ll be slamming back the alcohol.
To be clear, you appear to be expressing concerns regarding the hypothetical rise of of Artificial Super Intelligence (ASI), rather than of AI tools as they currently exist.
I would agree with that, however, I have no idea of what AI is capable of right now. I love using AI tools for my meager searches, but the technology seems to be ratcheting up so fast, can they keep control of it?
The exponential transformation will occur with recursive improvement — when AI designs Ai.
For the time being, I regard this as one of the more interesting questions that humankind must consider. (And the question becomes even more interesting if considered in the context of the Fermi Paradox.) Regarding the risk of ASI as it’s commonly discussed, consider both capability and agency. That is, in a future scenario (including in a near-future scenario), could ASI kill us, and would it be in its interest to do so?
Among doomsday end of world type scenarios where I’d want to have a bunker, AI would be fairly low on that list. I’d be more concerned about things like a nuclear war, pandemic / engineered pathogen, human violence following severe crime or societal collapse (cyber attack, financial collapse / hyperinflation, …), etc. I consider all of these extremely unlikely… too remote for the potential benefit to be worth the potential cost. However, if I had $100 billion+ like some who have built compounds, the potential benefit might exceed the drop in the bucket compared to NW cost, leading to a different conclusion.
I’m not really worried about AI taking us out. I’m more concerned that it will - in the long term - eliminate a lot of jobs that won’t be easily replaced. If I’m wrong and AI goes terminator on us I’ve got no interest in living in a dystopian hellscape so no prepping for me.
For the curious, an engineered pathogen, for example, is one of the many imagined risks of ASI, either through a bad human actor or government or through, in the most speculative scenarios, ASI self-generated agency.
This topic isn’t inherently political and is likely of broad general interest to the community. I’m therefore moving this thread out of the Political Forum and into the Cafe.
(Some posts have been edited to comply with the main forum rules)
At the moment, we’re probably around the stage of Artificial General Intelligence (AGI). This is where computers approximate the capabilities of individual humans. They’re much better than average humans at some tasks, much worse at others, much like with human variability. Along with this, AGI level technology can successfully impersonate humans, so the Turing Test may have been passed.
Artificial Super Intelligence (ASI) would be reached when computer technology exceeds the totatality of human capability across essentially all individual humans and across essentially all intellectual endeavors. Then it may do so at some modest multiple of collective human intelligence. Then it may do so at some extraordinary multiple of collective human intelligence. It’s this latter scenario that most stretches the imagination. What would an entity 1,000 or 1,000,000 times smarter than the smartest human do in the real world? Could it be aligned with human values? Should it be feared? Will this happen? When will this happen? The answers to these questions are not easily forthcoming, but the questions themselves, at least, should be of interest.
