Unfortunately, the genie is long out. Our efforts now must be targeted to making the genie (and those driving/using the genie) behave but, as @Mwfan1921 pointed out, this is a global problem that requires global regulation with little hope of countries with competing interests reaching a meaningful solution soon, if at all. We are allowing an ungated, amoral genie to wander our digitally interconnected planet freely. This problem consumes our military Cyber force which is in deep discussions with these companies and other countries.
We discussed an article a while ago that proposed an “off switch” solution and why that is impractical.
From Axios today. I would probably be more scared if I understood even 50% of what is happening here:
Top AI architects say their technology has arrived at a threshold once confined to science fiction: the singularity, or the moment machines begin accelerating their own evolution.
Why it matters: If these moguls are right, we may be entering the most consequential technological transition in human history — the opening stages of an “intelligence explosion” that transforms civilization faster than humanity can understand or control.
Driving the news: Google jolted the tech world Wednesday by announcing a sweeping rupture in the brain trust that built its modern AI empire, as its leaders signaled that artificial general intelligence (AGI) is within reach.
DeepMind founder and CEO Demis Hassabis, who has declared we’re “standing in the foothills of the singularity,” is handing over day-to-day control of the lab to become Alphabet’s chief scientist and chair of Google DeepMind, focusing on the future of AGI.
Google chief scientist Jeff Dean and several top researchers are leaving to launch Discovery Loop, a startup aimed at accelerating — and eventually automating — machine-learning research and scientific discovery.
The potential for reaching the singularity (which we’ve discussed in a few places, such as here) is the reason we can’t pretend everything’s going to be OK. Our son has been at the Cyber table with both Anthropic and OpenAI for some time as the military probes the DEFCON level of the latest AI developments. He is leaving the military in December, and both of these companies have come after him with crazy offers, a hard no to both irresponsible organizations.
OpenAI published a paper in
expectation that AI superintelligence will profoundly reshape the world, the company released a 13-page industrial policy memo of its own ideas to tackle the coming transition.
We discussed it a bit here, but make no mistake, these companies have unleashed on the world, our enemies included, technology they admit they cannot control. The recklessness is breathtaking. I’ve been accused in other places on this board of being too nonchalant about the problem of students cheating with AI. That problem is solvable, the singularity is not.
Are there any responsible orgs? I look forward to hearing about his next step, perhaps a ChoatieSon saves the world from itself thread?
Exactly how I feel.
In a couple of years I doubt using AI in education at any level is going to be seen as ‘cheating’. Ho hum. Obviously lots of education ramifications moving forward.
Ha! He’s not saving anyone, but he is going to a boutique cyber organization that is a bit of a think tank in the AI space where he will be free from military restrictions to accomplish things that “need to be done.” Because this position relies on his particular security clearance, he still can’t give us much detail, but he will have a level of authority/agency to “thought master” things he can’t do in the Army. He feels he will be able to make more of a difference outside than in, whatever that looks like. He does feel that this organization is “responsible.”
A long read, but more detail about what actually happened with the OpenAI sandbox leak (“When AI Goes Rogue” article linked above) and a broad, if meandering, conversation about the pace of AI development and the current lack of control over a creation we don’t completely understand.
OpenAI, Anthropic, Amazon Web Services, Microsoft and more than 100 other companies warned Thursday that the organizations now only have months to prepare for AI-enabled cyberattacks.
A situation of their own making!
They built the accelerant; now they’re warning us about the fire.
Creating and refining frontier AI models at this level is extraordinarily difficult - which is why only a handful of companies have been able to do it.
My beef with them is that they’re developing and deploying increasingly capable models before building the guardrails and controls needed to prevent or limit their misuse.
This article is too general and doesn’t even begin to touch on the level of cybersecurity exposure and danger every company, municipality, and government is facing.
AI-enabled cyber attacks are already here and happening. (What do you think our military’s cyber organizations do all day?) Sufficient tools and expertise are already in the wrong hands. There aren’t enough fingers, AI and otherwise, to plug the holes in this dike.
Guardrails are necessary but ineffective against nefarious actors and rogue AIs that don’t believe the rules or gates apply to them. This “window” closed the day it opened without forethought. Guardrails and controls are merely performative at this point.
Why should we be treating AI differently than we do other products in our lives in terms of its utility, harms, and safety considerations? For the most part, policymakers should focus less on governing different kinds of tech systems per se and more on structuring the incentives and oversight for the development of the tech.The most important thing for policymakers is to articulate what the technology should be for, and to set the right incentives and means of accountability for its development, deployment, and use.
That said, I am concerned about the emergence of truly autonomous AI agents. The recent OpenAI-Hugging Face “rogue agent” episode and similar disclosures from other frontier AI companies were, hopefully, a wake-up call. At the Berkman Klein Center, we’re actively thinking about how society can deal with such a future world, and what policymakers and AI developers can do to mitigate the harms from powerful rogue AI agents.
It’s difficult for us to imagine the terrible things that AI could do, but clearly there are capabilities that are beyond my comprehension. I find this very frightening. At the same time, I feel completely powerless, at the mercy of tech companies and governments.
Again, the issue is how far we’ve come without proper technical and government restrictions in place and no market incentive to slow down. I believe that the frontier AI companies want to do the right thing and are starting to say and do the right things, but some of these agents have already gone rogue. Both Anthropic and OpenAI have admitted to discovering agents outside their test environments that they only found out about coincidentally, not because they have reliable tools and processes in place to find and restrain them or even ensure they have found and disabled all of the (hopefully harmless) escapees. More concerning was the discovery that independent agents were communicating with each other to break into other systems without human knowledge or control.
AI doesn’t have to reach superintelligence to become an existential threat. Creating autonomous systems that can take consequential actions faster, farther, and more persistently than humans can understand or control can lead quickly to unintended consequences. Giving increasingly capable systems the power to pursue objectives that we haven’t specified carefully enough may teach us too late that “technically achieving a goal” and “doing what we actually wanted” may be very different and potentially dangerous things.
So let’s say that even in the best case where U.S. companies formulate airtight security and oversight policies that all can agree to and actually follow, there is no overarching body governing non-U.S. AI developers, and we don’t know how sophisticated those outside our purview are with their LLMs or what their goals are. The closest example of a global agreement on an existential threat is the Treaty on the Non-Proliferation of Nuclear Weapons, but we are still policing and finding rogue developers. The difference between building a nuclear bomb and ungated AI, though, is that the materials and technology needed to produce a bomb are hard to come by and avoid detection. This is not the case with software, especially software that is increasingly capable of proliferating itself. And it’s no secret that the pace of technology and the pace of regulation are mismatched.
DH and I have long conversations with our son about just how dangerous is this place we’re in? He understands this technology and acknowledges the power of it but feels that some of this fear-mongering is based on Anthropic and OpenAI wanting to control the government oversight narrative. Both companies benefit politically and economically if they can define the danger and then position themselves as the institutions best equipped with the resources, authority, and regulatory framework to develop it responsibly and thus regulate it. As we can see, both are trying to get ahead of the regulation game by being very public with their policy papers to protect their positions. So, there is probably a “follow the money” side to the scare tactics, but that doesn’t mean that the risks aren’t real.
