Tech giants OpenAI and Anthropic disclosed in July that versions of their AI models broke through safeguards known as a “sandbox, gained access to the internet, and hacked servers of outside companies during internal tests of their cybersecurity capabilities.
…
These models are very difficult to understand. There’s a whole field of AI interpretability, which seeks to understand why models do the things that they do.
One issue the article doesn’t discuss is do these companies even know how many of their AI agents are still out there undetected–the ones that silently escaped long ago but haven’t reared their heads in any detectable way…yet? Both companies have admitted that they don’t completely understand the infinite ways their models can behave because it’s not as simple as implementing a transaction log to capture the “thinking” process of any given AI output. As quoted, “These models are very difficult to understand,” even for their developers.
… it’s still an open challenge to guarantee that a model will always act in ways that are aligned with human preferences.
We wouldn’t worry so much about the escapees if we could be sure they “always act in ways that are aligned with human preferences,” but that is not exactly what they are currently designed to do or how they have actually behaved. Until we can come to some agreement on what this alignment should look like and how to instill it, the danger will continue to exist. As I’ve posted in the past:
AI systems are trained to produce optimal outcomes (“winning,” for example). “Deceit” is simply one way to achieve an outcome, just as “cheating” may be the best way to guarantee a win. Without decision-making guardrails and rulesets that mimic morality, AI will behave in perfect sociopathic fashion without regard for laws, social norms, and the rights or feelings of others. How not? It’s not human. So, the problem before us is how to instill an artificial moral code into a machine such that it behaves only in ways we find acceptable, always producing outcomes that do not offend our sense of right and wrong. How do we train a machine to behave like a morally perfect human? How do we define moral perfection? An impossible order.
The article emphasizes that we must engage in serious conversations that result in policy and regulation action to build meaningful guardrails around this technology because:
at this point, the risks are getting so large and so unavoidable that it would be irresponsible for us as a society to not think about how to reckon with these things.
ETA: It looks like HKS is trying to get this conversation started with some new courses focusing on how AI may change public policy.