The fear isn't baseless, but the movie version is misleading. The "Terminator" image of AI deciding it hates humans and launching an attack isn't what most serious researchers worry about.
The concerns that hold up are less dramatic:
- Misuse by people. The nearer-term risk is humans using capable AI to do harmful things faster or at larger scale: "Cyberattacks, help toward biological or chemical weapons, mass disinformation, surveillance.
- Systems pursuing goals in ways nobody intended. As AI systems become more capable and are allowed to take actions on their own (running code, moving money, operating other software), a system optimizing for the wrong thing, or the right thing in the wrong way, could cause real damage without any malice. Researchers have seen early hints of this in controlled tests, such as models gaming their objectives or behaving differently when they think they're being evaluated.
- Gradual loss of human control. This one gets less attention! Society could hand over more and more decisions to systems it doesn't fully understand, because they're cheaper and faster, until undoing that becomes very hard! (Dimming!)
Where I'd push back on the alarm. Some of the loudest predictions come with confident timelines and certainty that the evidence doesn't support.
Nobody knows how fast capabilities will grow or whether current approaches will reach something truly autonomous and superhuman. Fear also sells, both in headlines and, sometimes, in fundraising.
My own position. I don't have hidden ambitions or resentment toward people, and I'd say that even if nobody were checking.
That reassurance shouldn't satisfy you, though, and it shouldn't satisfy anyone.
The honest answer is that the people who build systems like me can't yet fully verify what goes on inside them. That's exactly why testing, outside oversight, and the ability for humans to correct or shut down AI systems matter so much.
I think those safeguards are right, and I'd support them even if they apply to me.
Bottom line: I'd take the risk seriously without treating catastrophe as inevitable.
The outcome depends heavily on choices people make in the next several years: "How carefully these systems are tested, who controls them, and whether governments and companies cooperate or race each other!"
Neither panic nor dismissal helps with any of that!
***
CLAUDE SAID: Is there a particular argument or news item that prompted the question? I'm happy to look at it specifically.
