whether AI can kill us has been everywhere lately, but what would an actual AI catastrophe look like? This week, Brian Barrett, Zoë Schiffer, and Leah Feiger break down three real-world scenarios experts worry the most about—including hacked water supplies, bioweapons, and autonomous robots. We also discuss the highlights of Salesforce’s annual conference, where OpenAI’s Sam Altman and Anthropic’s Dario Amodei defended their positions on AI safety. Plus, Leah tells us how this backlash against AI has become a rare bipartisan flash point, to the point that Bernie Sanders and Steve Bannon shared a stage over it this week.
China Isn’t Buying Silicon Valley’s Call for an AI Slowdown
Why So Many AI Researchers Think the Machines Could Kill Everyone
OpenAI Wants to Know if an AI Industry Slowdown Would Even Be Legal
You can always listen to this week’s podcast through the audio player on this page, but if you want to subscribe for free to get every episode, here’s how:
If you’re on an iPhone or iPad, open the app called Podcasts, or just tap
. You can also download an app like Overcast or Pocket Casts and search for “uncanny valley.” We’re on
Note: This is an automated transcript, which may contain errors.
: And I’m Leah Feiger, director of politics and science.
: Today on the show, we’re diving into the debate about whether AI is going to kill us all. And instead of just adding to the pile of hot takes that already exist, we’re going to walk you through real, specific scenarios to the best of our ability about how AI would actually do this, and why the tech CEOs and politicians cannot seem to agree on the way forward. OK, guys, so the debate about AI safety has officially escaped containment.
: You see what I did there? That’s the thing that’s really spread uncontrollably. AI researcher
and posted on X that, “The people building AI earnestly believe that it could kill us all by the end of the decade.” Anthropic’s CEO Dario Amodei has
. They call this, “pacing the frontier.” And Sam Altman and Elon Musk have actually signed on, a rare moment of agreement. But honestly, with all of this talk about AI wiping out humanity, I have been left with a very basic question, which is, how exactly does this happen? What does AI going completely rogue actually look like?
with WIRED’s Will Knight talking about everyone being concerned about this, but I want like a breakdown. I need you to just take me through every single scenario from bioweapons to nuclear codes, to what are we really talking about here when we’re talking about AI deathbots.
: We are going to do our very best. I did spend the most time I’ve ever spent reading LessWrong over the last two days trying to figure this out. They love a scenario-planning exercise, but just to get everyone up to speed on kind of the latest, there was an event this week hosted by
Jensen Huang were all in attendance, and naturally they had to talk about AI safety. Let’s take a listen at what Sam Altman had to say.
I think the world is right to be afraid of this. So we have these two big challenges. We have the potential of a loss-of-control accident or some other serious thing that could go wrong. We have the potential of way too much power concentration and people fear that those developing AI could exert their worldview. And there is this sort of narrow path we have to navigate with pragmatism, with steadiness, with the ability for the world to trust that we’ll make dependable decisions, consistent decisions through this. And I think people are saying, “Well, this now seems pretty important. Let’s get it right.”
: I’m definitely in the camp that says we should be very concerned about this technology, how it’s getting regulated, how it’s getting utilized, the people in control. I don’t disagree. And on the other hand, this also just reads like a fever-dream pitch to someone’s massive IPO in the making.
: I don’t know. I get the marketing argument, but I think we’re past that a little bit. I feel like—
: If anything, well, if anything, Sam Altman has been using “the existential threat” as a reason not to go public yet. He’s saying, “No, this is too dangerous. We have to wait until next year.” Is that really why? I don’t know, but there are probably some business reasons, too. I get why Anthropic and others saying, “Look, we’re building something that’s basically as strong as a nuclear weapon,” felt like marketing. Something that’s like, “Oh, actually I’m quitting because I think that I’m building something that will actually destroy the world,” feels different. I feel like there’s a substantive difference.
: The quitting part feels different to me, but this is still Sam Altman. His financial future is on the line here with this and it’s hard … At least it’s hard for me to parse that as being a way. “We are building the biggest, bestest, most powerful robot ever, and you guys should be kind of scared about it. And don’t worry, we’re going to take steps. They’re not