Aza Raskin, a cofounder of the Center for Humane Technology, has spent years sounding the alarm about where the race to build powerful AI is taking us. Just days after the center’s other cofounder visited the Vatican, Raskin unpacks the significance of Pope Leo XIV’s sweeping new encyclical on artificial intelligence, and exposes the incentive structures pushing Silicon Valley toward dangerous territory—while making the case that it’s not too late to change course.
This is an abridged transcript of an interview from Rapid Response, hosted by the former editor-in-chief of Fast Company Bob Safian. From the team behind the Masters of Scale podcast, Rapid Response features candid conversations with today’s top business leaders navigating real-time challenges. Subscribe to Rapid Response wherever you get your podcasts to ensure you never miss an episode.
I have to start with the pope. Pope Leo recently released an AI pronouncement called Magnifica Humanitas, a title that sort of has echoes of humane technology. Did you talk with the Vatican at all about this encyclical?
We’ve been working behind the scenes, certainly talking with the Vatican. The week that the encyclical came out, Tristan Harris, my cofounder, was actually at the Vatican. And every time we’ve interacted with the Vatican, what we’ve discovered is that even though we come from obviously very different backgrounds, there’s something that’s preserved around how we both view life as sacred and being human as sacred, and that the current technological overreach into our humanity is threatening. And that’s not just AI. That’s social media. That’s the internet. There’s been a long string of technologies that has been encroaching on our humanity that we now have to fight to preserve.
It’s hard to know these days, and we’re still in the early days of AI: Is artificial intelligence an inherently inhuman technology? Can it be humane?
Well, it’s a great question, but fundamentally, if you do not face your demons, they raise your children. The question is not whether AI is good or bad, but whether the incentives governing the race to deploy AI are good or bad. Recently, Sam Altman was asked, “What about all the energy that it takes to train AI?” And do you know his response? He asked, “Do you know how many resources it takes to train a human intelligence, all the food and the energy and the water that goes into those 20 years?” And what he’s implicitly asking is, “Who deserves these scarce resources more? AI, which is about to give your country double-digit GDP growth and all technological and military and medical advances? Or humans, who are sort of flubbing around?” Just like we were able to predict the future with social media by understanding that a race to the bottom of the brain stem, just a knife fight for human attention, would obviously lead to a more polarized and hyperpartisan and outraged and sexualized population.
The race for AI is going to lead to an antihuman future because it sets up a race where humans always lose.
The public mood about AI here in the United States certainly has shifted from being sort of euphoric and open to being decidedly wary, but that doesn’t seem to be slowing things down very much.
No. Actually, I think of this as similar to COVID, when there was a split reality that Americans were living. On the one hand, the stock market was higher than it had ever been. And on the other hand, everyday people were really struggling to make ends meet. And that’s what we’re seeing again here. This is almost like capital lifting off from labor completely. AI is sort of like the full automation of capital just reinvesting into capital. People think, “Oh, AI is just this blinking cursor. I go to ChatGPT, I go to Claude, I type something, and it gives me something back.” But now AI can run in a loop. It can have all the powers that a corporation does, all the money that a corporation does. Is its intent to help you and your family flourish and have a livelihood? Or is the intent to follow market incentives, to dominate, to extract as much as possible?
Well, obviously it’s the second one, and that’s how you know this is an antihuman future, and why I think people are starting to wake up to the fact that this is not in their best interest. The most recent poll that I read is that if you ask what percentage of Americans think that fully unregulated, go-as-fast-as-possible AI is a good thing, that we should be doing that, which is what’s happening, it’s 5%. Only 5% of people actually think that. So how we’re progressing AI is already decidedly not popular.
My colleague Reid Hoffman, who you know.
Very well, yeah.
He leans into the possible with AI, has a podcast with that name. Why not be optimistic that our good side will win out?
Well, just to say, I’m also a builder, right? I founded this nonprofit, the Earth Species Project, building frontier AI to translate animal language. And so we’re making breakthroughs now in understanding the languages of crows. It turns out 70% of crow communication was unknown to science until we started analyzing it. So I want everyone who’s listening to hear that I’m not anti-AI. Actually, I love getting to use this technology. It’s just that I’ve been through a couple waves of technology and know that we always get distracted by the possible of technology, and we don’t want to think about the probable of technology.
How instructive is social media’s evolution to the risks of AI?
Social media is a great example because social media is essentially a baby AI. Where is AI in social media? It’s the thing that’s deciding which news feeds hit your eyeballs. And it’s a very baby AI. It can’t even make its own content. All it can do is rearrange human content. But the question to ask is, was it actually optimizing for human flourishing and connection and understanding, or was it optimizing for engagement and reactivity? Well, it’s the second one.
In the beginning, the feel of social media was, “Oh, we’re all going to be connected. It’s going to help the Arab Spring.” All of these things.
That’s right.
And then it became something a little different.
Exactly. And so that was the possible of the technology. Often, at the very beginning of a technology, it’s not yet captured by market incentives. So you get this beautiful glimpse of a future, and then it gets captured by the incentives. Pretty much everything we predicted starting in 2013 and 2014 has come true. We are now forced to live in a world that didn’t reckon with the race to the bottom of the brain stem modifying everything about our world, from politicians having to become performers to the most extreme voices getting amplified to the most depressed and anxious generation. This is not the world that I want to live in. And I think there’s a version of technology that can be liberated if we can clearly see the probable.
And as tech is industrialized, though, this kind of disappointment is inevitable.
I don’t think it’s inevitable, but it is certainly the 95% to 99% case. And I am very inspired by the film The Day After. I think Tristan and I both are. This film came out in 1982. It was the most watched television event in world history. And it painted a very visceral picture of what happens the day after global nuclear war. It was seen by 100 million Americans. Reagan watched it. He became depressed. He says in his biography that it sort of created a shared common knowledge where everyone knew that everyone else knew what would really happen. And it created the space for the Reykjavik accords and the beginning of deep de-proliferation for nuclear weapons. And this is after, mind you, Oppenheimer in 1962 said, “It’s too late. We’ve already started proliferation. Every country is going to get nuclear weapons. We’re going to blow ourselves up.”
And so it’s really important because whenever we say that something is inevitable, it’s like casting a spell. When you say it’s inevitable, it means there’s nothing to do, which means no one does anything. And so it becomes true. We have to be crystal clear on the difference between it’s very, very hard and it’s impossible. And everyone who says it’s inevitable, we’re just in this industrialized race and it’s going to turn out this way, well, the question to ask is, have we even tried?