Bayesian Investor Blog

Book review: Red Heart, by Max Harms.

Red Heart resembles in important ways some of the early James Bond movies, but it’s more intellectually sophisticated than that.

It’s both more interesting and more realistic than Crystal Society (the only prior book of Harms’ that I’ve read). It pays careful attention to issues involving AI that are likely to affect the world soon, but mostly prioritizes a good story over serious analysis.

I was expecting to think of Red Heart as science fiction. It turned out to be borderline between science fiction and historical fiction. It’s set in an alternate timeline, but with only small changes from what the world looks like in 2025. The publicly available AIs are probably almost the same as what we’re using today. So it’s hard to tell whether there’s anything meaningfully fictional about this world.

The “science fiction” part of the story consists of a secret AI project that has reportedly advanced due to unusual diligence at applying small, presumably mundane, efficiencies. That’s only a little different from what DeepSeek’s AI sounded like last winter. In order to be fully realistic, it would also need some sort of advance along the lines of continual learning. The book is vague enough here that it might be assuming that other AI projects have implemented some such advance. That only stretches the realism a small amount.

Amazon quite reasonably classifies the book as a political thriller, even though it focuses more on artificial intelligence than on politics in the usual sense.

My biggest complaint is that the story occasionally mentions that the AI is rapidly becoming more capable, yet I didn’t get a clear sense of this speed. There are almost no examples of her trainers being surprised that she succeeded at some new task that had previously looked hard for her. There is no indication of when she crosses any key threshold, except when they give her new permissions.

Maybe much of that is realistic. The sudden capabilities foom of some fictional AIs seems too dramatic to satisfy my desire for realism. But that leaves the reader with confusing signs about the extent to which there’s a race between competing AI projects. The story stretches out over a longer period than I’d expect if they genuinely felt the urgency that their discussions suggest.

I would like to know what kind of evidence is driving the reports of urgency. But I can imagine that realistic versions of the evidence would be too subtle to readily understand. And I wouldn’t have wanted the story to fabricate unrealistically blatant breakthroughs in order to support the sense of urgency.

The story alternates between sometimes portraying the hero as an ordinary person, while at other times he looks like a mild version of James Bond.

He’s sufficiently young and inexperienced that this could have been a coming of age story. But we don’t see him growing. Whatever growth he needed likely happened before the start of the story. The author seems to want to emphasize that there’s a lot of luck needed for the story to have a nice ending. It may be important to hire the best and the brightest to handle an AI project, but the odds will still be lower than we want.

The story’s hero needed to have several key skills, but most of the time he doesn’t look special. It seems mostly like an accident that he ends up imitating James Bond. This approach mostly works, but feels strange. It makes the story a bit more realistic, at a modest cost to the story’s entertainment value.

There’s one minor spot that felt implausible. Near the middle, he thinks that he will be leaving China soon, and his main reaction is to worry about his relationships with minor characters. What, no emotions related to leaving the most important project ever? It’s not like he has an unemotional personality.

The main reason that I read Red Heart is its discussion of AI corrigibility (roughly: obedience), which I consider to be a critical and neglected part of how superhuman AI can be safe.

The story provides a decent depiction of how corrigibility would work if it’s implemented well. But it doesn’t provide enough detail to substitute for reading more rigorous technical writings.

The book’s treatment of multi-principal corrigibility is frustratingly brief but raises crucial questions. If we successfully build corrigible AGI, to whom should it be corrigible? The story gestures at problems with being corrigible to multiple people, but it implies, without much justification, that we might need to give up on the goal of having a large number of people empowered to influence the leading AI.

Red Heart is refreshing and a mostly realistic complement to the excessive gloom of If Anyone Builds It, Everyone Dies.