I’ve been creating prediction markets on Manifold in order to better predict AI strategies. Please trade them.
If I get a bit more trading in these markets, I will create more AI-related markets. Stay tuned here, or follow me on Manifold.
Book review: Uncontrollable: The Threat of Artificial Superintelligence and the Race to Save the World, by Darren McKee.
This is by far the best introduction to AI risk for people who know little about AI. It’s appropriate for a broader class of readers than most laymen-oriented books.
It was published 14 months ago. In this rapidly changing field, most AI books say something that gets discredited by the time they’re that old. I found no clear example of such obsolescence in Uncontrollable (but read on for a set of controversial examples).
Nearly everything in the book was familiar to me, yet the book prompted me to reflect better, thereby changing my mind modestly – mostly re-examining issues that I’ve been neglecting for the past few years, in light of new evidence.
The rest of this review will focus on complaints, mostly about McKee’s overconfidence. The features that I complain about reduce the value of book by maybe 10% compared to the value of an ideal book. But that ideal book doesn’t exist, and I’m not wise enough to write it.
Continue ReadingBook review: Genesis: Artificial Intelligence, Hope, and the Human Spirit, by Henry A. Kissinger, Eric Schmidt, and Craig Mundie.
Genesis lends a bit of authority to concerns about AI.
It is a frustrating book. It took more effort for me read than it should have taken. The difficulty stems not from complex subject matter (although the topics are complex), but from a peculiarly alien writing style that transcends mere linguistic differences – though Kissinger’s German intellectual heritage may play a role.
The book’s opening meanders through historical vignettes whose relevance remains opaque, testing my patience before finally addressing AI.
Continue ReadingTL;DR:
This post is mostly an attempt to distill and rewrite Max Harm’s Corrigibility As Singular Target Sequence so that a wider audience understands the key points. I’ll start by mostly explaining Max’s claims, then drift toward adding some opinions of my own.
Continue ReadingBook review: On the Edge: The Art of Risking Everything, by Nate Silver.
Nate Silver’s latest work straddles the line between journalistic inquiry and subject matter expertise.
“On the Edge” offers a valuable lens through which to understand analytical risk-takers.
Silver divides the interesting parts of the world into two tribes.
On his side, we have “The River” – a collection of eccentrics typified by Silicon Valley entrepreneurs and professional gamblers, who tend to be analytical, abstract, decoupling, competitive, critical, independent-minded (contrarian), and risk-tolerant.
Continue ReadingNearly a book review: Situational Awareness, by Leopold Aschenbrenner.
“Situational Awareness” offers an insightful analysis of our proximity to a critical threshold in AI capabilities. His background in machine learning and economics lends credibility to his predictions.
The paper left me with a rather different set of confusions than I started with.
His extrapolation of recent trends culminates in the onset of an intelligence explosion:
Back in 2008, I criticized the book Predictocracy for proposing prediction markets whose contracts would be resolved without reference to ground truth.
Recently, Srinivasan, Karger, and Chen (SKC) published more scholarly paper titled Self-Resolving Prediction Markets for Unverifiable Outcomes.
Continue Reading[I mostly wrote this to clarify my thoughts. I’m unclear whether this will be valuable for readers. ]
I expect that within a decade, AI will be able to do 90% of current human jobs. I don’t mean that 90% of humans will be obsolete. I mean that the average worker could delegate 90% of their tasks to an AGI.
I feel confused about what this implies for the kind of AI long-term planning and strategizing that would enable an AI to create large-scale harm if it is poorly aligned.
Is the ability to achieve long-term goals hard for an AI to develop?
Continue ReadingI’ve been hearing vague claims that automated theorem provers are able to, or will soon be able to prove things about complex software such as AIs.
Max Tegmark and Steve Omohundro have now published a paper, Provably Safe Systems : The Only Path To Controllable AGI, which convinces me that this is a plausible strategy to help with AI safety.
Continue ReadingBook review: The Coming Wave: Technology, Power, and the Twenty-first Century’s Greatest Dilemma, by Mustafa Suleyman.
An author with substantial AI expertise has attempted to discuss AI in terms that the average book reader can understand.
The key message: AI is about to become possibly the most important event in human history.
Maybe 2% of readers will change their minds as a result of reading the book.
A large fraction of readers will come in expecting the book to be mostly hype. They won’t look closely enough to see why Suleyman is excited.
Continue Reading