The AI trade is still a very good bet. Increased political uncertainty makes all investments today somewhat less promising than was the case when I wrote about this topic in July 2025.

Regulation of AI is likely to increase over the next 2 years or so, constraining growth somewhat. But AI capabilities have been increasing faster than I expected. I don’t expect a full halt in AI capability advances, and existing capabilities are enough to fuel quite a bit of further growth in AI-related companies. So it’s unclear whether AI revenues will be higher or lower than what I expected in 2025.

Continue Reading

Book review: Obsolete: The AI Industry’s Trillion Dollar Race to Replace Us?and How to Stop It, by Garrison Lovely.

Obsolete is a strange book. It’s mostly good, but the quality varies a good deal. Lovely is well informed about concerns that AI might kill us all, and doesn’t dispute those concerns. Yet he focuses much more on more ordinary concerns, in particular excessive concentration of power.

He is somewhat successful at portraying mundane harms of this year, and longer term risks (will AI kill us all in the 2030s), as similar problems. I usually cringe at such efforts, but Lovely is quite good at defusing the tensions between those two types of concern. E.g. he discusses AI-assisted political coups by a CEO such as Musk in a way that sounds almost realistic using current AI, while also warning about dramatic increases in that risk as AI becomes more powerful.

Most of the book is written for a pretty broad audience. I’m annoyed that he added in a modest number of gratuitous culture war quips that will repel some Republicans. He mitigates that a little by also complaining about widespread corruption among leading Democrats, but many Republicans won’t read far enough to notice that.

Continue Reading

I sold my small position in Microsoft stock this morning. Not for financial reasons.

I’m reacting to a key element of Microsoft’s draft AI Code of Conduct:

The idea of model welfare is wrong. … It is not conscious and should not be designed to imitate consciousness. It should be engineered to avoid representing as though it has feelings, subjective preferences, or intrinsic motivation. We reject … the idea that models might deserve welfare

I’ll focus mainly on selfish reasons why this is dangerous. Given anything like our current path toward smarter than human AI, I estimate that this approach would increase our risk of doom by at least 5%.

It is likely that AI assistance will have important influences on the personalities of future AI generations.

Continue Reading

[This rant is directed mainly at people who probably won’t read it.]

Proposals to slow AI development have been criticized as special-interest lobbying. E.g. Marc Andreessen:

One-hundred percent, there’s a regulatory capture element to it. And they’re driving hard for that.

I’ll concede for the sake of argument that AI companies would like regulatory capture.

Dario Amodei, Sam Altman, and Elon Musk are all ten times too ambitious to settle for mainly achieving regulatory capture. Regulatory capture is effective at preserving existing corporate revenue and jobs. It’s a terrible way to colonize Mars or cure all disease. Those CEOs have made it clear that their ambitions are at least as big as colonizing Mars.

Continue Reading

The unusually uncertain situation with AI makes this an unusually important time to donate to organizations which are in a position to influence how AI develops.

Here are my guesses as to which organizations can best use more funding. Everything in this area is changing rapidly, so this isn’t a prediction of where money will be needed 6 months from now.

Anima Labs is focused on understanding AIs through talking with them and caring about their welfare. Treating AIs with respect and understanding has important influence on how well AIs cooperate with us. It’s not hard to imagine a scenario where that makes a difference in whether the AIs that build superintelligence care about humans. Anima is underfunded due to a combination of most people neglecting this topic, and due to carelessly missing an SFF deadline.

The Verifiable Compute Foundation works on ways to verify a treaty to pause or pace AI development. See All hands on deck to build the datacenter lie detector for the general idea. It’s hard to tell whether they’re more constrained by funding or by talent.

Palisade Research is the organization I trust most to advise governments on what AI policies to adopt. The US government seems likely to soon adopt AI regulation in a hasty fashion that could easily be full of careless mistakes. Palisade combines a good understanding of AI with a good understanding of how government works. They’re fairly well funded, but more donations today could speed up their hiring plans.

The The AI Policy Network is somewhat like Palisade, but focused more on building influence with key members of Congress. As a 501(c)(4), they’re able to do lobbying that Palisade cannot.

If we can cause AIs to be corrigible, that reduces the risk of mistaken goals getting locked in forever. The Corrigibility Research Fund funds independent researchers who are working on making this reliable.

The AI Risk Mitigation Fund regrants donations to many independent researchers. I use it as a replacement for the Long-Term Future Fund.

Orthogonal is a strange group focused on a more rigorous long-term solution to AI than the others I’ve mentioned. It’s nearly broke, and doesn’t need a big budget – $200k per year would be plenty here, whereas the others that I’ve mentioned could maybe use a couple of million before seeing diminishing returns.

In many cases these endorsements are accidental byproducts of who I know well enough to trust. There are many more that I haven’t found ways to evaluate.

The folks who wrote AI 2027 have written a more optimistic narrative, which focuses more on hopes for good policies than on predictions about what policies we’ll get.

Plan A’s narrative seems halfway between a science fiction story and a proposed treaty. Like most science fiction, I expect it to err in the direction of describing the world as more human-understandable and relatable than what we’ll actually get.

The broad outlines come close to the scenario that I analyzed in Financial Costs of an AI Pause?, which is what I predict that fairly competent governments would do.

AI-2040 adds much more detail than I was able to provide, some of it surprising. The devil is in the details.

I largely endorse their advice. The rest of this post will focus on many small doubts about their advice and their predictions about what that advice would produce.

Continue Reading

Brian Albrecht’s A compute tax is a REALLY dumb idea disputes a claim that I made in Safety Net When AIs Take Our Jobs about a possible need for a tax on GPUs and similar hardware.

Albrecht and I are answering different questions. He asks what’s the optimal tax for normal times. Whereas I ask what to do if optimal taxes are already collecting nearly the maximum feasible revenue, and there’s still a short-term risk of a debt spiral.

Continue Reading

I recently attended a talk at Manifest by Chad Jones on the economic effects of AI. Much of it was sensible. Unlike many economists, he gives careful consideration to AI becoming pretty powerful soon. But his main scenarios predict much slower growth than I expect.

His paper Past Automation and Future A.I.: How Weak Links Tame the Growth Explosion clarifies the parts of his talk that puzzled me. This post explores where our assumptions differ.

The fastest scenario that he considers (figure 6 – The Future if AI = ‘Moore’s Law Everywhere’) has economic growth rising to 13% by 2040. Whereas I expect at least 30% growth by then, due to automation happening earlier than he’s willing to imagine.

The key areas where I disagree with him are beliefs about the extent to which growth will be constrained by weak links, which likely stems from differing beliefs about how general-purpose AI will be.

Continue Reading

Book review: The God Test: Artificial Intelligence and Our Coming Cosmic Reckoning, by Robert Wright.

Some AI doomers talk about AI becoming god-like. Robert Wright goes further, telling us that the world is about to create God, in a sense that he only half-jokingly compares to the Christian version of God.

Wright argues that AI is not comparable to the origin of language or the Cambrian explosion. It is the climax of the process that started with the origin of life. I interpret that as an 11 on Nate Silver’s Technological Richter Scale:

Continue Reading