Book Reviews

Superintelligence

Posted by Peter on July 28, 2014
Posted in: Artificial Intelligence, Book Reviews. Tagged: .

Book review: Superintelligence: Paths, Dangers, Strategies, by Nick Bostrom.

This book is substantially more thoughtful than previous books on AGI risk, and substantially better organized than the previous thoughtful writings on the subject.

Bostrom’s discussion of AGI takeoff speed is disappointingly philosophical. Many sources (most recently CFAR) have told me to rely on the outside view to forecast how long something will take. We’ve got lots of weak evidence about the nature of intelligence, how it evolved, and about how various kinds of software improve, providing data for an outside view. Bostrom assigns a vague but implausibly high probability to AI going from human-equivalent to more powerful than humanity as a whole in days, with little thought of this kind of empirical check.

I’ll discuss this more in a separate post which is more about the general AI foom debate than about this book.

Bostrom’s discussion of how takeoff speed influences the chance of a winner-take-all scenario makes it clear that disagreements over takeoff speed are pretty much the only cause of my disagreement with him over the likelihood of a winner-take-all outcome. Other writers aren’t this clear about this. I suspect those who assign substantial probability to a winner-take-all outcome if takeoff is slow will wish he’d analyzed this in more detail.

I’m less optimistic than Bostrom about monitoring AGI progress. He says “it would not be too difficult to identify most capable individuals with a long-standing interest in [AGI] research”. AGI might require enough expertise for that to be true, but if AGI surprises me by only needing modest new insights, I’m concerned by the precedent of Tim Berners-Lee creating a global hypertext system while barely being noticed by the “leading” researchers in that field. Also, the large number of people who mistakenly think they’ve been making progress on AGI may obscure the competent ones.

He seems confused about the long-term trends in AI researcher beliefs about the risks: “The pioneers of artificial intelligence … mostly did not contemplate the possibility of greater-than-human AI” seems implausible; it’s much more likely they expected it but were either overconfident about it producing good results or fatalistic about preventing bad results (“If we’re lucky, they might decide to keep us as pets” – Marvin Minsky, LIFE Nov 20, 1970).

The best parts of the book clarify many issues related to ensuring that an AGI does what we want.

He catalogs more approaches to controlling AGI than I had previously considered, including tripwires, oracles, and genies, and clearly explains many limits to what they can accomplish.

He briefly mentions the risk that the operator of an oracle AI would misuse it for her personal advantage. Why should we have less concern about the designers of other types of AGI giving them goals that favor the designers?

If an oracle AI can’t produce a result that humans can analyze well enough to decide (without trusting the AI) that it’s safe, why would we expect other approaches (e.g. humans writing the equivalent seed AI directly) to be more feasible?

He covers a wide range of ways we can imagine handling AI goals, including strange ideas such as telling an AGI to use the motivations of superintelligences created by other civilizations

He does a very good job of discussing what values we should and shouldn’t install in an AGI: the best decision theory plus a “do what I mean” dynamic, but not a complete morality.

I’m somewhat concerned by his use of “final goal” without careful explanation. People who anthropomorphise goals are likely to confuse at least the first few references to “final goal” as if it worked like a human goal, i.e. something that the AI might want to modify if it conflicted with other goals.

It’s not clear how much of these chapters depend on a winner-take-all scenario. I get the impression that Bostrom doubts we can do much about the risks associated with scenarios where multiple AGIs become superhuman. This seems strange to me. I want people who write about AGI risks to devote more attention to whether we can influence whether multiple AGIs become a singleton, and how they treat lesser intelligences. Designing AGI to reflect values we want seems almost as desirable in scenarios with multiple AGIs as in the winner-take-all scenario (I’m unsure what Bostrom thinks about that). In a world with many AGIs with unfriendly values, what can humans do to bargain for a habitable niche?

He has a chapter on worlds dominated by whole brain emulations (WBE), probably inspired by Robin Hanson’s writings but with more focus on evaluating risks than on predicting the most probable outcomes. Since it looks like we should still expect an em-dominated world to be replaced at some point by AGI(s) that are designed more cleanly and able to self-improve faster, this isn’t really an alternative to the scenarios discussed in the rest of the book.

He treats starting with “familiar and human-like motivations” (in an augmentation route) as an advantage. Judging from our experience with humans who take over large countries, a human-derived intelligence that conquered the world wouldn’t be safe or friendly, although it would be closer to my goals than a smiley-face maximizer. The main advantage I see in a human-derived superintelligence would be a lower risk of it self-improving fast enough for the frontrunner advantage to be large. But that also means it’s more likely to be eclipsed by a design more amenable to self-improvement.

I’m suspicious of the implication (figure 13) that the risks of WBE will be comparable to AGI risks.

  • Is that mainly due to “neuromorphic AI” risks? Bostrom’s description of neuromorphic AI is vague, but my intuition is that human intelligence isn’t flexible enough to easily get the intelligence part of WBE without getting something moderately close to human behavior.
  • Is the risk of uploaded chimp(s) important? I have some concerns there, but Bostrom doesn’t mention it.
  • How about the risks of competitive pressures driving out human traits (discussed more fully/verbosely at Slate Star Codex)? If WBE and AGI happen close enough together in time that we can plausibly influence which comes first, I don’t expect the time between the two to be long enough for that competition to have large effects.
  • The risk that many humans won’t have enough resources to survive? That’s scary, but wouldn’t cause the astronomical waste of extinction.

Also, I don’t accept his assertion that AGI before WBE eliminates the risks of WBE. Some scenarios with multiple independently designed AGIs forming a weakly coordinated singleton (which I consider more likely than Bostrom does) appear to leave the last two risks in that list unresolved.

This books represents progress toward clear thinking about AGI risks, but much more work still needs to be done.

The Hidden Reality

Posted by Peter on June 30, 2014
Posted in: Book Reviews, Life, the Universe, and Everything. Tagged: .

Book review: The Hidden Reality: Parallel Universes and the Deep Laws of the Cosmos, by Brian Greene.

This book has a lot of overlap with Tegmark’s Our Mathematical Universe

Greene uses less provocative language than Tegmark, but makes up for that by suggesting 5 more multiverses than Tegmark (3 of which depend on string theory for credibility, and 2 that Tegmark probably wouldn’t label as multiverses).

I thought about making some snide remarks about string theory being less real than the other multiverses. Then I noticed that what Greene calls the ultimate multiverse (all possible universes) implies that string theory universes (or at least computable approximations) are real regardless of whether we live in one.

Like Tegmark, Greene convinces me that inflation which lasts for infinite time implies infinite space and infinite copies of earth, but fails to convince me that he has a strong reason for assuming infinite time.

The main text is mostly easy to read. Don’t overlook the more technical notes at the end – the one proposing an experiment that would distinguish the Many Worlds interpretation of quantum mechanics from the Copenhagen interpretation is one of the best parts of the book.

Rule of the Clan

Posted by Peter on June 16, 2014
Posted in: Book Reviews, Politics. Tagged: , .

Book review: The Rule of the Clan: What an Ancient Form of Social Organization Reveals About the Future of Individual Freedom by Mark S. Weiner.

This book does a good job of explaining how barbaric practices such as feuds and honor killings are integral parts of clan-based systems of dispute resolution, and can’t safely be suppressed without first developing something like the modern rule of law to remove the motives that perpetuate them.

He has a coherent theory of why societies with no effective courts and police need to have kin-based groups be accountable for the actions of their members, which precludes some of the individual rights that we take for granted.

He does a poor job of explaining how this is relevant to modern government. He writes as if anyone who wants governments to exert less power wants to weaken the rule of law and the ability of governments to stop violent disputes. (His comments about modern government are separate enough to not detract much from the rest of the book).

He implies that modern rule of law and rule by clans are the only stable possibilities. He convinced me that it would be hard to create good alternatives to those two options, but not that alternatives are impossible.

To better understand how modern individualism replaced clan-based society, read Fukuyama’s The Origins of Political Order together with this book.

Our Mathematical Universe

Posted by Peter on April 14, 2014
Posted in: Artificial Intelligence, Book Reviews, Life, the Universe, and Everything. Tagged: , , .

Book review: Our Mathematical Universe: My Quest for the Ultimate Nature of Reality, by Max Tegmark.

His most important claim is the radical Platonist view that all well-defined mathematical structures exist, therefore most physics is the study of which of those we inhabit. His arguments are more tempting than any others I’ve seen for this view, but I’m left with plenty of doubt.

He points to ways that we can imagine this hypothesis being testable, such as via the fine-tuning of fundamental constants. But he doesn’t provide a good reason to think that those tests will distinguish his hypothesis from other popular approaches, as it’s easy to imagine that we’ll never find situations where they make different predictions.

The most valuable parts of the book involve the claim that the multiverse is spatially infinite. He mostly talks as if that’s likely to be true, but his explanations caused me to lower my probability estimate for that claim.

He gets that infinity by claiming that inflation continues in places for infinite time, and then claiming there are reference frames for which that infinite time is located in a spatial rather than a time direction. I have a vague intuition why that second step might be right (but I’m fairly sure he left something important out of the explanation).

For the infinite time part, I’m stuck with relying on argument from authority, without much evidence that the relevant authorities have much confidence in the claim.

Toward the end of the book he mentions reasons to doubt infinities in physics theories – it’s easy to find examples where we model substances such as air as infinitely divisible, when we know that at some levels of detail atomic theory is more accurate. The eternal inflation theory depends on an infinitely expandable space which we can easily imagine is only an approximation. Plus, when physicists explicitly ask whether the universe will last forever, they don’t seem very confident. I’m also tempted to say that the measure problem (i.e. the absence of a way to say some events are more likely than others if they all happen an infinite number of times) is a reason to doubt infinities, but I don’t have much confidence that reality obeys my desire for it to be comprehensible.

I’m disappointed by his claim that we can get good evidence that we’re not Boltzmann brains. He wants us to test our memories, because if I am a Boltzmann brain I’ll probably have a bunch of absurd memories. But suppose I remember having done that test in the past few minutes. The Boltzmann brain hypothesis suggests it’s much more likely for me to have randomly acquired the memory of having passed the test than for me to actually be have done the test. Maybe there’s a way to turn Tegmark’s argument into something rigorous, but it isn’t obvious.

He gives a surprising argument that the differences between the Everett and Copenhagen interpretations of quantum mechanics don’t matter much, because unrelated reasons involving multiverses lead us to expect results comparable to the Everett interpretation even if the Copenhagen interpretation is correct.

It’s a bit hard to figure out what the book’s target audience is – he hides the few equations he uses in footnotes to make it look easy for laymen to follow, but he also discusses hard concepts such as universes with more than one time dimension with little attempt to prepare laymen for them.

The first few chapters are intended for readers with little knowledge of physics. One theme is a historical trend which he mostly describes as expanding our estimate of how big reality is. But the evidence he provides only tells us that the lower bounds that people give keep increasing. Looking at the upper bound (typically infinity) makes that trend look less interesting.

The book has many interesting digressions such as a description of how to build Douglas Adams’ infinite improbability drive.

The Great Degeneration

Posted by Peter on March 2, 2014
Posted in: Book Reviews, U.S. Politics.

Book review: The Great Degeneration: How Institutions Decay and Economies Die, by Niall Ferguson.

Read (or skim) Reinhart and Rogoff’s book This Time is Different instead. The Great Degeneration contains little value beyond a summary of that.

The other part which comes closest to analyzing US decay is a World Bank report about governance quality from 1996 to 2011 which shows the US in decline from 2000 to 2009. He makes some half-hearted attempts to argue for a longer trend using anecdotes that don’t really say much.

Large parts of the book are just standard ideological fluff.

Self Comes to Mind

Posted by Peter on November 20, 2013
Posted in: Book Reviews, The Human Mind. Tagged: , , .

Book review: Self Comes to Mind: Constructing the Conscious Brain by Antonio R. Damasio.

This book describes many aspects of human minds in ways that aren’t wrong, but the parts that seem novel don’t have important implications.

He devotes a sizable part of the book to describing how memory works, but I don’t understand memory any better than I did before.

His perspective often seems slightly confusing or wrong. The clearest example I noticed was his belief (in the context of pre-historic humans) that “it is inconceivable that concern [as expressed in special treatment of the dead] or interpretation could arise in the absence of a robust self”. There may be good reasons for considering it improbable that humans developed burial rituals before developing Damasio’s notion of self, but anyone who is familiar with Julian Jaynes (as Damasio is) ought to be able to imagine that (and stranger ideas).

He pays a lot of attention to the location in the brain of various mental processes (e.g. his somewhat surprising claim that the brainstem plays an important role in consciousness), but rarely suggests how we could draw any inferences from that about how normal minds behave.

Reinventing Philanthropy

Posted by Peter on September 25, 2013
Posted in: Book Reviews. Tagged: , .

Book review: Reinventing Philanthropy: A Framework for More Effective Giving, by Eric Friedman.

This book will spread the ideas behind effective altruism to a modestly wider set of donors than other efforts I’m aware of. It understates how much the effective altruism movement differs from traditional charity and how hard it is to implement, but given the shortage of books on this subject any addition is valuable. It focuses on how to ask good questions about philanthropy rather than attempting to find good answers.

The author provides a list of objections he’s heard to maximizing the effectiveness of charity, a majority of which seem to boil down to the “diversification of nonprofit goals would be drastically reduced”, leading to many existing benefits being canceled. He tries to argue that people have extremely diverse goals which would result in an extremely diverse set of charities. He later argues that the subjectivity of determining the effectiveness of charities will maintain that diversity. Neither of these arguments seem remotely plausible. When individuals explicitly compare how they value their own pleasure, life expectancy, dignity, freedom, etc., I don’t see more than a handful of different goals. How could it be much different for recipients of charity? There exist charities whose value can’t easily be compared to GiveWell’s recommended ones (stopping nuclear war?), but they seem to get a small fraction of the money that goes to charities that GiveWell has decent reasons for rejecting.

So I conclude that widespread adoption of effective giving would drastically reduce the diversity of charitable goals (limited mostly by the fact that spending large amount on a single goal is subject to diminishing returns). The only plausible explanation I see for peoples’ discomfort with that is that people are attached to beliefs which are inconsistent with treating all potential recipients as equally deserving.

He’s reluctant to criticize “well-intentioned” donors who use traditional emotional reasoning. I prefer to think of them as normally-intentioned (i.e. acting on a mix of selfish and altruistic motives).

I still have some concerns that asking average donors to objectively maximize the impact of their donations would backfire by reducing the emotional benefit they get from giving more than it increases the effectiveness of their giving. But since I don’t expect more than a few percent of the population to be analytical enough to accept the arguments in this book, this doesn’t seem like an important concern.

He tries to argue that effective giving can increase the emotional benefit we get from giving. This mostly seems to depend on getting more warm fuzzy feelings from helping more people. But as far as I can tell, those feelings are very insensitive to the number of people helped. I haven’t noticed any improved feelings as I alter my giving to increase its impact, and the literature on scope insensitivity suggests that’s typical.

He wants donors to treat potentially deserving recipients as equally deserving regardless of how far away they are, but he fails to include people who are distant in time. He might have good reasons for not wanting to donate to people of the distant future, but not analyzing those reasons risks making the same kind of mistake he criticizes donors for making about distant continents.

War

Posted by Peter on August 31, 2013
Posted in: Book Reviews, Politics. Tagged: , .

Book review: War in Human Civilization by Azar Gat.

This ambitious book has some valuable insights into what influences the frequency of wars, but is sufficiently long-winded that I wasn’t willing to read much more than half of it (I skipped part 2).

Part 1 describes the evolutionary pressures which lead to war, most of which ought to be fairly obvious.

One point that seemed new to me in that section was the observation that for much of the human past, group selection was almost equivalent to kin selection because tribes were fairly close kin.

Part 3 describes how the industrial revolution altered the nature of war.

The best section of the book contains strong criticisms of the belief that democracy makes war unlikely (at least with other democracies).

Part of the reason for the myth that democracies don’t fight each other was people relying on a database of wars that only covers the period starting in 1815. That helped people overlook many wars between democracies in ancient Greece, the 1812 war between the US and Britain, etc.

A more tenable claim is that something associated with modern democracies is deterring war.

But in spite of number of countries involved and the number of years in which we can imagine some of them fighting, there’s little reason to consider the available evidence for the past century to be much more than one data point. There was a good deal of cultural homogeneity across democracies in that period. And those democracies were part of an alliance that was unified by the threat of communism.

He suggests some alternate explanations for modern peace that are only loosely connected to democracy, including:

  • increased wealth makes people more risk averse
  • war has become less profitable
  • young males are a smaller fraction of the population
  • increased availability of sex made men less desperate to get sex by raping the enemy (“Make love, not war” wasn’t just a slogan)

He has an interesting idea about why trade wasn’t very effective at preventing wars between wealthy nations up to 1945 – there was an expectation that the world would be partitioned into a few large empires with free trade within but limited trade between empires. Being part of a large empire was expected to imply greater wealth than a small empire. After 1945, the expectation that trade would be global meant that small nations appeared viable.

Another potentially important historical change was that before the 1500s, power was an effective way of gaining wealth, but wealth was not very effective at generating power. After the 1500s, wealth became important to being powerful, and military power became less effective at acquiring wealth.

The Origins of Political Order

Posted by Peter on June 26, 2013
Posted in: Book Reviews, Freedom, Politics. Tagged: , , , , .

Book review: The Origins of Political Order: From Prehuman Times to the French Revolution, by Francis Fukuyama.

This ambitious attempt to explain the rise of civilization (especially the rule of law) is partly successful.

The most important idea in the book is that the Catholic church (in the Gregorian Reforms) played a critical role in creating important institutions.

The church differed from religions in other cultures in that it was sufficiently organized to influence political policy, but not strong enough to become a state. This lead it to acquire resources by creating rules that enabled people to leave property to the church (often via wills, which hardly existed before then). This turned what had been resources belonging to some abstract group (families or ancestors) into things owned by individuals, and created rules for transferring those resources.

In the process, it also weakened the extended family, which was essential to having a state that impartially promoted the welfare of a society that was larger than a family.

He also provides a moderately good description of China’s earlier partial adoption of something similar in its merit-selected bureaucracy.

I recommend reading the first 7 chapters plus chapter 16. The rest of the book contains more ordinary history, including some not-too-convincing explanations of why northwest Europe did better than the rest of Christianity.

Radical Abundance

Posted by Peter on June 8, 2013
Posted in: Book Reviews, Molecular Assemblers (Advanced Nanotech).

Book review: Radical Abundance: How a Revolution in Nanotechnology Will Change Civilization, by K. Eric Drexler.

Radical Abundance is more cautious than his prior books, and targeted at a very nontechnical audience. It accurately describes many likely ways in which technology will create orders of magnitude more material wealth.

Much of it repackages old ideas, and it focuses too much on the history of nanotechnology.

He defines the subject of the book to be atomically precise manufacturing (APM), and doesn’t consider nanobots to have much relevance to the book.

One new idea that I liked is that rare elements will become unimportant to manufacturing. In particular, solar energy can be made entirely out of relatively common elements (unlike current photovoltaics). Alas, he doesn’t provide enough detail for me to figure out how confident I should be about that.

He predicts that progress toward APM will accelerate someday, but doesn’t provide convincing arguments. I don’t recall him pointing out the likelihood that investment in APM companies will increase dramatically when VCs realize that a few years of effort will produce commercial products.

He doesn’t do a good jobs of documenting his claims that APM has advanced far. I’m pretty sure that the million atom DNA scaffolds he mentions have as much programmable complexity as he hints, but if I only relied on this book to analyze that I’d suspect that those structures were simpler and filled with redundancy.

He wants us to believe that APM will largely eliminate pollution, and that waste heat will “have little adverse impact”. I’m disappointed that he doesn’t quantify the global impact of increasing waste heat. Why does he seem to disagree with Rob Freitas about this?