Idea Futures

Some comments on last weekend’s Foresight Conference:

At lunch on Sunday I was in a group dominated by a discussion between Robin Hanson and Eliezer Yudkowsky over the relative plausibility of new intelligences having a variety of different goal systems versus a single goal system (as in a society of uploads versus Friendly AI). Some of the debate focused on how unified existing minds are, with Eliezer claiming that dogs mostly don’t have conflicting desires in different parts of their minds, and Robin and others claiming such conflicts are common (e.g. when deciding whether to eat food the dog has been told not to eat).

One test Eliezer suggested for the power of systems with a unified goal system is that if Robin were right, bacteria would have outcompeted humans. That got me wondering whether there’s an appropriate criterion by which humans can be said to have outcompeted bacteria. The most obvious criterion on which humans and bacteria are trying to compete is how many copies of their DNA exist. Using biomass as a proxy, bacteria are winning by several orders of magnitude. Another possible criterion is impact on large-scale features of Earth. Humans have not yet done anything that seems as big as the catastrophic changes to the atmosphere (“the oxygen crisis”) produced by bacteria. Am I overlooking other appropriate criteria?

Kartik Gada described two humanitarian innovation prizes that bear some resemblance to a valuable approach to helping the world’s poorest billion people, but will be hard to turn into something with a reasonable chance of success. The Water Liberation Prize would be pretty hard to judge. Suppose I submit a water filter that I claim qualifies for the prize. How will the judges test the drinkability of the water and the reusability of the filter under common third world conditions (which I suspect vary a lot and which probably won’t be adequately duplicated where the judges live)? Will they ship sample devices to a number of third world locations and ask whether it produces water that tastes good, or will they do rigorous tests of water safety? With a hoped for prize of $50,000, I doubt they can afford very good tests. The Personal Manufacturing Prizes seem somewhat more carefully thought out, but need some revision. The “three different materials” criterion is not enough to rule out overly specialized devices without some clear guidelines about which differences are important and which are trivial. Setting specific award dates appears to assume an implausible ability to predict how soon such a device will become feasible. The possibility that some parts of the device are patented is tricky to handle, as it isn’t cheap to verify the absence of crippling patents.

There was a debate on futarchy between Robin Hanson and Mencius Moldbug. Moldbug’s argument seems to boil down to the absence of a guarantee that futarchy will avoid problems related to manipulation/conflicts of interest. It’s unclear whether he thinks his preferred form of government would guarantee any solution to those problems, and he rejects empirical tests that might compare the extent of those problems under the alternative systems. Still, Moldbug concedes enough that it should be possible to incorporate most of the value of futarchy within his preferred form of government without rejecting his views. He wants to limit trading to the equivalent of the government’s stockholders. Accepting that limitation isn’t likely to impair the markets much, and may make futarchy more palatable to people who share Moldbug’s superstitions about markets.

I once proposed using life expectancy as the primary indicator of what society should try to maximize.

Recently there have been reports that life expectancy is negatively correlated with standard measures of economic growth. I accept the conclusion that depressions and recessions are less harmful than is commonly believed, but I want to point out the dangers of looking at only the life expectancy in the same year as an event that influences life expectancy. Depressions may have harmful effects that take a decade to show up in life expectancy figures (e.g. long-term wealth effects, effects on willingness to wage war, etc). So I’d like to see how life expectancy averaged over the ensuing 10 or 15 years correlates with a year’s gdp change.

Book review: The Soulful Science: What Economists Really Do and Why It Matters by Diane Coyle.
This book provides a nice overview of economic theory, with an emphasis on how it has been changing recently. The style is eloquent, but the author is too nerdy to appeal to as wide an audience as she hopes. How many critics of economics will put up with quips such as “my Hamiltonian is bigger than yours!”?

The most thought-provoking part of the book, where she argues that economics has a soul, convinced me she convinced me she’s rather confused about why economics makes people uncomfortable.
One of her few good analogies mentions the similarities between critics of evolution and critics of economics. I wished she had learned more about the motives of her critics from this. Both sciences disturb people because their soulless autistic features destroy cherished illusions.
Evolutionary theory tells us that the world is crueler than we want it to be, and weakens beliefs about humans having something special and immaterial that makes us noble.
Likewise, economics tells us that people aren’t as altruistic as we want them to be, and encourages a mechanistic view of people that interferes with attempts to see mystical virtues in humans.

Some of her defenses of mainstream economics from “post-autistic” criticism deals with archaic uses of the word autistic (abnormal subjectivity, acceptance of fantasy). These disputes seem to be a disorganized mix of good and bad criticisms of mainstream economics that don’t suggest any wholesale rejection of mainstream economics. It’s the uses of autistic that resemble modern medical uses of the term that generate important debates.

She repeats the misleading claim that Malthusian gloom caused Carlyle to call economics the dismal science. This suggests she hasn’t studied critics of economics as well as she thinks. Carlyle’s real reason (defending racism from an assault by economists) shows the benefits of economists’ autistic tendencies. Economists’ mechanistic models and lack of empathy for slaveowners foster a worldview in which having different rules for slaves seemed unnatural (even to economists who viewed slaves as subhuman).

I just happened to run across this thought from an economist describing his autistic child: “his utter inability to comprehend why Jackie Robinson wasn’t welcomed by every major league team”.

She tries to address specific complaints about what economists teach without seeing a broad enough picture to see when those are just symptoms of a broader pattern of discomfort. Hardly anyone criticizes physics courses that teach Newtonian mechanics for their less-accurate-than-Einstein simplifications. When people criticize economics for simplifications in ways that resemble creationists’ complaints about simplifications made in teaching evolution, it seems unwise (and autistic) to avoid modeling deeper reasons that would explain the broad pattern of complaints.
She points to all the effort that economists devote to analyzing empirical data as evidence that economists are in touch with the real world. I’ll bet that analyzing people as numbers confirms critics’ suspicions about how cold and mechanistic economists are.

She seems overconfident about the influence economists have had on monetary and antitrust policies. Anyone familiar with public choice economics would look harder for signs that the agencies in question aren’t following economists’ advice as carefully as they want economists to think.

I’m puzzled by this claim:

The straightforward policy implication [of happiness research] is that to increase national well-being, more people need to have more sex. This doesn’t sound like a reasonable economic policy prescription

She provides no explanation of why we shouldn’t conclude that sex should replace some other leisure activities. It’s not obvious that there are policies which would accomplish this goal, but it sure looks like economists aren’t paying as much attention to this issue as they ought to.

She appears wrong when she claims that it’s reasonable to assume prediction market traders are risk neutral, and that that is sufficient to make prediction market prices reflect probabilities. Anyone interested in this should instead follow her reference to Manski’s discussion and see the response by Justin Wolfers and Eric Zitzewitz.

A number of people have compared the final forecasts for the election (e.g. this), but I’m more interested in longer term forecasting, so I’m comparing the state-by-state predictions of Intrade and FiveThirtyEight on the dates for which I saved FiveThirtyEight data a month or more before the election.

Here is a table of states where Intrade disagreed with FiveThirtyEight on one of the first four dates for which I saved FiveThirtyEight data or where they were both wrong on July 24. The numbers are probability of a Democrat winning the state’s electoral votes, with the Intrade forecast first and the FiveThirtyEight forecast second.

State 2008-07-24 2008-08-22 2008-09-14 2008-10-01
CO 71/68 60/53 54.5/46 67.5/84
FL 42/29 34.5/28 30/14 55.2/70
IN 38/26 34.1/15 20/11 38/51
MO 50/26 32.9/13 22.1/11 42.5/48
NC 30/22 25/21 14/7 51/50
NV 51.2/49 49/45 44.9/32 55/66
OH 65/53 50/38 40/29 53.5/68
VA 60.5/50 52.3/36 42/22 59/79

On July 24, both sites predicted Florida, Indiana, and North Carolina wrong. FiveThirtyEight got Indiana right on Oct 1 when Intrade was still wrong, but Intrade got North Carolina right on that date (just barely) while FiveThirtyEight rated it a toss-up.
Intrade got Nevada right on July 24 (just barely) while FiveThirtyEight got it wrong (just barely).
For Virginia, Intrade was right in July and August while FiveThirtyEight was undecided and then wrong.
FiveThirtyEight got Colorado wrong on September 14, but Intrade didn’t.
FiveThirtyEight got Ohio wrong on August 22, while Intrade got it right.
Intrade rated Missouri a toss-up on July 24, while FiveThirtyEight got it right.

On September 14, FiveThirtyEight was fooled by McCain’s post convention bounce by a larger margin than Intrade, but by Oct 1 FiveThirtyEight was more confident about correcting those errors.
For states that were not closely contested, there were numerous examples where Intrade prices where closer to 50 than FiveThirtyEight. It’s likely that this represents long-shot bias on Intrade.

In sum, Intrade made slightly better forecasts for the closely contested states through at least mid September, but after that FiveThirtyEight was at least as good and more decisive. Except for Intrade’s Missouri forecast on July 24, the errors seem largely due to underestimating the effects of economic problems – errors which were also widespread in most forecasts for other things affected by the recession.

In the senate races, I didn’t save FiveThirtyEight forecasts from before November 1. It looks like both Intrade and FiveThirtyEight made similar errors on the Alaska and Minnesota races.
[Update on 2009-01-13: contrary to initial reports, they apparently got the Alaska and Minnesota races right, although there’s still some doubt about Minnesota.]

On the other hand, Intrade had been fairly consistently (but not confidently) saying since early July that California’s Proposition 8 (banning same-sex marriage) would be defeated. Pollsters as a group did a somewhat better job there by issuing conflicting reports.

I’ve made a change to the software which should fix the bug uncovered last weekend.
I’ve restored about half of the liquidity I was providing before last weekend. I believe I can continue to provide the current level of liquidity for at least a few more months unless prices change more than I currently anticipate. I may readjust the amount of liquidity provided in a month or two to increase the chances that I can continue to provide a moderate amount of liquidity until all contracts expire without adding more money to the account.
I’m not making new software public now. I anticipate doing so before the end of November.

Last night an Intrade trader found and exploited a bug in my Automated Market Maker, manipulating DEM.PRES-TROOPS.IRAQ until Intrade rejected one of the market maker’s orders for lack of credit and the software shut down.
The bug involves handling of partial executions of orders, and doesn’t appear to be easily fixable (what happened looks nearly identical to the scenarios I had analyzed and thought I had guarded against).
For the moment, I’ve reduced the market maker’s order size to one contract, which will prevent further exploitation but provide much less liquidity.
I will try to fix the bug sometime in November and increase the order size (on the contracts that don’t get expired at election time) by as much as I can without adding more money to the market maker’s account. I will also analyze the information provided by the markets shortly after the election.

The stock market reacted to today’s defeat of the bank bailout bill with an unusually big decline. Yet the news wasn’t much of a surprise to people watching Intrade, whose contract BAILOUT.APPROVE.SEP08 was trading around 20% all morning. Why did the stock market act as if it was a big surprise?
Did Intrade traders make a lucky guess not based on adequate evidence? Did they have evidence that the stock market ignored? Could the stock market have priced in an 80% chance of the bill being defeated (if so, that would seem to imply that passage would have caused the biggest one-day rise in history)? Could the stock market have been reacting to other news which just happened to coincide with the House vote? (It looks like the market had a short-lived jump coinciding with news that House leaders hoped to twist enough arms to reverse the vote, but I wasn’t able to watch the timing carefully because I was at the dentist).

It seems like one of these must be true, but each once seems improbable.

Arnold Kling, whose comments on the bailout have been better than most, was surprised that the bill failed.

I covered a few of my S&P 500 futures short positions at near the end of trading, but I’m still positioned quite cautiously (I made a small profit today).

To deter any suspicion that the comparisons I plan to make between Intrade’s predictions and polls are comparisons I selected to make Intrade look good, I’m announcing now that I intend to use FiveThirtyEight.com as the primary poll aggregator. I intend to pay attention to predictions that are more long-term than I focused in 2004, so the comparison I’ll attach the most importance to will be based on the first snapshot I took of FiveThirtyEight.com’s state by state projections, which was on July 24.

Also, as of last week, one of the Presidential Decision Markets that I’m subsidizing, DEM.PRES-OIL.FUTURES, has attracted enough trading (I suspect from one large trader) to make me reasonably confident that it’s showing the effects of trader opinion rather than the effects of my automated market maker (saying that oil futures will drop if the Democratic candidate wins, and rise if he loses).

Book review: Infotopia: How Many Minds Produce Knowledge by Cass R. Sunstein.
There’s a lot of overlap between James Surowiecki’s The Wisdom of Crowds and Infotopia, but Infotopia is a good deal more balanced and careful to avoid exaggeration. This makes Infotopia less exciting but more likely to convince a thoughtful reader. It devotes a good deal of attention to conditions which make groups less wise than individuals as well as conditions where groups outperform the best individuals.
Infotopia is directed at people who know little about this subject. I found hardly any new insights in it, and few ideas that I disagreed with. Some of its comments will seem too obvious to be worth mentioning to anyone who uses the web much. It’s slightly better than Wisdom of Crowds, but if you’ve already read Wisdom of Crowds you’ll get little out of Infotopia.