Carpe Datum

Cargo Cult Science

I often assign a reading to the students in my class at the beginning of the term. It is a commencement address given by the famous Physicist Richard Feynman called "Cargo Cult Science". Here is a brief excerpt:

In the South Seas there is a cargo cult of people. During the war they saw airplanes with lots of good materials, and they want the same thing to happen now. So they've arranged to make things like runways, to put fires along the sides of the runways, to make a wooden hut for a man to sit in, with two wooden pieces on his head to headphones and bars of bamboo sticking out like antennas--he's the controller--and they wait for the airplanes to land.

They're doing everything right. The form is perfect. It looks exactly the way it looked before. But it doesn't work. No airplanes land. So I call these things cargo cult science, because they follow all the apparent precepts and forms of scientific investigation, but they're missing something essential, because the planes don't land.

Now it behooves me, of course, to tell you what they're missing. But it would be just about as difficult to explain to the South Sea islanders how they have to arrange things so that they get some wealth in their system. It is not something simple like telling them how to improve the shapes of the earphones.

But there is one feature I notice that is generally missing in cargo cult science. That is the idea that we all hope you have learned in studying science in school--we never say explicitly what this is, but just hope that you catch on by all the examples of scientific investigation.

It is interesting, therefore, to bring it out now and speak of it explicitly. It's a kind of scientific integrity, a principle of scientific thought that corresponds to a kind of utter honesty--a kind of leaning over backwards. For example, if you're doing an experiment, you should report everything that you think might make it invalid--not only what you think is right about it: other causes that could possibly explain your results; and things you thought of that you've eliminated by some other experiment, and how they worked--to make sure the other fellow can tell they have been eliminated.

Details that could throw doubt on your interpretation must be given, if you know them. You must do the best you can--if you know anything at all wrong, or possibly wrong--to explain it. If you make a theory, for example, and advertise it, or put it out, then you must also put down all the facts that disagree with it, as well as those that agree with it.

There is also a more subtle problem. When you have put a lot of ideas together to make an elaborate theory, you want to make sure, when explaining what it fits, that those things it fits are not just the things that gave you the idea for the theory; but that the finished theory makes something else come out right, in addition. In summary, the idea is to give all of the information to help others to judge the value of your contribution; not just the information that leads to judgement in one particular direction or another.

The easiest way to explain this idea is to contrast it, for example, with advertising. Last night I heard that Wesson oil doesn't soak through food. Well, that's true. It's not dishonest; but the thing I'm talking about is not just a matter of not being dishonest; it's a matter of scientific integrity, which is another level.

The fact that should be added to that advertising statement is that no oils soak through food, if operated at a certain temperature. If operated at another temperature, they all will--including Wesson oil. So it's the implication which has been conveyed, not the fact, which is true, and the difference is what we have to deal with

That last part, the difference between the intellectual honesty and the deceit of advertising, is really on display in the current administration:

Bush and White House Press Secretary Ari Fleischer went out of their way Thursday to cite a new survey by "Blue-Chip economists" that the economy would grow 3.3 percent this year if the president's tax cut proposal becomes law. ...

Deputy White House Press Secretary Claire Buchan insisted Friday that the survey, which mentioned "the likelihood that some version of the Bush administration's latest stimulus package will be enacted," justified the president's claim. [Editor of the Blue-Chip newsletter] Moore said that a survey taken in January before the president announced his plan forecast 3.3 percent annual growth between the last quarter of 2002 and the last quarter of 2003. A survey taken in February reached the same consensus.

So, you see, what the Bush administration is *implying* is that his proposal will cause economic growth. The fact is that if you give the economists' survey any credibility, they feel the plan will have zero impact this year (which if true, makes you wonder about the supposed claims that tax cuts lead to growth which lead to more revenue, etc., which is demonstrably false anyway, even back during the Reagan years). So the reality is that the administration takes a fact that leads to a completely different conclusion and plays that fact as if it supports their case.

They did the same thing with the Al Qaeda broadcast I mentioned previously, claiming that it backed up their claims of alliance between Iraq and Al Qaeda, even though it did the opposite. It's so brazen, it's like a Saturday Night Live skit or something from The Onion. But it's not a skit. It's reality. And the difference is what we have to deal with.