We are developing the social individualist meta-context for the future. From the very serious to the extremely frivolous... lets see what is on the mind of the Samizdata people.

Samizdata, derived from Samizdat /n. - a system of clandestine publication of banned literature in the USSR [Russ.,= self-publishing house]

The economics of bad science

A typical reaction to global warming skepticism is to point to all the institutions that endorse global warming and argue that this would require a grand conspiracy if global warming were false.

I argue that all that is needed is for incentives to align in a certain direction. The awarding of grants, the publication of papers and the media attention all point in one direction and there is positive feedback between them.

As reported in the New York Times, Diederik Stapel literally made up results of psychological experiments that were never done. It is not necessary to go quite that far.

Fraud like Stapel’s — brazen and careless in hindsight — might represent a lesser threat to the integrity of science than the massaging of data and selective reporting of experiments. The young professor who backed the two student whistle-blowers told me that tweaking results — like stopping data collection once the results confirm a hypothesis — is a common practice. “I could certainly see that if you do it in more subtle ways, it’s more difficult to detect,” Ap Dijksterhuis, one of the Netherlands’ best known psychologists, told me.

Journals and reviewers can play a part:

If Stapel was solely to blame for making stuff up, the report stated, his peers, journal editors and reviewers of the field’s top journals were to blame for letting him get away with it. The committees identified several practices as “sloppy science” — misuse of statistics, ignoring of data that do not conform to a desired hypothesis and the pursuit of a compelling story no matter how scientifically unsupported it may be.

The adjective “sloppy” seems charitable. Several psychologists I spoke to admitted that each of these more common practices was as deliberate as any of Stapel’s wholesale fabrications. Each was a choice made by the scientist every time he or she came to a fork in the road of experimental research — one way pointing to the truth, however dull and unsatisfying, and the other beckoning the researcher toward a rosier and more notable result that could be patently false or only partly true. What may be most troubling about the research culture the committees describe in their report are the plentiful opportunities and incentives for fraud. “The cookie jar was on the table without a lid” is how Stapel put it to me once. Those who suspect a colleague of fraud may be inclined to keep mum because of the potential costs of whistle-blowing.

So there are incentives to take an easy path of painting a simple, neat picture because it is more persuasive and saleable.

Stapel did not deny that his deceit was driven by ambition. But it was more complicated than that, he told me. He insisted that he loved social psychology but had been frustrated by the messiness of experimental data, which rarely led to clear conclusions. His lifelong obsession with elegance and order, he said, led him to concoct sexy results that journals found attractive. “It was a quest for aesthetics, for beauty — instead of the truth,” he said.

What the public didn’t realize, he said, was that academic science, too, was becoming a business. “There are scarce resources, you need grants, you need money, there is competition,” he said. “Normal people go to the edge to get that money. Science is of course about discovery, about digging to discover the truth. But it is also communication, persuasion, marketing. I am a salesman.

It is not just money; the rewards are the respect and admiration of one’s peers. In my talk on open source software on Friday I mentioned that this is one of the reasons individuals give away their source code or donate their time to open source projects. It feels good to make something that others find impressive.

I am lucky enough to work in software. There, the most aesthetically pleasing solution is usually the best one. And software can not easily be faked; it becomes apparent very quickly if it does not work. I can imagine software that appears to do what it claims to do without actually doing it, such as an encryption program that leaks your secrets. Open source software has largely solved this problem. In fact, science could learn a lot from open source software.

H/T Watts Up With That?

24 comments to The economics of bad science

  • Mr Ed

    Here’s a tip for all those with no science background. Whenever you come across a report of X being linked to Y, check to see if a mechanism has been established. If not, there is no science.

  • PersonFromPorlock

    My objection to calling academic science ‘a business’ is that it leaves us with no word for prostitution.

  • steve

    “I am lucky enough to work in software. There, the most aesthetically pleasing solution is usually the best one.”

    I am in soft disagreement with this statement.

    The most aesthetically pleasing solution is usually language dependent.
    So often this is only optimizing easy maintenance or reuse. Not a bad
    goal. But,optimizations for something else like performance in number
    crunching, memory footprint, or quick response times are often ugly
    looking unless that is what the language was designed for in the first
    place.

  • chuck

    Might take a look at this. Fernando has been involved in the “reproducible science” movement that aims to expose both the data and code involved in science publications, a necessity if computational results are to be considered in any sense ‘reproducible’.

  • “Open source software has largely solved this problem. In fact, science could learn a lot from open source software.”

    Was not “science” supposed to be exactly like this? You remember? You “write the code” and publish it to see if anyone can break it? Sort of like replicating an experiment to prove or disprove a theory?

  • Rob Fisher (Surrey)

    Steve: “optimizations for something else like performance in number
    crunching, memory footprint, or quick response times are often ugly
    looking”

    Semantics I suspect. I am thinking not how the code looks, necessarily, but of the beauty and elegance of the solution to whatever the problem is, whether that is optimising for maintenance or speed or whatever.

  • Just slightly off-topic, but fun.
    By Friday morning, Fort Worth, Texas will possibly have a low temperature of 38 degrees Fahrenheit. Previous record low was 41 degrees, set back in 1954.
    For the 3rd of May, this is known in Texas as “Freezing Your Ass Off”.
    Enjoy the warming !!!

  • I forgot to include the link, for those who want to follow along. Sorry.

    http://www.wfaa.com/weather

  • Dishman

    Science is dead…

    right next to God.

  • Julie near Chicago

    What Mr. Ed said. Note his word “established.” As opposed to “suggested,” “proposed,” “theorized,” “reasonable,” etc.

  • RogerC

    @Richard Quigley

    Was not “science” supposed to be exactly like this? … You “write the code” and publish it to see if anyone can break it? Sort of like replicating an experiment to prove or disprove a theory?

    Universities always like to describe themselves as research focused, but the reality is that they’re publication focused (publications being the most obvious evidence of research done), so naturally academics are too. There are few publications to be had from replicating someone’s research and saying “Yep, got exactly the same result”, so in practice much peer review consists of sense-checking the submitted paper looking for logical faults (either statistical or methodological) rather than in actually repeating the experiment.

    As usual, it boils down to incentives. If you don’t think you’re going to be caught (because noone’s going to bother repeating your work) there are no incentives not to cheat in your research and copious ones to do so, especially if it’s just a little bit, you know, to “tidy it up”. Likewise if you’re not going to be rewarded for repeating someone else’s work then there’s no incentive to do so, and an opportunity cost (at the least) to be avoided by not bothering.

    Seriously, you’d think psychologists of all people would be able to work this out…

  • Roger C, I strongly agreed with this:

    Likewise if you’re not going to be rewarded for repeating someone else’s work then there’s no incentive to do so, and an opportunity cost (at the least) to be avoided by not bothering.

    Even leaving out political incentives (which certainly exist), this problem is getting worse as science gets more complex. It’s harder than it used to be to replicate an experiment. The very words “replicate an experiment” have an old fashioned air; it is more a case of replicating a vast programme of research. There may be only a handful of places in the world with a particle acclerator handy!

  • Rob Fisher, you wrote, “A typical reaction to global warming skepticism is to point to all the institutions that endorse global warming and argue that this would require a grand conspiracy if global warming were false.”

    Quite. The next day after your post, we have this link from today’s Guardian: How climate scientists are being framed.
    The writer says,

    “Another name for this game might be “What’s the most offensive and ridiculous thing we can get away with saying about climate scientists?”

    To play, you need to first pretend thousands of studies, inquiries and reports into climate change and the science behind it never happened.

    You also need to accept a conspiracy theory so elaborate it would make the forger of Barack Obama’s birth certificate green with envy. The New World Order might also be seriously cheesed off.

  • Jaded Voluntaryist

    I love the Guardian comments section. Whenever you feel like giving in and becoming another right-on leftie Borg, 5 minutes reading “Comment is Free” is all it takes to convince you that would be a baaaaaaaad idea.

    I particularly liked the one one grumbling that while there is an Oil Industry, there is no such thing as a “Climate Industry”. Speaking from NE Scotland, the heartland of Scotland’s oil boom, I feel qualified to comment.

    There is more evidence of a Climate Industry than there is an Oil Industry in my immediate area. I’m a bit south of Aberdeen, if I were closer the reverse may be true. Locally there are plans for major windfarms all over the place. But much more commonly there are the odd turbine or two, put up by farmers to claim the subsidies on land they weren’t using for anything anyway. Most of the time their blades aren’t even turning.

    In my nearest harbour there is an ex-RNLI lifeboat with the words “survey” written on the side. It is here to conduct a seismic survey of the sea bed so that windfarms can be built out there as well. It sets off explosions underwater and uses the reflected boom to measure the density of the sea floor. Some blamed its activities for the repeated strandings of sea mammals along the Scottish coast last year. This was denied.

    Whereas, in my immediate vicinity, there are just a handful of factories making components for the oil industry. So the “Climate Industry” has gone from not existing to being the most visible growth industry in my area in the course of a few years. I would assert that it does indeed exist.

    It’s amazing to watch people rush to build turbines which by design hardly ever even make back their own construction costs. I guess there must be money in it somewhere, eh?

  • Mr Ed

    A scientist working as such has to have an economic reason for so doing, salary, materials, working premises, electricity etc. as all costs that have to be borne. It is not economic activity to do science as such. Funding is required, and the most obvious sources of funding are the State or charities devoted to, e.g. heart disease, or businesses looking for drugs or say, paints. Commercial science is focused on specific and ultimately economic ends.

    If the State has an agenda that it can and must control the climate, and by that the living standards and behaviour of its population, and it pays out money to scientists who work in that field of study (I do not reach for the label ‘science’), then it might not be that odd if there is formed an ‘Intellectual Bodyguard’ for the State’s agenda.

    Peer review of Isaac Newton’s theory of Gravity might have got us nowhere, as it was a monumental breakthrough.

    Peer review of Isaac Newton’s alchemy would have got us nowhere, as it was bunk.

    Peer review is as good as the peers involved.

  • Thank-you for that link, Natalie. Glad to see the extent to which I am not attacking straw men!

  • Jaded Voluntaryist

    Psychology and Climate Science have a lot in common. Both involve trying to look at extremely complex systems as a whole, and infer small things about their behaviour in specific circumstances. In both fields it is much, much harder to control for extraneous variables than it would be in a field like Physics or Chemistry. There is no amount of lead shielding that can prevent a participant from thinking about their laundry list when they should be doing your experiment. Likewise, since the climate system is the whole planet, extraneous variables are essentially uncontrollable. You can’t build a new Earth with the exact parameters needed for your experiment.

    Likewise in both fields there is a lot of money floating around for people who can produce the sorts of data that is in fashion at the moment. However, when it comes to accuracy and honesty, Psychology actually has an advantage over Climate Science. In Psychology you can actually test your predictions experimentally using participants. In Climate Science you just sit in your office and build a model.

    A Psychologist who is known for never leaving his office and never running any experiments, but who then publishes a paper in which he claimed to have tested 80 undergraduates will be found out in short order. Stapel got around this by actually collecting data, but not using it. However, even this simple degree of transparency is lacking in Climate Science, since they are expected to just sit in their offices running simulations on their computer. The opportunities for fraud are enormous.

    As are the opportunities for self deception. When one is immersed in a piece of data, it is very easy to run stats or build a model that makes your raw data say what you want it to say, and to do this without any real malice. In Psychology you’d maybe get one publication this way, but when you were unable to reproduce the results the interest would go away. In Climate Science everyone is using the same data, and the same models. So these acts of self delusion can spread from researcher to researcher unchecked.

    Running experiments in Psychology is hard. Just when you think you’ve got people figured out, a pattern of results that was completely unexpected turns up. It can be very difficult to build a coherent theory of what exactly is going on. All of the Psychologists I have ever met are fundamentally honest people. They overcome this difficulty by studying esoteric effects minutely. They interest themselves in spending 10 years finding out how you process the word 2 to the right of the one you are looking at. They avoid making grand claims, but keep their claims testable and specific.

    Climate Science, for the most part, seems to do the opposite.

  • Kevin B

    I’m of the old fashioned conspiracy theory persuasion and the title of that conspiracy is Sustainability – or, as it used to be known in the good old days when a chap could speak his mind, call a spade a spade and all that, Eugenics.

    Now you may well ask if I have any proof that some shadowy powers, perhaps lizard like in nature, are controlling the world enough to push their evil designs on the planet. Well, I give you Professor Paul Ehrlich.

    I don’t mean that the good Prof is one of those lizardly powers – goodness me no – but how else can you explain the fact that this fourth rate thinker with his monomaniacal desire to cull the human race is still being taken seriously after all these years of constantly failing predictions? And he’s not just being taken seriously by some third rate academic institution in the backwoods of the USA; governments and the ruling elite are still feting him and his wife as peerless seers.

    And don’t talk to me about James Hansen. How did he keep his job at one of NASA’s prestigious institutions while openly fiddling the temperature record, making failed modelling prediction after failed modelling prediction and getting repeatedly arrested for going on protest demos that would shame a bunch of liberal arts students?

    Then there’s all those third rate academic bureaucrats that wind up as UK Chief Scientific Advisors or heads of Scientific Institutions. Of course what else are you going to do with them – a few years in a non-job then a gong as reward for climbing the greasy pole – but have you noticed how they all witter on about population when they’re giving private talks and about Climate Change and how we need to de-industrialise to ‘combat’ it in publc?

    Of course this doesn’t neccessarily mean that these worthies are in favour of culling all the chavs and morons and coloured people while retaining enough salt-of-the earth peasants and artisans to keep them in the style to which they are accustomed. They just want to save the planet, (from all the chavs and morons and coloured people). And indeed most of them really do believe that a lump of rock that’s been carrying on quite happily on its own for billions of years suddenly needs their expert stewardship to prevent its collapse into … well some sort of disaster anyway.

    But the whole daft scenario, based on such obvious bluster, backed up with such shonky science, is so ridiculous on its face that it must be being driven by a dark and evil conspricacy.

    Otherwise, our elite minds would never fall for it. I mean we’re not that stupid. Are we?

  • BigFatFlyingBloke

    Humans are precisely that stupid.

    I also read a study from Bristol University today about how children who process/filter information more slowly are more prone to holding onto delusions or suffering from various types of psychotic or schizoid behavior.

    The holding onto delusions bit seems quite apt in the case of climate science.

  • SC

    I read the article on Stapel a few days ago. What struck me most about it was that the political element in the whole thing has been brushed under the carpet. He wasn’t just promoting his own career, he was promoting the progressive cause, because all his research was explicitly intended to provide support for leftist aims. Yet there just isn’t any mention of this in that article, nor do you see it mentioned much in other articles about him. If he had been a conservative or libertarian and his studies has supported those aims then of course that would have been big headlines. But when it’s yet another leftist lying to support the ’cause’ then the media goes very silent about the political element.

    The story then is just to do with his own personal advancement, and the pressures that academics are under to produce, etc., and people then shake their heads and say ‘I can’t understand why such a high-minded man would so blatantly and cynically fake whole studies just to get a bit of fame and a bit more money, the human mind really is a mystery’. Except it’s not in this case, because he’s really just another in the long line of leftists who will stop at nothing to promote the cause. Add that in with the fame and the prestige and the money and it’s no surprise at all.

    Notice that “He insisted that he loved social psychology but had been frustrated by the messiness of experimental data, which rarely led to clear conclusions”. That doesn’t sound like someone who loves social psychology, which is an inherently messy field. It sounds like someone who loves the idea that leftist views could (apparently) get support from science, and was frustrated that this was more difficult to achieve in practice than he thought.

    It’s also damning that he said that he came up with stuff that other academics in his field would expect to be true, so they wouldn’t look too closely at it. In other words, the field is full of lefties. If he had published findings which went against the left, then we all know what would have happened then, his work would have been pored over with a microscope for flaws. But publish something confirming that people are natural racists and everyone in the field just nods, and says ‘Well, of course’.

  • bloke in spain

    The only surprise is that anyone’s surprised.
    That’s what universities are designed to do, isn’t it? You shut up a bunch of supposedly intelligent people as far away from the real world as possible & let them compete in what sort of nonsense they can convince each other of. Why sensible people avoid coming into contact with those who’ve been through the process. It can be contagious.

  • RogerC

    @SC

    Notice that “He insisted that he loved social psychology but had been frustrated by the messiness of experimental data, which rarely led to clear conclusions”. That doesn’t sound like someone who loves social psychology, which is an inherently messy field. It sounds like someone who loves the idea that leftist views could (apparently) get support from science, and was frustrated that this was more difficult to achieve in practice than he thought.

    I found this insightful. The social sciences have always been vulnerable to an individual or a group with an agenda trying to use it to “prove” this or that pet theory about human nature, which in turn bolster’s this or that group’s political position. It’s why proper, diligent peer review needs to be encouraged and rewarded.

    Getting back to the original article, in the open source world, finding bugs *is* something that gets you rewarded, so there’s an incentive not just to write code (the equivalent of producing papers), but to diligently review code and to look for bugs (peer review). This is a huge advantage! The big problem with science as it’s currently set up to do business is that there’s plenty of incentive to write the code in the first place, but none at all to review it properly.

    “Given enough eyeballs, all bugs are shallow” – Eric S. Raymond. Science just doesn’t seem to get this at the moment, or doesn’t care.

  • software can not easily be faked; it becomes apparent very quickly if it does not work.

    I think that the main distinction is not necessarily between open source vs proprietary, or social “sciences” vs “hard” sciences. Rather, it is between science (in the sense of empirical inquiry into reality, truth-seeking etc), and its useful* applications (often known as ‘engineering’). For the purpose of my point what Rob does is (software) engineering – while what the likes of Stapel and Mann were supposed to be doing was science. Instead, what they were doing was social and political engineering under the guise of science. *The answer to the question whether the products of this engineering work or do not work depends on who the customer is, and what that customer’s demands and specifications are (i.e. what does one mean by ‘works’ or by ‘useful’).

  • Orson

    This Post by Rob is more Truthful and fact-based than you know.

    Rupert Darwall’s new book “The Age of Global Warming: a History” explains how a natural science (climatology and the environment) devolved into politicized social science of ecologism, stamping out the dissent real science needs in order to remain reality and Truth-seeking focused.

    The late Stanford University professor and National Academy of Sciences, Steven Schneider, rode the first Earth Day in 1970 with the aim of becoming a scientific leader/savior – first riding the global cooling scare during the 1970s, and then the global warming scare of the decades since.

    In a lecture in 2010, Schneider explained that “climate science” was more like education policy or health policy, and thus had to be assessed not by ordinary cannons of science – sound data collection, replicated analyses, testable hypotheses, experimental replication, falsifiability and logical consistent – but by a consensus of published papers touting their inductive findings. Falsifiability only enters into it on the scale of decades, he explained. In other words, climate science is no longer a sound physical science but a social one, subjected to all the usual corruption that any social sciences are rife with (pace Rob ABOVE).

    That I can generate great volumes of observational data proving that the sun goes around the earth doesn’t make it any less false; that I can save my pet theory by even greater data proving my epicycular auxiliary hypotheses to rescue it from challenge still doesn’t make it True. “Climate science” is indeed lost in a circular, self-interested miasma.

    A year ago the volume of internet traffic going to alarmist global warming blogs collapsed to a fraction of its former glory, and several leading skeptic bloggers went into the repose of semi-retirement, reflecting confidence of winners.

    Meanwhile, in the US, the rhetoric of alarm has devolved into promoting trendy hysterics: from “global warming” to “climate change” to “global weirding” to “climate chaos” now. Just days ago, the Department of Meteorology and Climate Science at San Jose State University (ie, Silicon Valley), was found testing the book burning of global warming skeptic book!

    This isn’t the behavior of the self-confident winners, much less actual scientists. More like “climatism” – the very ideology examined in Steve Goreham’s book in the photo.

    Recently, the 91 year-old atmospheric chemist Vincent Gray in New Zealand – and leading skeptic -states baldly that alarmist novelty in the scientific literature seems dried up; he has a lot less to catch up on to dispute because too much of it amounts to recycling of old dispatched claims or even obvious fraud.

    The world has ceased warming for the past 15 (or more) years. The post-Climategate trickle noticing this fact has become a steady stream now.

    The IPCC working group I (ie, the science section) report for next year was leaked last December, showing that recorded world temperatures have almost dropped completely out of the range of climate model predictions, as Dr Roy Spencer shows here, when looking at the satellite data in the lower troposphere (ie, where weather happens and people actually live).

    Lastly, icons of establishment alarm like The Economist and Der Spiegel have recently followed a trail blazed since last year and since by the New York Times environmentalist reporter turned blogger into climate skepticism of global warming alarms components – Andrew Revkin. Namely, is Arctic ice melting worryingly? No, it’s but complicated. Is the weather now anomalously weird? No. Or is the 15-year standstill in temperatures while CO2 increased 25% worrying? No.

    I trust the Church pf Climate Alarmist Orthodoxy no more than I trust any other self-interested bureaucracy. But various scandals of the past two years, is not since Climategate in 2009, take the sheen if not the brain-dead “trust the experts” rituals of this religion.

    It always helps when the priests of any Othodoxy beclown themselves – and it is happening with too much regularity not to inspire ridicule.