Mathematical physicist John Baez made a Google Plus post about finding trends in data. David Friedman responded. My emphasis:
The problem is that, absent a theory, you don’t know what the shape of the function should be and different assumptions about the shape will lead to very different fits. If the ultimate reason to fit the curve is to test a theory and the person doing the fitting wants to believe in the theory, as we often do, it’s tempting to find some functional form that gives a result producing the desired outcome. I gather there is now even software out there that will do the specification search for you. The researcher can to some extent control the problem by specifying his form in advance, but there is always the temptation, if the result turns out wrong, to find some reason to try a different form—and if you don’t do so and as a result don’t publish, someone else with better luck in his first try or fewer scruples does. In the limiting case you try a hundred specifications and report the best fit as confirmed at the .01 level—the same result you would get with a hundred tries on random data. And the same thing can happen with a hundred perfectly honest researchers if only the significant result ends up published.
One solution, of course, is to make your data freely available so that other people can analyze it for themselves. The other solution, and the one that I think best from the standpoint of an outsider trying to decide whose theories and models to believe, is to evaluate by prediction rather than by the fit to past data. If the model is wrong and looks right when applied to past data because the past data was used to choose the specification and parameters, it is quite likely to go wrong on future data.
After being in lots of online arguments on climate issues, I decided to apply that approach to the IPCC models. I concluded that they had done a worse job of predicting the rate of warming than a straight line fit from 1910, when the current warming trend started, to the date of the first IPCC report. That strikes me as a reason to have low confidence in current projections coming out of the same approach.
For details see:
And for a more general sketch of the argument for taking prediction as better evidence of a correct theory than the fit to past data, see:
Update: What is particularly fascinating to me is the idea that 100 perfectly honest researchers will make models and by chance one of the models will validate against old data and that is the one that gets published. So there is a publication bias.