We are developing the social individualist meta-context for the future. From the very serious to the extremely frivolous... lets see what is on the mind of the Samizdata people.

Samizdata, derived from Samizdat /n. - a system of clandestine publication of banned literature in the USSR [Russ.,= self-publishing house]

David Friedman on prediction

Mathematical physicist John Baez made a Google Plus post about finding trends in data. David Friedman responded. My emphasis:

The problem is that, absent a theory, you don’t know what the shape of the function should be and different assumptions about the shape will lead to very different fits. If the ultimate reason to fit the curve is to test a theory and the person doing the fitting wants to believe in the theory, as we often do, it’s tempting to find some functional form that gives a result producing the desired outcome. I gather there is now even software out there that will do the specification search for you. The researcher can to some extent control the problem by specifying his form in advance, but there is always the temptation, if the result turns out wrong, to find some reason to try a different form—and if you don’t do so and as a result don’t publish, someone else with better luck in his first try or fewer scruples does. In the limiting case you try a hundred specifications and report the best fit as confirmed at the .01 level—the same result you would get with a hundred tries on random data. And the same thing can happen with a hundred perfectly honest researchers if only the significant result ends up published.

One solution, of course, is to make your data freely available so that other people can analyze it for themselves. The other solution, and the one that I think best from the standpoint of an outsider trying to decide whose theories and models to believe, is to evaluate by prediction rather than by the fit to past data. If the model is wrong and looks right when applied to past data because the past data was used to choose the specification and parameters, it is quite likely to go wrong on future data.

After being in lots of online arguments on climate issues, I decided to apply that approach to the IPCC models. I concluded that they had done a worse job of predicting the rate of warming than a straight line fit from 1910, when the current warming trend started, to the date of the first IPCC report. That strikes me as a reason to have low confidence in current projections coming out of the same approach.

For details see:

http://daviddfriedman.blogspot.com/2014/03/have-past-ipcc-temperature.html

And for a more general sketch of the argument for taking prediction as better evidence of a correct theory than the fit to past data, see:

http://daviddfriedman.blogspot.com/2010/03/prediction-vs-explanation.html

Update: What is particularly fascinating to me is the idea that 100 perfectly honest researchers will make models and by chance one of the models will validate against old data and that is the one that gets published. So there is a publication bias.

62 comments to David Friedman on prediction

  • Nick (Blame Frenchmen) Gray

    True, whilst the climate is heating (certainly Australia is getting hotter!), the outcomes are uneven. A decade ago, Tim Flannery talked Sydney into investing in a desalination plant, since we would be in drought conditions for ages. When the massive rains came, a few years ago, the project was sidelined, and now seems to be forgotten. Still, we are having more bushfires than before, and our Snowy Mountains have far less snow on them than in previous years, even if some people try to tell me that climate heating has stalled. I don’t have faith in the IPCC, but I can use my own eyes.

  • chuck

    The trying of multiple models is the same sort of problem as determining the significance of peaks in, say, data from the LHC. If you look at enough random data you *will* find peaks, some of large excursion. Consequently, that possibility needs to be accounted for when computing significance.

  • Regional

    Nick,
    I don’t know what Australia you’re living in?
    It’s not getting hotter, cooler if any thing, but it’s negligible.

  • Nick (Blame Frenchmen) Gray

    So how come the snowline keeps rising? How come the coral of the Great Barrier Reef is suffering from too much hot water? Where is this other Australia that you live in?

  • Regional

    Nick,
    You shouldn’t rely on the ABC for information. Remember the ABC champion Flannery and all his predictions, no one came true.

  • Nick (Blame Frenchmen) Gray

    But the lack of snow has been reported elsewhere- they need snow machines to have just enough snow to keep the tourists reasonably happy. The ABC is not my only source of information.

  • Bombadil

    There are two separate issues in the AGW debate.

    1) Is the planet warming?
    2) If it is, can the warming be blamed on human activity?

    Regarding (1), this seems like it ought to be a simple issue to resolve but apparently it is not. Go figure.

    Issue (2) seems more problematic yet, to me. What exactly is the falsifiable hypothesis about anthropogenic global warming? Because I assert, ala Karl Popper, that absent a falsifiable hypothesis the AGW “theory” cannot properly be considered science.

    Back to the subject of the post: but how will you unravel the natural cycles of cold and hot that the Earth has experienced for the past millions of years (c.f. ice ages ad nauseum) and the effect of one-off events like comet impacts and volcanic eruptions, tangled together with the possible effects of variable sunspot activity, from the effects of CO2 emissions on our overall climate? The complexity of the data sets, and the stated (leftist) political motivations of the people making the AGW argument, leave me skeptical.

    Let me say that my confidence in climate models with preposterously wide-ranged “predictions” is low. Yet I am a reasonable man … how do you propose to convince me?

  • Regional

    Nick,
    I can remember seasons with very little snow, one or two years is not a trend.

  • Snorri Godhi

    Devoted readers of Samizdata will have noticed that this post is closely related to Perry Metzger’s devastating arguments against Pikettian extrapolation, and also to Brian Micklethwait’s post about Detlev Schlichter’s post about apriorism.
    The connection is that extrapolation and prediction require a theory: the data do not speak for themselves about the future (as Hume saw very clearly). Nor can we take for granted that a theory is true simply because it fits the data observed to date.

    A minor point is that John Baez’s curve fitting is (cubic spline) interpolation, not extrapolation: a simplified description of past+present temperature, rather than a prediction of future temperature. Even so, as he acknowledges, there is legitimate disagreement about what model, and model parameters, are appropriate.

  • bobby b

    Long, polite way of saying “they cooked their data”, which has been the primary objection since forever.

    But the same low-info people will simply refuse to read it, since our respective priests tell us all that anything that fails to support The Cause is venal and apostasy and blasphemous. And fattening and stupid. And racist.

    That’s the new threshold: you don’t convince converts by setting out logical and rational arguments; you have to get them to switch gods.

  • Rob Fisher (Surrey)

    bobby b: the interesting thing is that you don’t *need* it to be deliberate. Here is a mechanism by which data could conceivably be cooked by mistake. It’s helps us defend against accusations of conspiracy theorising.

    I was already aware of confirmation bias and the bias of studies that go looking for correlations and inevitably find one. What’s new and fascinating to me here is the idea that there is a publication bias: a hundred different researchers *honestly* attempt to fit a model to data and one of them finds a match by chance and *that one* gets published. But it means nothing unless it successfully makes a prediction.

  • Mr Ed

    “they cooked their data”

    Another adverse side-effect of warming.

  • Kevin B

    The ‘global warming’ scenario makes an interesting case study. It was driven on the political side by the likes of Maurice Strong, John Holdren and Crispin Tickel and on the scientific side by such as James Hanson and Michael Mann. One thing most of these people had in common was they are followers of the prophets of doom. Malthus, Ehrlich and the Club of Rome are their spiritual leaders and ‘population explosion’, ‘peak oil’ and ‘resource depletion’ are their mantras.

    Catastrophic Anthropogenic Global Warming was a handy means to a pretty dismal end for these guys; the end being world government, with control of production, consumption and, above all, population as the key goals.

    Once CAGW became the dominant paradigm in climate research, (and all the money produced by the politicians was for global warming research), then the stage was set for the later researchers to participate in the less deliberate, perhaps even innocent, manipulation of data to confirm the theory.

    Science is tribal, as is all human social interaction, and once the tribal elders are backing a particular paradigm, then it takes a major coup to topple that narrative and install a new one.

    And, of course, it isn’t just climate science that suffers in this way. From a review of The Big Fat Surprise by Nina Teicholz:

    What Keys did [after he was challenged by other scientists] was spend the rest of his career wallowing in the confirmation bias. Instead of following the scientific method and trying to refute his diet-heart hypothesis, he made it his mission to look for anything and everything that confirmed it. And ignored or belittled any conflicting data.

    Keys’ formidable powers of persuasion along with his academic credentials led over time to his diet-heart hypothesis being accepted by just about everyone. Anyone who dared to disagree was attacked with great vitriol in the pages of any journal in which the opposing argument appeared.

    Thanks to his non-stop promotional abilities, Keys ran roughshod over his detractors, and in his annus mirabilis, 1961, scored three major triumphs. First, he graced the cover of Time, he wrangled the American Heart Association (AHA) into his low-fat corral, and he got the National Institutes of Health (NIH) to buy into his theory. The AHA and NIH coups were particularly important because the first was an enormous lobbying agency and the second was the largest source of funding. With these two aboard Keys could both promote his anti-fat ideas to doctors and the public and could get the funds to do studies to confirm his bias.

    It is interesting to note that part of Ancell Keys’ method was to use PR via the media to establish the ‘importance’ of his cause and then co-opt a lobbying organisation and a governmant agency on his side. Very typical of the way much science is done, and not just these days. Even back in the days of the gods such as Einstein and Newton, political clout and media savvy were just as important as scientific acumen.

  • CaptDMO

    Of the 99.9% of “scientists” agreeing on the gub’mint’s “we must redistribute wealth to “fix” this” theory, how many rely on present, or future, gub’mint funding to “agree”? How many of their “papers” boil down to “According to the latest IPCC report…” (as well as NASA)
    Two years ago, the cycle of “weather” produced some impressive flooding, last winter produced some of
    the highest snow banks, and most school “snow days” in a decade. It was STILL snowing in May.
    Yeah,yeah, I know, this too PROVES AGW, and the need to necessarily ration electricity for “plug in” cars, as WELL as adulterate fuel with “extra” processed foodstuffs.

  • Laird

    BTW, CaptDMO, that “99%” figure is a fraud. No one really knows where it came from, and there is nothing to support it, but it keeps being regurgitated by CAGW enthusiasts as if it were true.

  • Jerry

    Nick,
    The snows on Mt Kilamanjaro are retreating, yet it never gets above 30 degrees F. Why is that?

  • Tedd

    Once again, Snorri beats me to the punch. Theory is always the ultimate goal of science. Data is merely there to help choose the best theory at any point in time. The purpose of fitting different types of curves to data is to generate hypotheses for further testing. But it is always the predictive power of each hypothesis that should determine which one prevails. The closest science can ever come to truth is a theory that survives, and that is always conditional on not being superseded by a theory with better predictive power.

  • Tedd

    To continue my thought: This is also why social sciences (including economics) are less effective than physical sciences. It’s usually infeasible in the social sciences to control or allow for enough variables to properly evaluate the predictions of theory. The attempt to do so is riddled with opportunities for bias, whether intentional or not. So there is endless, irresolvable debate about the predictive power of one theory versus rival theories.

  • Natalie Solent (Essex)

    The second, 2010, link is not working for me. I found the piece anyway by looking in the archives and its url is:
    http://daviddfriedman.blogspot.co.uk/2010/03/prediction-vs-explanation.html

    This link does work for me. The only difference I can see is that the link that works for me has blogspot.co.uk whereas the link that doesn’t work for me has .com in place of .co.uk, i.e.:

    http://daviddfriedman.blogspot.com/2010/03/prediction-vs-explanation.html

    Don’t ask me why this is so, since he is an American living in America.

  • It’s usually infeasible in the social sciences to control or allow for enough variables to properly evaluate the predictions of theory.

    Tedd, with that in mind, I’m curious to know what do you make of the evolution theory. Not just Darwin’s, of course, but any such possible theory. (Same question to Snorri). Thanks.

  • Richard Thomas

    In the Erisian Archives is an old memo from Omar to Mal-2: “I find the Law of Fives to be more and more manifest the harder I look.”

  • Snorri Godhi

    Rushing to beat Tedd once again to the punch, i’ll give a tentative answer to Alisa.
    (Actually i am not at all sure that i am addressing Alisa’s question, but i am eager to establish priority anyway.)

    The way i look at Darwin’s theory is:
    A. it is a fact (though all facts are theory-laden) that there is reproduction with random mutations.
    B. it is a logical necessity (tautology) that the organisms most likely to reproduce, are most likely to reproduce.

    Therefore, Darwinian evolution is as much of a fact as reproduction with random mutations; a fact which would be silly to deny.

    What is NOT a fact, is:
    C. Darwinian evolution is the ONLY mechanism responsible for the emergence of complex life forms, e.g. us.

    I am not sure about the logical status of C. Is it falsifiable? to falsify it, we’d have to establish that another mechanism is at play — but a theory about another mechanism would not be verifiable, only falsifiable!

  • Snorri Godhi

    PS: i take the opportunity to qualify my praise of Hume:
    He saw very clearly that data about the past are not a reliable guide to the future.
    What he did not see, it that data by themselves are no guide whatsoever about the future: Hume thought that data by themselves allow us to predict the future, except that this prediction is not logically justified.
    To the best of my knowledge, Whewell, or perhaps John Herschel, were the first to see that data by themselves do not allow even a rash guess about the future, in the absence of a theory.

    (Paul Marks might be happy with my putting Hume in his place.)

  • bobby b

    “Here is a mechanism by which data could conceivably be cooked by mistake. It helps us defend against accusations of conspiracy theorising.”

    I do see your point here, and I recognize the utility of the approach, both as a shield against being easily labeled as conspiracy-lunatics, and as a means to give CAGW followers an easier out by calling them “misled” instead of knowingly complicit in venality.

    In my mind, though, it seems much like comforting various Hutus who had actually wielded their machetes against their neighbors the Tutsis by allowing that they had acted badly but only because they were in thrall to evil people – that they were not independent sources of evil themselves.

    Following the idea that hate has to end somewhere if it is to end at all, such thinking likely is the best – maybe the only – way to end a previously endless cycle. Allowing those CAGW followers to claim their own victim status will probably get us out of CAGW theater far faster than were we to hold each and every adherent’s feet to the fire forever.

    I think I’m still too bitter to feel that way in my heart yet. I think it IS a conspiracy, and I believe that the anti-CAGW evidence is now so strong that a continued advocacy of CO2 elimination can only be pursued in bad faith.

    I would be a poor diplomat.

  • Nick (Blame FrenchMEN) Gray

    Regional, today’s ‘The Australian’ (not a rabid fan of the ABC) has a Business story about how The Reject Shop has had to suffer a share price plunge, because of the unseasonably warm weather. Are they closet scammers?

  • Nick (Blame FrenchMEN) Gray

    Jerry, climate change would mean shifting wind patterns, so the water may not be reaching the mountain, or not cooling water vapour in the air, so snow doesn’t form.

  • lucklucky

    There is nothing tricky, it is perfectly clear that we don’t know enough.
    The graphic is meaningless.
    Seems John Baez is another arrogant human that substituted the Gods with Scientism to explain everything in the world.

  • Thanks Snorri (wow, Tedd is slow :-P)

    That makes sense. However, I was thinking along the lines of reproducing the experiment , and the human inability to do so to any reasonable degree of accuracy when it comes to as prolonged a process as the evolution. I did get the superficial impression that this was what Tedd had in mind as well, more or less. I’m sure he will put me straight when he is back here.

  • Mr Ed

    I’m curious to know what do you make of the evolution theory. Not just Darwin’s, of course, but any such possible theory.

    Proponents of evolution do not make any convincing argument for the development of systems such as a mammalian immune system. These are extremely complicated and a disorder can rapidly prove fatal. Whereas they can show that an immune system confers a selective advantage, the question of how a mammal developed the entire chain of systems require for antigen recognition, recall and processing with (i) an imperfect system simply not working in any effective manner and (ii) the perfect system needing to come together as a working unit to be effective. Similarly with vision, developing all the systems needed for vision, processing, focusing, and co-ordinating with locomotion and auditory inputs when a single gene defect might knock out eyes as functioning organs and cause blindness (and therefore swift death in almost inevitable prior to civilisation), none have given satisfactory answers yet. Nor have they explained how chromosome jumps occur in higher mammals, where, if apes have a common ancestor and and different chromosome numbers, as each mammal is born one at a time, when the first ‘human’ appeared with the ‘right’ number of chromosomes, losing one down to 23, how did she or he survive and breed with another human (or sufficiently close species) so that chromosome disparities did not prevent the fused egg from developing? Whilst mules can arise from horse/donkey crosses, they are usually sterile. There is still a lot of explaining to be done.

    On the other hand, bacterial evolution of antibiotic resistance is so well-known as to be indisputable in existence and mechanism.

  • Snorri Godhi

    Hi Alisa: perhaps Popper’s Poverty of Historicism is the book to read on this. I say perhaps, because i have not yet got around to reading it.
    Also, i am not yet sure i understand your question (and of course i cannot speak for Tedd) but if i do, then my view is that prediction is as impossible in evolution as in the social “sciences”.
    I do believe, however, that there are a few “laws of history” which cannot be safely ignored, such as:
    There will always be a ruling class, ie a minority holding a disproportionate power of coercion.
    The State grows (power of coercion of the ruling class increases) over time, until the system collapses and the old ruling class is replaced by a new ruling class.
    What can’t go on forever, won’t.
    All economic bubbles eventually collapse.
    All fiat currencies eventually collapse.

    None of the above is my original idea, of course.
    (The reason these laws cannot be used for prediction (except possibly the first) is that they do not specify a time frame, eg even if there is a reversal in the growth of the State,
    one can always claim that the reversal is temporary.)

    I believe that something similar can be said about evolutionary biology, eg it can be said perhaps that the complexity of ecosystems increases up to a mass extinction; but i don’t feel as arrogantly confident about evolutionary biology as i do about “laws of history”.

  • Clovis Sangrail

    von Neumann said

    With four parameters I can fit an elephant, and with five I can make him wiggle his trunk.

    Until you understand what Neumann was driving at you won’t understand Friedman’s point.
    It’s not about 100 people fitting models and one of them happening to fit, it’s about one person torturing a model until it does fit the (past) data. Statisticans call this “overfitting” and we know it’s the road to ruin.

    The only possible test of whether a model has any validity as a description of reality is to see whether it fits new data!

    If you look in detail at climate models, they have so many tunable parameters [hence the phrase "wiggle room"] that it’s obvious that the elephant [in the room] not only can wiggle its trunk but can, in addition, get stoned, have sex with a passing octopus and eat tripe all at the same time.

  • Richard Thomas

    I think one has to be careful discussing evolution as there are two aspects. The first is the process and the second is as a historical explanation for how things got to be as they are. If you start crossing the streams, so to speak, confusion abounds. The first is indisputable in my opinion and can be scientifically tested, the second is not so much science but archaeology (I find it extremely convincing, personally) and not really more subject to science than “was the kitchen light on yesterday?”

  • Richard Thomas

    Mr Ed, the “Come together as a working system” argument is an argument debunked many times over and often just indicates a (usually willful) ignorance or denial of pathways of increasing complexity. They visual system, for example, has many stages ranging from simplistic photosensitive dermal cells all the way up to the fantastic (but flawed) system we humans have now and many if not all of these stages can be found in nature if one bothers to look. Indeed, our visual system is an argument for an evolutionary process as it has several flaws that could be easily solved but we *are* in a place of “If I were you, I wouldn’t start from here” for an evolutionary process to do so.

    Evolution in the wrong direction fatal to an organism? Big whoop. We have millions upon billions of organisms and millions of years. That is kinda the point of “Survival of the fittest” after all. Don’t fit, don’t survive.

    And you have issues with the chromosome thing (which I don’t have an answer for) by saying that horse donkey crosses are usually sterile? Well, another way to parse that is as “sometimes fertile”. Billions of organisms, millions of years and all that…

  • Mr Ed

    it has several flaws that could be easily solved

    Ah, you lapse into animism, Richard. However, it appears that you have not read what I wrote: I said that there is not yet a convincing explanation, not that it did not happen, and do we really have billions of humans and millions of years? How did they sustain themselves without capitalism?

    We have millions upon billions of organisms and millions of years.

    Not in the human race, a very small population and not millions of years.

    How did each chromosome jump occur? How did the F1 find a mate? Even if that happens once, that gives us 3 people, maybe 4 if we have twins etc. Think about the regression, day after day, each birth, each offspring reaching adolescence. I noted that this was glossed over in University, it is just an assumption, and he who asserts should prove.

  • Richard Thomas

    If we wish to posit that there are gaps in the theory currently, I have little issue with that. Just as there were gaps in the periodic table when Mendeleev first put it together. In my opinion, the rest of the theory is so sound and the gaps that do exist getting filled in at a sufficient rate (Archaeopteryx) that the burden is almost to show that these gaps disprove evolution if one deems them of much importance to the overal argument (not that they are not important in and of themselves of course).

    Though one must also be careful of falling into the trap of rationalization. That one can posit a mechanism does not necessarily mean that that is the mechanism in play, merely that there is at least one plausible explanation.

  • Snorri, I’ll try and put a finer point on what I mean: before we even get to discussing predictability, we have have to discuss reproducibility of the conditions under which the prediction may materialize in the future. For example, while the lay person may simply ask a scientist ‘Is it going to rain tomorrow?’, a serious answer from a serious scientist would be along the lines of (very crudely simplified): ‘Yes, if X, no if Y’. The problem with both climatology (much less so with meteorology) and evolutionary science is that X or Y conditions cannot be realistically reproduced. If one wishes to falsify a certain theory, one cannot do so, because the experiment cannot be repeated.

  • Snorri Godhi

    Alisa: it would help if you gave me a concrete example of X and Y, especially since i do not pay much attention to weather forecasts, so i am not sure what they could be.
    But an example about evolution came to mind yesterday, though i left it out:
    as long as there is an ecological niche still unexploited, there is some very small probability that it will be exploited in the next century by some random mutation. Small in the next century, but 10x larger in the next millennium, and still growing afterwards.
    (It is, however, difficult to identify unexploited ecological niches.)

    So before giraffes appeared, one could perhaps have predicted that, as long as tall trees survive, there is a good chance of some very tall herbivore species appearing in the future.
    What could not have been predicted, is when; and also that the tall herbivore would be a ruminant and would have a patchy color pattern.

    If this doesn’t help we can just continue to brainstorm.

  • Snorri Godhi

    Climatology is different. In climatology there is no “amplified” randomness afaik, as there is in biological and technological evolution, and political history.
    In fact, there might be more amplified randomness in weather than in climate, ie the butterfly effect.
    The reason climatology cannot make predictions afaik, is that we do not know enough about the physics of the Earth, and of the Sun.
    Also, climatologists have a class interest in publishing scientific claims which lead to more funding for their discipline, and i am not aware of any other science in which this is as easy as in climatology.

  • Richard Thomas

    Snorri, perhaps a historical predictive opportunity would have been before Australia was “discovered” was that in the absence of easy migration, parallel evolution would lead to analogs of animal species found elsewhere. Though pretty much this, in finches on the Galapagos, helped Darwin in his formulation.

  • Richard Thomas

    [Convoluted postulation of chromosome jumping explanation per Mr Ed deleted]

    Why can’t one of the chromosome in question just keep getting shorter, the required genetic material being absorbed by other chromosomes, until it is gone completely? If evolutionary pressure is to fewer chromosomes (less risk of damage or transcription errors), this seems like a viable pathway.

  • I’ll have to think about it some more Snorri – many thanks for engaging with me so far.

  • Snorri Godhi

    Richard: good point about Australia; but it needs a qualification: parallel evolution would happen if, and only if, similar ecological niches are to be found in Australia.

    As for the chromosome number issue: aren’t there scientifically-informed speculations out there? i don’t know of any, but i’d be surprised if nobody has thought about this before.

  • Mr Ed

    As for the chromosome number issue: aren’t there scientifically-informed speculations out there?

    Yes, mine for starters.

  • Tedd

    Alisa and Snorri:

    How flattering to see my name so often on this page, when I’m not even the one posting! Yes, I’m really slow to post these days. As usual, Snorri has covered it well, so I only have one small point to add.

    I’m reliably informed that the process of natural selection can be demonstrated in only a few weeks, using rapidly-reproducing species such as certain plants or insects. I say “reliably informed” because I’ve never actually done the experiments myself, but I have no reason to doubt the claims of those who say they have. So, in that narrowed-down sense, I’m comfortable that natural selection is a sound theory with predictive capability.

    As a personal matter, I also have no problem accepting the idea that natural selection explains how we all, as sentient, self-aware beings, come to be here. But that would be an example of the kind of prediction of a theory that’s extremely difficult to falsify experimentally, which I think was Alisa’s point.

  • Yes Tedd, that was my point, and specifically the time-frame problem, on which you touched directly. So I’d say tentatively and based on your indirect report, that the theory may be falsifiable for very primitive creatures, but will probably remain unfalsifiable for more complex ones (possibly until we can find a way to travel in time?). Or am I missing something?

  • Tedd

    Alisa:

    No, I don’t think you’re missing something.

    It’s not unlike Newton’s theory of gravity. If you test it with bowling balls in a vacuum you’ll find that you can’t plausibly falsify it. You can then predict with pretty good confidence that it will also work for baseballs, snowballs, or furballs in the same situation. You might also believe (as Newton did) that it can predict the motion of planets around stars, and you’d be right. But you need a different test to be as confident about that. So it’s entirely reasonable, in the absence of a competing theory about why complexity matters, to extrapolate the predictive ability of natural selection to any degree of complexity. But confidence drops the further away you get from the cases you can actually test.

    I guess the other thing about natural selection is that there’s an absence of competing theories that are falsifiable even in principle. Intelligent design advocates have made a heroic effort with their probability based theories. But, to me, when you consider the size of the universe (and all possible universes) the anthropic principle suggests that a probability based argument (against natural selection as an explanation for mankind) can never be very convincing. The numbers on the pro side are so incomprehensibly large that it’s hard to imagine any plausible assumptions on the con side that could outweigh them.

  • Indeed. Good analogy with Newton’s gravitation too – a special case of some other, more generalized theory, etc.

  • Mr Ed

    It’s not unlike Newton’s theory of gravity.

    I think that you are referring to Galileo’s observations on the comparative velocities of falling bodies and mass, and indirectly addressing the friction arising from fluid resistance, which applies regardless of gravity.

    Newton’s Law of Gravity was about bodies attracting each other, the inverse square law applying to the force which was related to the mass of the body concerned, perhaps the most staggeringly impressive deduction in history.

    Pedantry yes, but one should always give Newton his dues. And Robert can sling his Hooke.

  • Tedd

    Mr Ed:

    No, I was referring to Newton’s law. I don’t think Galileo applied his theory about falling objects to planetary bodies. But, of course, Newton’s theory encompasses the correct parts of Galileo’s theory.

  • Snorri Godhi

    “I guess the other thing about natural selection is that there’s an absence of competing theories that are falsifiable even in principle.”

    Tedd: you might have noticed — though no reproach is implied if you didn’t — that this is closely related to what i wrote @9:47 pm a few days ago: that Darwinian evolution is a logical, inescapable consequence of reproduction with random mutation; but a more falsifiable, hence preferable, theory is not ipso facto ruled out.
    The reason “we” prefer Darwinism is that we do not have a more falsifiable theory that can account for the facts: intelligent design is even less falsifiable than Darwinism (and to the extent that it is falsifiable, it has been falsified) and Lamarckism cannot account for the facts (and in addition it has some logical problems).
    (Any other alternatives?)

    About the analogy to Newton’s theory i am more skeptical. A better analogy might be to the ancient Greek project to describe everything with arithmetic (Pythagoras) and later with geometry (Plato, Euclid, and much later Galileo). Saying that geometry is the language of nature does not allow any predictions BY ITSELF, but it’s a prerequisite in making predictions in physics. The same with Darwin’s theory in biology.

  • Mr Ed

    Tedd,

    Newton’s law of gravity has nothing to say about the velocity of

    baseballs, snowballs, or furballs

    per se, it is only concerned with the forces acting between masses.

    It was Galileo who showed that the Platonic view that the speed of a falling body was independent of mass, and by so doing, he was paving the way for Newton.

    The problem with theories of evolution is that biology has a large element of Rutherford’s stamp collecting to it, and the likes of Dawkins come up with what is really a form of animism ‘the Selfish Gene’ to explain things that have happened, but they cannot show what has happened.

    One theoretical biologist asked me, in the course of a discussion ‘What is this, a court of law?’ To which I responded ‘Are your standards lower than a court of law?’ to which he had no answer.

  • Tedd

    Mr Ed:

    No, you missed the substance of my analogy. The analogy was to the extension of a theory that’s well established in one domain to a much wider domain, thus requiring a different kind of experiment. Galileo’s theory of falling bodies was much less universal in scope that Netwon’s universal theory of gravity, so Newton’s law is a much better choice for the analogy I was making. That’s why I chose it.

  • Nick (Blame FrenchMEN) Gray

    As for applications to living forms, here is an interesting idea that Isaac Asimov once mentioned, when discussing evolution verses creation. A creationist once pointed out the very long odds of life occuring on Earth, and taking just the right paths to lead to life-forms like us. Asimov pointed out the hidden assumption in that argument- that only one possible path leads to life. Could the creationist prove that other paths could not lead to a viable, living, system? Can anyone?
    In fact, I can point to a counter-example. Gangreen is such an odd life-form that I often wonder if it is a second form of evolutionary life, which developed later than our own branch.
    Anyone else have an idea?

  • Snorri Godhi

    A bit late, i bring you Popper’s own contribution to this debate: an abridged quote from Objective Knowledge, chapter 7, section 2. (Without the cuts, this quote actually fills an entire page.)

    Darwin’s discovery of the theory of natural selection has often been compared to Newton’s discovery of the theory of gravitation. This is a mistake. Newton formulated a set of universal laws [...]. Darwin’s theory of evolution proposed no such universal laws. There are no Darwinian laws of evolution. [...]

    Nevertheless, Darwin’s revolutionary influence upon our picture of the world around us was at least as great, though not as deep, as Newton’s. For Darwin’s theory of natural selection showed that it is in principle possible to reduce teleology to causation by explaining, in purely physical terms, the existence of design and purpose in the world.

    [...]

    Although this was a great achievement, we have to add that the phrase in principle is a very important restriction. [...]

    Read the whole thing!
    My only remark (on the 2nd paragraph of the quote) is that, in my immodest opinion, Darwin’s influence is deeper than Newton’s, because Darwin changed the way we think, more deeply than Newton did.

  • Tedd

    Snorri:

    I’m going to have to read that book, one of these days!

    But I did not intend to compare the theories, as I explained to Mr Ed. I merely used how one theory developed as an analogy for understanding how the other theory developed.

  • Mr Ed

    Tedd

    You did state:

    Theory is always the ultimate goal of science.

    I would be bolder, and say that a Law is always the ultimate goal of science. Not all science is amenable to laws, such as the diversity of beetles, however, ultimately all life has to follow the laws of chemistry and ultimately those of physics.

  • Snorri Godhi

    Tedd: yes, i see your point now, but i felt compelled to quote Popper since i happened to (re)read that passage today.
    Incidentally, since Objective Knowledge is a collection of revised essays, you don’t have to read it cover to cover. Chapter 1, which i quoted in the apriorism thread a few weeks ago, is particularly enlightening. If you read The Logic of Scientific Discovery, it’s more than just a useful summary; and if you didn’t, it’s a good introduction.

    Mr Ed: i assume that by “law” you mean a theory with predictive power. In this case, science as you define it would exclude, not only (most of) biology, and of course economics and the social “sciences”, but also logic, mathematics, statistics, cybernetics, computer science, etc.
    Not sure how many people would agree with this restriction.

  • Mr Ed

    Snorri.

    In science, a theory can be refined, and tested. A law is, by definition, immutable or it is not a law.

    Mathematics has proofs. Biology is Rutherford’s ‘stamp collecting’ in the main. ‘Computer science’ isn’t, really. Social sciences simply aren’t. Cybernetics is engineering, engineers are usually reliable, practical, straightforward and intelligent in my experience (and hence despised by Lefties).

    Economic law is violated daily, and denied even as reality dawns.

  • Snorri Godhi

    Mr E: then i misunderstood you, and i still do not understand.
    However, you are wrong about cybernetics: it is math, not engineering.

  • Bombadil

    A scientific law is just a theory with a beard (see Newton’s “Laws”). In empirical science, NOTHING is immutable. That’s the point.

  • Snorri Godhi

    What Bombadil said; to which i add: a scientific theory is just a hypothesis with a stubble.