From what one can glean from a thorough reading of the world’s press agencies’ output, which by the way is something I am required by a (corporate) intelligence agency to do on a daily basis, this culture of attaching importance to the source, rather than the actual data is a the problem at the heart of any intelligence gathering operation. When this problem is finally addressed (it won’t be, but we can wistfully dream) then we will actually have genuine intelligence being fed to the right people who can make an properly informed decision.

Nice informative article by the way, but I regret I do not ‘rate’ Mr Ahlberg too greatly.

if you’d started using Recorded Future’s predictions to buy US stocks on January 1, 2009, you would have made an annual return of 56.69 per cent. (The S&P 500 had an annualised return of 17.22 per cent over the same period.) Between May 13 and August 5 this year, as markets behaved with vertiginous abandon, their strategy returned 10.4 per cent; in contrast, the S&P 500 lost 9.9 per cent of its value.

That’s great, exciting, encouraging, stimulating. I want to believe, I truly do.

But I would bear in mind that the quoted returns were undoubtedly provided by the company, and the start and end points were chosen by them, too.

First, you need a crapload of computing power to do it. I am sure Google and the CIA can pull it off, but we could not, to the extent where we thought it would work. You can only run so many machines web-mining Twitter, CNN, Drudge, BBC, WSJ, etc.

Second, it does produce a limited amount of actionable knowledge, but also a crapload of noise. My company mostly invests on a daily basis, guessing whether the market, or some sectors of it, would go up or down the next day. Our model that incorporates web-chatter has done okay – up 24% this year, but this has included swings from down 17% to up 54%. If this were clients’ money, as opposed to house money, the clients probably would have bailed en masse when seeing down 17% on their statements. You cannot run a hedge fund with this kind of volatility.

Third, without a long history (and we do not have it), it’s hard to tease out useful contribution from luck. If on a single day, where the model made 14% by being beta 2 short (US rating downgrade), we had been wrong, all the gain would have been gone. It’s very hard (nigh impossible) to backtest such models – it’s just how data sources work.

To recap, I definitely would look at web mining as a source, but I would not bet my 401(k) on it. If, however, you are looking for longer term trends, and you are looking to improve your hit rate from 53-47 to 60-40, yep, definitely count them in.

To be frank, I wonder how much SIS/CIA are looking at that sort of stuff and keeping it inside.

40 years ago I was acquainted with a Special Branch officer who claimed that about half his work time was devoted to simply reading the newspapers: mainstream national titles, local press and the publications of groups like the National Front and the Socialist Workers’ Party.

A US diplomat, not a spy, once told me that her job largely consisted of reading the trade and business press and going to product launches and receptions.

Allow for boasting, distortion and the certain knowledge that neither was telling me any secrets and it still shows that open-source intelligence gathering is certainly not cutting edge stuff.

So there is a definite contrast between analysis and received information. And perhaps therein lies the link between Science fiction and libertarianism.