Samizdata

“Open-source intelligence has always been crucial, but for most of the cold war it was neglected by western intelligence agencies,” says Calder Walton, a research associate at Cambridge University and author of the book Empire of Secrets, to be published in 2013. “That was the archetypal intelligence war: intelligence necessarily involved information that couldn’t be gained from any other source — human agents or telephone tapping.” That doesn’t mean covert intelligence was more effective, though: Daniel Moynihan, a former US senator, compared CIA reports gathered from secret sources with Soviet documents recovered after the fall of the Berlin Wall and found they significantly overestimated Soviet capabilities. But he discovered that western think tanks using publicly available material, such as the RAND Corporation, were much more accurate. US diplomat George Kennan estimated in 1997 that “95 per cent of what we need to know about foreign countries could very well be obtained by the careful and competent study of perfectly legitimate sources of information open and available to us”.

Excerpt from an article in Wired, the tech and futurism magazine, about a Swedish investment firm, Recorded Future, that is taking the use of social networks and other systems to new heights in its attempt to get a jump on the market. In the process, it sheds new light on how the intelligence-gathering process works.

Here’s another couple of paragraphs:

The 20 employees of Recorded Future aren’t foreign-policy experts. They aren’t traders either, but if you’d started using Recorded Future’s predictions to buy US stocks on January 1, 2009, you would have made an annual return of 56.69 per cent. (The S&P 500 had an annualised return of 17.22 per cent over the same period.) Between May 13 and August 5 this year, as markets behaved with vertiginous abandon, their strategy returned 10.4 per cent; in contrast, the S&P 500 lost 9.9 per cent of its value. They’re data experts: computer scientists, statisticians and experts in linguistics. And in the data, they think, lies the future.

All Recorded Future’s predictions, whatever the field, are based on publicly available information — news articles, government sites, financial reports, tweets — fed into the company’s own algorithms. The result, it claims, is a “new tool that allows you to visualise the future” — one that is changing how government intelligence agencies gather information and how giant hedge funds place bets. On its website, Recorded Future states: “We don’t grant interviews and we don’t issue press releases.” But behind closed doors, the company is developing the technology that has been described be one tech blog as an “information weapon”.

The businesses was founded by a chap called Christopher Ahlberg, a former member of Sweden’s special forces and a serious entrepreneur. In its own way, this article is just another example of how Sweden is not quite the socialist nation that it is sometimes said to be, either by its starry-eyed admirers or detractors. There is a lot of entrepreneurial zest up there in the frozen north, it seems.