torsdag 25 september 2014

Are you a fool or a statistician?

Why you need to know statistics

The world is a dangerous place, not least financial markets. One reason is humans' disposition to look for patterns and our less developed sense for sound statistical analysis. If you don't know the basics in statistics, you are constantly at risk of being fooled or lose money.

You have probably heard about plants trading currencies or rats trading oil futures and bonds. If not, maybe you have at least heard about the up-or-out trainee model at trading firms? All of these work... for a while.

The reason they work is because they are automatically trend following, or a sort of spontaneous correlation systems.

Assume the plant gets sunlight and water or added nutrition when its trades are profitable. Since, its trades are derived from the way it grows, a trend of growth and good trades will emerge. As long as the underlying market moves in long enough trends at a time and the model is calibrated in tune with the length and strength of the market trends, things will work out well for mr Green.

The same thing works with rats. And it does too with humans. Hire a bunch of uneducated kids and keep anyone who is profitable, no matter what his "models" or MO are. When somebody is on a roll, gradually give them a larger mandate, while losers are fired. Up or out. Simple.

There is always someone who happens to take his cues from the variables that are working right now. It might be macro, micro, earnings, cash flow, eye balls, weather, super bowl, presidential cycle, seasons, weekdays or just about anything. It works while it does, and then suddenly it doesn't

Students, traders and professional investors and their bosses all make the same old mistake of not understanding statistics, in particular the difference between correlation and causation. As human beings we are wired to see patterns in random data, to postulate causation from correlation; in addition we are blind to contradictory information and extra susceptible to supporting information (confirmation bias of our own preconceptions, beliefs and hopes).

Taleb does his best to educate his readers about the impact of outliers and non-trend events, in his books The Black Swan, Fooled by Randomness and Antifragile

Kahneman (Thinking, Fast and Slow) guides the reader through a maze of human psychological shortfalls when it comes to ordinary decisions about money, vacations and painful procedures. It all comes down to understanding statistics and inoculating oneself against the most common day to day mistakes.

Everyday you are bombarded with claims based on statistics from surveys and other research. "cancer from Z", "weight loss from X", "muscle gains from Y", "Republican candidate gets x %" and so on. The packaging is often convincing, the message delivered by an authority (not seldom a disguised charlatan), using convincing and confusing jargon, the setting scientific, the charts smartly cropped, the conclusions and advice strong, easy to understand and presented in the form of bullet points... and spiced with a bit of moral: "do the right thing". Add in a couple of (irrelevant) anecdotes delivered by ordinary Joes and the deception is complete...

...Unless you actually know statistics and see through the charade, the too small sample sizes, the lack of established theoretical causation, too short testing periods, circular reasoning, graphical correlation in nominal values but low statistical correlation in changes, scaling errors etc.

This is a funny comic strip about what you can learn from a course in statistics (or reading the right books):
xkcd


Poor statisticians and short memories

In addition to all of the above, our memories are too short. That is especially true in financial markets. Only the winners are left in the market and they soon leave too or lose (apart from a couple of 7 sigma guys who are smart and lucky and probably not as good as it looks superficially [Buffett e.g.]).

After 5 years of trending markets most investors forget why the market crashed the last time, assume it's not relevant anymore and begin to repress how bad it was - or even that the market ever crashed at all.

In 1929 the perhaps most famous economist of his time, Irving Fisher, stated that the stock market had reached a permanently high plateau... and then the market crashed. He said that despite being theoretically well versed in the devastating crashes of 1901, 1906 and 1919. If his memory was that short, what do you think your neighbour's is?

Once a generation have lived through a boom and bust cycle, or two, they lose interest in stock market speculation and never come back. That leaves only the new fools and the old foxes, just as in online poker rooms, in particular the new ones that keep popping up. They lure in newbies that soon get fleeced by the veterans that move from room to toom, playing 10s of hands simultaneously, basically robbing the unknowing masses penny by penny.

The losers leave, new soon-to-be losers keep arriving, perhaps making money for a while, enough to believe enough in themselves to make a big bet (correlation vs causation; they happened to make money, to catch a trend by luck, not beacuse they were smart), increase the bet further on drawbacks ("cheaper! always buy on dips!"), add leverage (loans), feel rich for a while, only to come crashing down when the pro:s have cashed out and there are no more newbies and other ignorant buyers left to support the market for the time being.

5 kommentarer:

  1. Good article. And I can identify with much of it.

    "up-or-out trainee" -- didnt know of this phenomenon. Interesting. I guess it's all about results and making money?

    SvaraRadera
    Svar
    1. Up or out is thought to address two issues: 1) only talented people rise in the organization (unfortunately, "talent" is only defined as "producing results using a black box model during a specific period in time on a specific market") and 2) motivate newbies and low-levellers to work harder and smarter, i.e., to become more ambitious (if there actually were a tangible process where 'more is better' this works; that's why there are bonuses. Regarding untangible investment processes in changing market places, it simply won't work long term. However, short term and in trending markets during a certain part of a cycle, it may work wonderfully. I would rather use a trend following statistical/mathematical model instead)

      Radera
  2. Thank you for the book recommendations. Very interesting! (PS: Ludvig's blog brought me here too)

    SvaraRadera