Trees Don’t Reach the Sky; Perspectives on Regression to Mean

Success = Talent + Luck; Great Success = a little more talent+ a lot of luck

This is how Daniel Kahneman, Nobel laureate in economics, responded on being asked for his favorite equation. Napolean Bonaparte had once quipped “I’d rather have a lucky general than a good one.”

Regression to mean (RTM) is one such phenomenon which in a sense is manifestation of luck. Understandably RTM provides the philosophical underpinnings for many decision-making systems. It can be the savior especially when we are tempted to extrapolate current trends into the future – nudging us towards better decisions.

Tomorrow is another day

Natural systems offer the best and most explicit examples of regression to mean. Trees stop growing at some juncture because the rate of growth after remaining above average for a sustained period has to regress to the mean – in this case zero growth. It has been scientifically established that children of very tall parents tend to be less tall (and vice versa).

In practical term RTM suggests that performance in any area is unlikely to go on improving or growing worse indefinitely. We swing back and forth in everything we do, continuously regressing towards what will turn out to be our average performance. Perhaps this is what explains the lack of consistency in performance of fund managers. Similarly, it is rare to see the progeny of top cricket players, or Bollywood super stars to reach the heights achieved by their parents. When inflation in an economy hits extreme levels it is only a matter of time before it starts moderating.

“Tomorrow is another day” (asserted by the gritty, main protagonist Scarlett O’Hara when she is a in a miserable condition, in the US civil war era novel “Gone With the Wind”) is amongst the most powerful and immortal words in literary history. It rehashes what most religious and philosophical scriptures say that times – whether good or bad, always change.

Parents, offspring and correlation

Regression to mean had been so counter intuitive and seemed so strange to the human mind that as a concept it was discovered two hundred years after the discovery of differential calculus.  Francis Galton, the high-bred British scientist and explorer had first observed this effect during his experiments on size of seeds in successive generations. He found irrefutable evidence that the offspring did not resemble the parent seeds in size. Instead the next generation would always move towards mediocrity. It would become less large if the parent seeds were large, and larger than the parents if the parents were small.

Indeed, it was Galton’s analysis which eventually led to the concept of correlation which measures how closely two series vary relative to one another. The two series here can be movement in value of rupee and stock price of Infosys, or rainfall and crops, economic growth and stock index changes, or inflation and interest rates.

As he proceeded with these experiments and as he involved some prominent statisticians of the era Galton hit upon the notion that regression to mean occurs whenever the correlation between two measures is imperfect.

It exists even if we are oblivious to it

So often, experts on business news channels dish out something like “US Fed rate hike” as a reason for  “the xx % decline in the Sensex this week’’ when a much better explanation can be that “the Sensex fell since it had risen by yy% last week”.

The issue is that we dislike, and fear our inability to explain an event as much as we hate admitting the significance of luck in our life. The associative memory in human mind with its urge to dig up causal explanations is the prime mover here. This is the reason why we are at times unable to apply RTM and this is why it took so long for RTM to be discovered.

Stock markets and RTM

If the P/E (Price/Earnings) multiple of a stock has been growing rapidly, to levels substantially ahead of historical averages, then even a layman can guess that the stock may face a correction. Or vice versa- if some stock has seen its valuation multiple get crushed to abysmally low levels then the stock may be ripe for a sharp uptick. Of course there are many market participants who are unable to read the tea leaves, extrapolate the current run up or decline in a stock well into future, and thus end up burning their fingers.

In the field of investment management there are two strategies that are rooted in RTM. Many investors follow regression to mean as the main driver for their forecasts of economic growth, industry trends, company earnings, valuation multiples, or of the stock prices directly. On the other end of the spectrum, there are momentum investors who work with expectation that the stock price or valuation multiple will defy regression to mean in near future and place their bets accordingly.

The Academic Evidence

In 1985, economists Richard Thaler and Werner DeBondt analyzed the three year returns of more than a thousand stocks from 1926 to 1982. Stocks that had risen more than or fallen less than the market average in each three-year period were categorized as winners. Similarly stocks that fell more than or grew less than the market average in a three-year period were termed as losers. Then the average performance of each group was calculated over the subsequent three years. The results demonstrated regression to mean at work in the stock markets in unambiguous terms. Over this period of 1926-1982, the loser portfolios outperformed the market by 19.6% three years after portfolio creation. Winner portfolios on the other hand trailed the market returns by 5%.

The above results remarkable as they are, do seem logical if seen in light of the fact that stock markets – which are effectively melting pots of human brains – due to their inherent behavioral idiosyncrasies tend to overreact to any new information in the short term. As a result, once the new information reveals its full imprint and after market participants have done proper impact analysis the stock moves much lesser than it did when the initial set of information came up.

Handle with care

Spare a thought for some investors who during the great crash, after seeing US stocks slump by 50% between second half of 1929 and early 1930, put serious money to work in the stock market assuming that RTM would take center stage soon, only to see 80% of their investment vanish into thin air over next three excruciatingly painful years.

So why is it not easy to use RTM to become rich by investing in markets?

First, RTM itself can be predicted but it is difficult to predict the timing of the beginning of regression. Thus, before the regression happens the gap from mean can widen causing debilitating losses. Remember what Keynes had to offer ‘Markets can remain irrational longer than one can be solvent”.

Further, mean regression often drives extreme movement on the other side. Thus P/E multiple of a stock when retracing from a level of 21x towards its long-term average of 15x may keep on sliding down till 8-9 x instead of settling at 15x. In addition, it can continue fluctuating randomly around the mean for a long time.

Finally, the mean itself can keep changing and may even have some subjectivity. If recent data points are settling towards one extreme then mean can shift over time to present a new normal replacing the earlier norm. For an investment professional a dilemma can be as to which mean to take – over 3 years, 5 years, 10 years or 20 years?

Practical reasons are that vs natural systems man made systems become complicated due to presence of too many neural drivers and hence RTM does not work systematically. Human psyche as we all know is less dependable than the nature, despite all the latter’s vagaries.

RTM is a good tool and a logical starting point in many situations but is must be looked at in context of changing realities versus history. Peter Bernstein in his famous book “Against the Gods” quoted Galton as urging us to “revel in more comprehensive views than the average”.