Computation complexity makes beating Renaissance's Medallion not easy

Presumably the best hedge fund in the history - Renaissance Technologies' Medallion Fund - has averaged 35% annual return after fees over 11 year.

Since RenTec shares approximately 40% of the profits generated by the fund on top of the 5% annual management fee, gross returns should be closer to 50% per year.

I have been wondering how hard it is to generate such high returns independent on market conditions. As I mentioned some time ago, with the perfect future knowledge you can achieve virtually any return you want ;)

Unfortunately, even if one is able to score big once, repeating the success may be challenging - take John Paulson's story as a cautionary example.

Hence, Medallion's persistence is even more interesting.

As an experiment I attempted to generate purely random investment strategy that satisfies tree criteria:

1. annual strategy return is positive
2. annual strategy return is higher than market return over the same period
3. annual strategy return is 50% per year or slightly higher

These three conditions must be met at any time.

The strategy tested operated on S&P500 index.

Fig. Random Test - price, daily returns, volatility, returns 1Y, difficulty

Even that 50% annual return is minuscule in comparison with the maximum theoretical return, generating successful strategy may be quite computationally challenging at times. Over the last 4 years, the complexity varied by some 17 000 (seventeen thousand times)!

Fig. Maximum Theoretical vs Desired Returns 1Y

Fig. Random Test - difficulty of generating desired strategy

In addition, even that complexity is inversely correlated with volatility, the correlation is not perfect:

> cor(returns[,"intensity"],tail(v.year,l),method="kendall")

[1] -0.8658321

Definitely market volatility makes generating high returns easier, but seems not to be the only factor at play.

Note: the strategy was generated on the daily data, while Medallion most probably operates in the HFT regime. Higher frequency increases the maximum theoretical return and - at least theoretically - makes generating higher returns easier.

Fig. Random Test - Volatility

It is also worth mentioning that the market conditions over the recent 9 months seem to paradoxically be pretty challenging for generating high returns. What's more, complexity has been increased for quite some time now. 

Is Fed/ECB/BoJ's quantitative easing responsible for that? Is it possible that this situation leads to the herding behavior among investors? What will happen when they start unwinding positions in a situation when assessment of liquidity may be misleading for high activity of high frequency traders?

Fig. S&P500 over 30 years - note falling volume

[ R code ]