The noisiness of financial factors

Stay informed with free updates

Simply sign up to the European companies myFT Digest — delivered directly to your inbox.

Last summer, three researchers published an interesting paper that hammered home the importance of good data, and how subtle input changes (inadvertent or deliberate) can radically alter the conclusion.

The paper — “Noisy Factors”, by Pat Akey, Adriana Robertson and Mikhail Simutin — showed how the returns of several big theoretical drivers of investment returns could vary wildly depending just on what timeframe one measured.

Here’s the synopsis:

The Fama-French factors are ubiquitous in empirical finance. We find that factor returns differ substantially depending on when the data were downloaded, and only a small portion of these retroactive changes is explained by revisions to the underlying data. We show that these changes have large effects in two widely-studied contexts: mutual fund performance and cross-sectional equity pricing. Model evaluation tests suggest that more recent vintages do not perform better. Our findings have significant implications for the integrity of finance research and underscore the importance of understanding the provenance of third-party data.

For people not on top of their quant historiography (FOR SHAME), the Fama-French referenced are Nobel laureate Eugene Fama and his frequent collaborator Ken French.

Back in 1993 they published a bombshell paper that showed how smaller stocks and cheaper stocks systematically outperformed the broader equity market in the long run, an apparent violation of Fama’s own Efficient Markets Hypothesis.

Fama and French proposed that these extra returns were compensation for an extra risk, and later extended their work to also show that strongly profitable and investment-liberal companies also underperformed. Together with the market factor (beta), this is known as the Fama-French five-factor model.

Since then, both academics and practitioners have refined these factors and supposedly discovered so many new ones that many now call it a “factor zoo”. And quite a few think that many supposed factors — or risk premia, as they’re sometimes called — are actually bogus figments of excessive data mining.

By excluding or including some slices of the market (eg microcaps) or being -cough- selective about the timeframe used, you can come up with spurious factors that look cool in a tenure-boosting paper but bomb in real markets — the finance industry’s own “replication crisis”. However, the Fama-French factors are considered canonical.

Akey, Robertson and Simutin don’t question the five-factor model per se — many other researchers have found similar results, and (to varying degrees) in international markets as well.

But they found that “changes to factor returns [in French’s online data library] are frequent, often substantial, and impact conclusions about first-order questions in finance”.

Until the code is made public, we do not believe that academic finance can justify the continued use of French’s factors. To be clear, nothing in this analysis speaks to the validity of the three-(or five-) factor model, only to this particular source of factor data. Moreover, we do not believe that there is a viable econometric or statistical solution that can salvage French’s factors. The evidence does not support the conclusion that the “noise”we document is, or can be reasonably approximated by, classical measurement error. Because the changes appear to be the result of intentional modifications to the code, it is unrealistic to assume that we can predict what change[s] might look like going forward.

French and Fama responded to this just before Christmas, in a paper we spotted last week thanks to Robeco’s chief quant strategist Piet van Vliet.

The title — Production of U.S. Rm-Rf, SMB, and HML in the Fama-French Data Library — hints that this is not a page turner. Even for the standards of academic finance this is esoteric stuff, exhaustively detailing the various data sources, their approach, methodology etc.

Fama and French don’t explicitly mention the Noisy Factors paper, but they do update the estimated strength for various factors, such as upgrading value (HML, or high-minus-low, in the table below) and downgrading size (SMB, or small-minus-big).

[Zoom]

However, as van Vliet’s colleague Matthias Hanauer pointed out on X/Twitter, the kicker is arguably in the last paragraph:

A final warning is in order. The details of factor construction are arguable, and there is no magic. After decades of experience, asset pricing research clearly recognizes that factor models, no matter how constructed, leave holes in the explanation of expected asset returns. Moreover, parameter instability and statistical estimation error combine to imply that expected return estimates for specific assets or portfolios from asset pricing models are unreliable. The appropriate caveat is: use at your own risk.

The statistician George Box once said that “all models are wrong, but some are useful”. Factors aren’t laws of physics, but they are a handy framework for understanding markets.

However, as Fama and French note above, no model is better than the data you use, and no dataset is without its foibles. So caveat quantor.

Further reading:
— The hidden ‘replication crisis’ of finance (FT)
— A quant’s winter tale (FTAV)