The Factor Zoo Problem in Quantitative Investing

The factor zoo problem is the proliferation of hundreds of published return factors—momentum, value, quality, volatility, profitability, and dozens more—many of which appear to predict returns only because researchers tested so many hypotheses that a few were bound to succeed by chance alone, not because they represent genuine risk premiums.

The Multiple-Comparison Problem at Scale

Quantitative investing rests on the premise that certain characteristics—valuation ratios, momentum, profitability, debt levels—systematically predict returns. If a stock is cheap (high book-to-price), it tends to outperform. If a stock has high earnings quality, it tends to outperform. These are the “factors,” and they represent either compensated risks (investors demand a premium for holding risky stocks) or behavioral anomalies (irrational investors misprice stocks in predictable ways).

The crisis emerges when researchers test hundreds of potential factors against historical data. Each factor hypothesis can be formulated in multiple ways: momentum measured over 6 months, or 12 months, or 24 months? Value measured as price-to-earnings, or price-to-book, or enterprise-value-to-earnings? Quality measured by ROE, or ROA, or cash-flow stability? As the number of tests multiplies, the probability that at least one will show a “significant” relationship by pure chance grows rapidly.

This is the multiple-comparison problem (or “p-hacking” in popular terms). If you test 100 independent hypotheses at the conventional 5% significance level, you expect 5 of them to appear to work simply due to random noise, even if no real relationship exists. With 400 published factors and countless variations on each, the statistical contamination is severe. A factor that appears to deliver a 5% annual excess return over the past 20 years may owe that outperformance entirely to curve-fitting, not to an exploitable edge.

Survivorship and Publication Bias

Academic journals have strong incentives to publish novel, positive results. A researcher who discovers a new factor that beats the market over 1980–2020 publishes it. A researcher who tests 50 factors and finds 49 of them worthless never publishes, or publishes one negative meta-analysis that is less visible than the positive discoveries. This publication bias creates a catalog of “successful” factors that vastly overstates the true win rate.

Additionally, factors published decades ago have already benefited from decades of out-of-sample performance. A momentum factor discovered in 1990 had 20–30 years of live market testing by 2020. The factors in academic journals are thus a survivorship-biased set: the ones that happened to work and were promoted by successful academics or fund managers. The hundreds that failed were never published.

When a researcher audits all factors mentioned in published papers—including those that are rarely cited—and applies corrections for multiple testing, the statistical significance of most factors shrinks dramatically or disappears entirely.

Transaction Costs and Scale Constraints

Paper backtests ignore the real costs of trading. A factor based on buying the smallest-cap stocks with the lowest valuations might show a 6% annual excess return in simulation. But actually executing that strategy means hitting micro-cap equities with low liquidity, wide bid-ask spreads, and high trading costs. The empirical (slippage-adjusted) return shrinks to 2%. If a factor generates frequent rebalancing—flipping 30% of the portfolio monthly—the costs compound, and the after-cost return may approach zero.

Shorting, essential to many factor strategies, incurs borrow costs that can eat 2–5% annually for illiquid names. A long/short factor that shows 8% annual alpha in a backtest might deliver 1% after real-world shorting costs. Many factors also face liquidity constraints: if the strategy requires holding 2% of average daily volume, the portfolio size is capped, and larger capital bases cannot achieve the backtested returns.

The Collapse of Factors in Live Trading

The most damaging evidence is live performance. Factors that appeared robust in historical data often fail once capital is deployed. The classic case is the “value factor”—long cheap stocks, short expensive ones—which delivered strong historical returns over decades. Yet in the 2010s, particularly the second half, value massively underperformed growth. Quant managers who had bet on value’s persistence lost money and market share. The factor had been real once; it is less clear whether it was real or just timing luck.

Similarly, low-volatility strategies—the notion that lower-volatility stocks deliver higher risk-adjusted returns—showed strong historical evidence. Yet implementation often disappoints, especially when volatility regimes shift (e.g., in 2020 pandemic volatility spike, low-volatility stocks proved costly to hold).

These reversals are not always evidence that factors are false. Markets are dynamic, investor preferences shift, and a factor can persist for decades and then fade. But they do suggest that the in-sample out-performance was partly overstated, or that structural changes rendered the factor less valuable.

Filtering Signals: What Serious Quants Do

Institutions managing billions in factor-based strategies employ several filters to avoid the factor zoo trap:

Economic Rationale
A factor must have a plausible economic explanation. If you find that stocks with names starting with “A” outperform stocks with names starting with “Z,” that is pure noise—no economic reason exists for the relationship. Genuine factors rest on either risk (leverage, illiquidity, business cyclicality) or a behavioral story (investor mispricing, limits to arbitrage). Momentum is credible because of behavioral inertia; a random technical indicator is not.

Out-of-Sample Testing
Backtest the factor on a hold-out period the researchers did not see during development. If a factor works for 1980–2010 but fails for 2010–2020, it was likely overfit to the first period. Walk-forward testing (rolling windows of development and testing) is more rigorous: develop a factor on the first 5 years, test it on the next 2 years, then roll the window forward.

Robustness Across Geography and Asset Classes
If a factor works in US large-cap equities but not in international equities or small-cap equities, it may be a historical accident. Factors that work globally—value premium in both the US and Japan, momentum in both equities and commodities—are more credible because they are harder to explain away as region-specific noise.

Cross-Validation and Significance Correction
Use statistical methods to correct for multiple testing (e.g., False Discovery Rate correction). If 400 factors are tested and 20 show nominal significance, how many survive a 5% FDR threshold? Usually far fewer—perhaps 3 to 5. These survivors are the true candidates.

Ensemble and Diversification
Rather than betting on a single factor, combine many factors with low correlation. Even if some are noise, the ensemble may be more stable. This mirrors the principle of diversification applied to factors themselves.

The Living Zoo

The factor zoo remains unsolved. New factors are published annually. Some genuinely represent new sources of return (e.g., climate risk, ESG risk, alternative data signals). Others are noise. The challenge for quantitative managers is to distinguish signal from luck without falling prey to the same multiple-comparison bias that afflicted the published literature.

Practical experience matters: managers who have lived through market cycles, regulatory changes, and strategy reversals gain intuition about what is durable and what is ephemeral. But even seasoned teams can be fooled by regime shifts. The only certainty is that most published factors will not deliver their backtested returns to new capital, and skepticism toward novel factor discoveries is justified.

See also