Ignoring Base Rates

One of the most insidious cognitive errors in investing is base rate neglect: the tendency to ignore or underweight the underlying statistical likelihood of an event when making decisions. Instead, we fixate on recent examples, compelling stories, and vivid outcomes. This leads investors to chase yesterday's winners, avoid yesterday's losers, and fundamentally miscalibrate their expectations about what's likely to happen.

Quick definition: Base rate neglect occurs when you ignore the actual statistical frequency of an event (the "base rate") and instead rely on memorable examples, recent performance, or emotional narratives to make decisions.

Key Takeaways

• Base rates are the underlying probabilities that govern outcomes; ignoring them leads to systematic errors in stock selection and market timing decisions.
• Recent winners feel safer than they are; recent losers feel riskier than they are. This is the essence of recency bias compounded with base rate neglect.
• The base rate of stock outperformance is low: fewer than 20% of active mutual funds beat the S&P 500 over rolling 15-year periods, yet investors continue to chase manager names and track records.
• Windfalls and sudden wealth often lead poor decision-making because people anchor to the exceptional event rather than the base rate of such events.
• Recognizing base rates forces you to think probabilistically: What fraction of companies that look like this one succeed? What percentage of investors following this strategy win long-term?

Understanding Base Rates

A base rate is the underlying statistical frequency of an event in a population. In investing, base rates answer questions like: How many high-growth tech companies ultimately create lasting shareholder value? What fraction of value stocks recover within five years? How often do company CEOs successfully pivot their industries?

The problem is that humans are poor at using base rates naturally. We prefer vivid, recent, and emotionally resonant information. If a tech stock exploded 300% last year, we remember that vividly. If a hundred tech stocks crashed 90%, we may forget them. This creates a skewed sample in our minds.

Consider a simple example: You read about a company trading at 1× sales with a charismatic CEO who is "disrupting an industry." Your emotional reaction might be, "This feels like an Amazon story." But the base rate for companies matching this profile is not favorable. The majority of cheaply valued, high-growth companies underperform, fail, or plateau. The few that become generational winners get all the attention. You're confusing the availability of a memorable outcome with its actual probability.

The Base Rate of Active Management

One of the clearest demonstrations of base rate neglect is how investors approach active management. The base rate is stark: studies from S&P Dow Jones Indices show that over 15-year rolling periods, fewer than 20% of large-cap mutual funds outperform the S&P 500 after fees and taxes. Over longer periods (20+ years), the underperformance rate exceeds 90%.

Yet trillions of dollars flow into actively managed funds. Why? Because investors ignore the base rate and chase the visible winners: the 3% of managers with five-star ratings or three consecutive years of outperformance. These are outliers. The base rate tells you that this manager, like 80% of their peers, will likely underperform the index going forward.

This is not random. Funds that outperform tend to outperform by 1–2% annually, while underperformers underperform by the same amount. With fees of 1% or more, the median active fund destroys wealth relative to a simple index. Yet the allure of "beating the market" is so strong that investors continuously update their belief that this time they've found the exceptional manager.

Base Rates in Stock Selection

When you're evaluating individual stocks, base rates are equally important. Consider a few examples:

Turnaround plays: A company has been struggling for years, just hired a new CEO, and promises a "complete transformation." The media narrative is compelling. But what's the base rate for corporate turnarounds succeeding? Historical data suggests only 10–15% of struggling large companies successfully execute a multi-year turnaround that significantly revalues equity. The rest decline further, get acquired, or muddle through. Yet investors regularly assign 40%+ probability to the turnaround narrative based on recent news and confidence in the CEO.

Momentum stocks: A stock has outperformed the market for the last 12 months, and financial media is calling it the "stock of the year." What's the base rate for a stock that has beaten the market by 50% in the last year to beat it again in the next year? Studies of momentum show short-term persistence (three to twelve months), but long-term reversal dominates. Within three years, yesterday's big winners tend to underperform. Yet investors pour money into momentum funds, trusting the narrative of the winning stock's "strong fundamentals" without checking the base rate of past winners.

Dividend stocks: A stock offers a 6% dividend yield in an environment where the average S&P 500 stock yields 1.5%. The narrative is, "You're getting paid well to wait." The base rate for such high-yielding stocks? Most have cut dividends within three years, and many have seen their stock prices fall alongside the cuts. The stock was cheap for a reason. Yet investors buy them as "income," ignoring the base rate that high yields often precede pain.

Ignoring the Base Rate of Your Own Decision-Making

This extends beyond stocks. What's the base rate for your own investment decisions? If you've selected five stocks this year, how many have beaten the market at the one-year, three-year, and five-year marks? Most active individual investors don't track this rigorously, so they don't know their actual base rate of success. When one of their picks succeeds, they remember it vividly and recall their brilliant reasoning. When three fail, they attribute it to "bad luck" or "market conditions," not acknowledging that their base rate is below market.

Buffett has long observed that most people would be better served by buying an index fund than trying to pick stocks. This is not because stock picking is impossible—it's because the base rate of beating the market through stock picking is so low that the expected value is negative after costs.

The Danger of Extrapolating Narratives

Base rate neglect intersects dangerously with narrative fallacy. A CEO has been featured in Fortune magazine as a visionary. The company's product is "innovative" and solving a "massive market." The base rate for innovative startups with massive addressable markets is that most fail or deliver mediocre returns. But the narrative is so compelling that investors assign 80% probability to success, ignoring that the base rate says 70% of funded ventures struggle.

Consider the wave of SaaS (Software-as-a-Service) startups that went public from 2010–2020. Many were hyped as the "future of enterprise software." The base rate for venture-backed enterprise software companies is that the majority never achieve sustainable profitability or positive cash flow. Yet investors poured capital into names like WeWork, Uber, and countless unprofitable SaaS firms as if the narratives made them exceptions to the base rate. Some became winners, but many destroyed capital.

Using Base Rates to Improve Decisions

To counter base rate neglect, impose discipline:

1. Ask the raw question: Before analyzing a stock's fundamentals, ask: What is the base rate for companies like this? Is it a turnaround? The base rate of successful turnarounds is ~10–15%. Is it a high-growth small-cap in a nascent industry? The base rate for such companies creating lasting value is ~5–10%. Is it a dividend stock yielding twice the market average? The base rate for avoiding dividend cuts is below 40%.

2. Use reference classes: When evaluating a company, place it in a reference class of similar companies. How many companies in that reference class (low-price-to-sales tech stocks, or disruption-story biotech names, or mature utilities trading at 12× earnings) have beaten the market by 10% or more over rolling five-year periods? The answer is: far fewer than you think.

3. Track your own base rate: Maintain a personal investment journal. Document your decisions and their outcomes. What fraction of your picks beat the market? What fraction have lost money? Over decades, this creates an unvarnished record of your base rate. Most investors who do this discover their rate is below average, which then informs a more humble, index-heavy approach.

4. Demand statistical evidence: When you hear a compelling investment narrative, demand base rate data. If a fund manager claims their stock-picking skill is exceptional, ask: What is the rolling five-year outperformance rate for managers like you (same asset class, investment style) in the data? Usually, the answer is: barely above 50%, which means they're not exceptional at all.

Real-World Examples

Example 1: The Theranos Base Rate

Theranos was a privately held startup that claimed revolutionary blood-testing technology. The narrative was incredibly compelling: a female founder, Silicon Valley backing, a massive addressable market, and media acclaim. Investors (including prominent venture capitalists and Walton family members) assigned enormous probability to success.

The base rate they should have considered: What fraction of private biotech companies making extraordinary claims about technology succeed? Historically, it's below 5%. The company had no published peer-reviewed results, faced regulatory hurdles typical of medical devices, and was operating in a capital-intensive industry. These base rates alone should have warranted extreme skepticism. Instead, narrative won, and capital flowed in. The company eventually collapsed in fraud.

Example 2: The Tech Bubble of 1999

During the late 1990s, internet companies with no profits, no clear business models, and negative cash flows were being valued at billions of dollars. Investors ignored the base rate: What fraction of unprofitable internet companies in a bubble environment have created lasting shareholder value? Historically, nearly none. Yet because the narrative was "The internet changes everything," investors suspended base rate thinking. Pets.com, Webvan, and countless others collapsed. The few survivors (Amazon, eBay) get the attention; the 90%+ that failed are forgotten.

Example 3: The Dividend Trap

An investor in 2005 noticed that certain mortgage REITs offered 10–12% yields while the S&P 500 yielded 1.5%. The narrative: "REITs are boring, mature investments. This yield is safe." The base rate: What fraction of mortgage REITs maintain their dividends during credit cycles? Very few. When the 2008 financial crisis hit, most cut dividends by 50–90%. An investor who bought based on the narrative and ignored the base rate suffered a double hit: capital loss plus dividend cuts.

Common Mistakes

1. Confusing availability with probability: You read about one company that became a 10-bagger after a CEO change. You then assume CEO changes are bullish. The base rate says most CEO changes are followed by median performance at best.

2. Extrapolating from outliers: A handful of stock-pickers beat the market over a decade. You assume you can too, ignoring that the base rate of individual investors beating the market (after taxes and fees) is below 10%.

3. Trusting narratives over statistics: The story of a company is compelling, so you assume the probability of success matches the quality of the narrative. It doesn't. Storytelling ability and business success are uncorrelated.

4. Forgetting survivorship bias: The companies you read about in success stories are the survivors. You don't read about the 50 competitors that failed in the same space. The base rate includes all competitors, not just the winners.

5. Updating on recent evidence too strongly: One strong quarter of earnings and you assume the turnaround is "confirmed." The base rate says single quarters are noise; you need years of evidence to confirm a fundamental change in a business.

FAQ

Q: If base rates are so low, should I just buy an index fund and never pick stocks? A: For most investors, yes. The base rate of beating the market after costs is so low that the expected value of stock-picking is negative. If you do pick stocks, you must acknowledge you're playing a low-probability game and size accordingly.

Q: Can I beat the base rate if I have special knowledge or skill? A: Perhaps, but prove it to yourself first with a journal. Track your actual returns versus the index for five years. If you're in the top 20% of your reference class (i.e., you're genuinely exceptional), then maybe continue. Most who think they're above average discover they're not.

Q: How do I know what the base rate is for a particular strategy? A: Use academic research, historical databases, and aggregated mutual fund data. S&P Dow Jones publishes annual "SPIVA" reports showing what fraction of funds beat their indices. The CRSP (Center for Research in Security Prices) has historical data on stock returns. Start with those.

Q: Does base rate analysis apply to bonds, real estate, and other assets? A: Absolutely. What fraction of real estate investors beat the market? What fraction of bond pickers outperform bond indices? The answers are similar to equities: far fewer than you think.

Q: Can I use base rates to time the market? A: Not reliably. The base rate for successfully timing the market is so low that attempting it is generally a negative expected value activity. A better use of base rates is to decide your strategic allocation and stick to it.

Q: What if I have a strong conviction about a stock that violates the base rate? A: Strong conviction is not evidence. It's a feeling. If your conviction conflicts with a low base rate (e.g., buying a high-yield stock that historically cuts dividends), size the position very small (1–2% of your portfolio) and acknowledge it's a high-risk bet, not a core holding.

Related Concepts

• Recency bias: The tendency to weight recent events more heavily than historical patterns; base rate neglect amplifies this.
• Narrative fallacy: The human tendency to create coherent stories from random data; we use narratives as a proxy for probability, ignoring base rates.
• Hindsight bias: After an outcome occurs, we retroactively assign high probability to it ("I knew that would happen"), even though the base rate before the fact was low.
• Survivorship bias: Visible successful outcomes (Amazon, Netflix) dominate our perception, causing us to underestimate the base rate of failure among similar-seeming companies.
• Confirmation bias: Once we've decided a stock is a winner, we cherry-pick evidence that confirms our thesis and ignore base rate data that contradicts it.

Summary

Base rate neglect is a silent wealth destroyer. It leads investors to chase narratives, overweight recent winners, and underestimate the likelihood of failure for compelling-sounding investments. The antidote is rigor: ask what the underlying statistical likelihood of success is for any investment decision, compare it to your emotional assessment, and let data drive your decisions.

The most sobering base rate in investing is this: The majority of individual investors underperform the market. If you're tempted to pick stocks, ask yourself whether you have credible evidence that you're in the exceptional minority. For most people, the honest answer is no, and index investing is the rational default.

The greatest investors (Buffett, Lynch, Munger) became great partly by recognizing base rates, avoiding the false narratives that seduced others, and having the discipline to act only when the odds were genuinely in their favor. That's not glamorous, but it's the path to long-term wealth.

Next

Continue to the next article: The Illusion of Control