The Index Fund Revolution — What the Data Says About Active vs Passive

The Original Heresy

When Vanguard launched the First Index Investment Trust in 1976 — the first index fund available to retail investors — the Wall Street reaction was contempt. It was called “Bogle’s Folly.” The fund aimed simply to match the performance of the S&P 500 index rather than to beat it. Settling for average returns seemed like an admission of failure. The fund raised $11 million against a target of $150 million in its initial public offering. It was widely expected to close.

John Bogle’s argument was arithmetic, not philosophy. If the aggregate of all investors in a market holds the market portfolio (because someone must hold every share that exists), then before fees, the average actively managed dollar in that market will earn the market return. After fees — which are real and compounding — the average actively managed dollar must earn less than the market return. The only question is how much less.

The fees Bogle was attacking were substantial. Actively managed mutual funds in the 1970s charged 1-2% annual expense ratios; some charged additional sales loads of 5-8% upfront. Against a long-run equity return of 7-9% real, a 1.5% annual fee is not a small drag. Over a 30-year investment horizon, the difference between a 7% and 5.5% annual return (the market return and the return after a 1.5% fee) is the difference between tripling and roughly doubling your inflation-adjusted wealth.

The Arithmetic of Active Management

William Sharpe formalized Bogle’s intuition in “The Arithmetic of Active Management” (1991). Before costs, the average actively managed dollar must earn the market return — this is a mathematical identity, not an empirical claim. The aggregate of all active investors is the market; the market must earn the market return; therefore the average active investor earns the market return.

After costs, the average active investor earns less than the market return by the amount of the costs. An investor who holds the market portfolio and incurs no trading costs earns the market return exactly. An investor who actively manages — incurring transaction costs and management fees — earns the market return minus those costs. The only way to beat the market net of costs is to be above-average before costs by an amount exceeding your costs.

Some active managers are above-average before costs. A small minority are above-average by enough to cover their costs — these are the genuinely skilled managers who can identify mispriced securities. The question is how large this minority is and whether you can identify them in advance.

What the Data Shows

SPIVA (S&P Indices Versus Active) produces annual scorecards comparing active fund performance to relevant benchmarks. The consistent findings over twenty years of data:

Over one year, approximately 50-60% of active US large-cap managers underperform the S&P 500. This is close to what random chance plus a small cost drag would predict.

Over five years, approximately 75-80% underperform.

Over fifteen years, approximately 90% underperform.

The pattern holds across asset classes and geographies, though the degree of underperformance varies. Large-cap US equities (the most efficient market segment) show higher rates of underperformance than small-cap or international equities (less efficient segments where research may have more value). But even in less efficient segments, the majority of active managers underperform over long periods.

The persistence question is critical: are the managers who outperform in one period more likely to outperform in the next? If skill is persistent, you could identify it from past performance and expect it to continue. Studies consistently find very weak persistence — past outperformance predicts future outperformance only weakly, and at short time horizons, barely above what chance would predict. The managers who happened to be in the top quartile this year are only slightly more likely to be in the top quartile next year.

Why Active Management Persists

If the average active manager underperforms, and the top managers are hard to identify in advance, why does active management command trillions in assets?

Marketing and narrative. Individual investors are attracted to stories of outperformance. Investment management is sold with stories of superior research, seasoned professionals, and market-beating performance. The stories are compelling. The data is complicated.

Survivorship bias. The funds that have underperformed and closed are not in the databases investors look at when evaluating past performance. The surviving funds have, by definition, survived — they tend to have records better than the full universe of funds, most of which no longer exist. Performance looks better than it was because the failures have been removed.

Short evaluation horizons. Individual investors typically evaluate performance over 1-3 years, not over 15-year periods where underperformance becomes most visible. An active manager can appear competitive over short periods by luck; the selection for skill requires long horizons.

Agency and career incentives. Institutional investors (pension funds, endowments) are managed by investment committees and consultants who are evaluated in part by their ability to explain their decisions to stakeholders. Holding index funds requires no expertise and provides no narrative. Selecting active managers provides a story and a person to blame if performance disappoints. The career incentive points toward active management even when the expected return for beneficiaries points toward indexing.

Hope and the lottery effect. Even if the expected outcome of active management is negative (after fees), the distribution includes the possibility of exceptional managers who produce dramatically above-average returns. For investors who overweight this possibility — consistent with prospect theory’s probability weighting — the expected value calculation is unfavorable but the distribution feels attractive.

The Bogle Revolution’s Legacy

The index fund’s growth from Bogle’s Folly to the dominant investment vehicle has been extraordinary. Passive funds now hold more assets than active funds in the US equity market. The fee compression has been dramatic: S&P 500 index fund expense ratios have fallen from 0.2% in the 1970s to 0.03% or less today. Vanguard alone manages over $7 trillion; BlackRock’s iShares (also primarily index-based) manages similarly.

The shift has had secondary effects. Fee competition has compressed active management fees. The distribution of assets toward passive funds has raised questions about market efficiency (if fewer active managers are conducting price discovery, does that impair the mechanism by which information gets into prices?) and about corporate governance (large passive holders own all companies; do they have the incentive or ability to discipline underperforming management?).

The market efficiency question has an empirical answer so far: markets have not become observably less efficient as passive share has grown. The corporate governance question is more live — passive holders have adopted proxy voting guidelines but their engagement model is necessarily different from concentrated active holders who have strong incentives to push for value-maximizing changes.

What Bogle Got Right and What He Underweighted

Bogle’s core insight — that costs are the primary determinant of net investment returns over the long run, and that minimizing costs by holding the market is the best strategy for most investors — is supported by fifty years of data. His conviction that the arithmetic of active management condemns the average active investor to underperformance before considering any other factor was correct and has been confirmed.

What the simple narrative underweights: there are genuinely skilled active managers, and in less efficient markets (small-cap, emerging market, private equity, venture capital) the dispersion of outcomes is larger and the opportunity for skill to add value is greater. The case for passive is strongest in efficient large-cap liquid markets and weakest in illiquid, opaque markets where information advantages are larger and sustainable. Institutional investors with access to top-quartile managers in less efficient markets may rationally choose active strategies. The retail investor in US large-cap stocks almost certainly should not.

The broader lesson is not passive vs active as ideology. It is: be skeptical of claimed edges, account for costs honestly, evaluate performance over long enough horizons to separate skill from luck, and don’t pay for alpha you’re not receiving.