Private vs Public Indices

Quick definition: Public indices are developed and maintained by commercial index providers (MSCI, Russell, S&P) and licensed to multiple fund managers; private indices are proprietary methodologies developed by individual fund managers exclusively for their own funds.

Key Takeaways

• Public indices serve multiple clients and must remain neutral and transparent; private indices are created by fund managers for their own strategies and may reflect competitive advantages
• Vanguard pioneered large-scale proprietary index development, creating "in-house" indices to reduce licensing fees and customize methodology to their investment philosophy
• Private index advantages include lower licensing costs, customization flexibility, and potential competitive advantages; disadvantages include less external scrutiny and potential lack of transparency
• Public indices benefit from broad market adoption and standardization, creating network effects and benchmark recognition; private indices face challenges in gaining investor confidence
• Understanding the public-versus-private distinction helps investors evaluate fund manager credibility and assess whether customized indices represent genuine improvements or just marketing differentiation

What Are Public Indices?

Public indices are developed by third-party index providers like MSCI, Russell Indices, S&P Dow Jones, and FTSE. These providers develop the methodology, maintain ongoing index data, handle reconstitution, and make governance decisions about index changes. Multiple fund managers license these indices and create funds that track them.

Public indices must operate under high standards of transparency and governance. Index methodologies are published in detailed documentation. Changes to indices are announced well in advance, allowing stakeholders to understand and prepare. Index governance includes committees that review and approve changes, ensuring a rigorous decision-making process.

The largest public indices—like the S&P 500, Russell 2000, MSCI USA, and MSCI Emerging Markets—are tracked by hundreds of billions of dollars in passive investments across thousands of funds. This scale creates a network effect: the more money tracking an index, the more important it becomes as a benchmark, and the more credible it is as a standard.

Public indices also enjoy the benefit of external scrutiny. Academics, traders, investors, and regulators all monitor public indices. This attention helps keep index providers honest and ensures that methodologies remain sound.

What Are Private Indices?

Private indices are proprietary methodologies developed and maintained by individual fund managers exclusively for their own funds. Vanguard pioneered this approach in the 1990s, developing its own U.S. large-cap, mid-cap, and small-cap indices instead of licensing Russell, S&P, or MSCI indices. Over time, Vanguard extended proprietary indices to international, emerging markets, and bond categories.

A private index is owned and controlled by the fund manager. The index methodology is proprietary, though not necessarily secret. Vanguard publishes detailed documentation explaining how its proprietary indices work, much like public index providers do. However, Vanguard controls the methodology, can change it at will (subject to regulatory constraints), and does not license it to competitors.

Private indices have two key motivations: cost reduction and customization. By developing indices internally, Vanguard avoided paying licensing fees to MSCI, Russell, and S&P. For a manager of $1+ trillion in passive assets, the cumulative licensing fees saved can be billions of dollars annually. Vanguard passes much of this savings to investors in the form of lower fund expense ratios.

Additionally, Vanguard could customize its indices to reflect its investment philosophy. Vanguard emphasizes broad diversification and minimal turnover. Its proprietary indices were designed to achieve these goals more directly than public indices, which often prioritize liquidity or other factors.

Vanguard's Proprietary Index Strategy

Vanguard is the most prominent example of a fund manager using private indices at scale. The company created the "Vanguard Total Stock Market Index" that covers the entire U.S. stock market, from the largest mega-cap to the tiniest micro-cap stock. This is broader and lower-turnover than any public large-cap or small-cap index.

Vanguard's international indices similarly avoid licensing fees while achieving similar exposures to public indices. By developing proprietary indices, Vanguard created a sustainable competitive advantage: lower costs than competitors who license public indices. This advantage has persisted for decades.

Importantly, Vanguard's proprietary indices are not obviously better than public alternatives—they're similar in philosophy and composition. The primary advantage is cost. However, this is not trivial: saving 0.01–0.03% annually in licensing fees compounds to meaningful outperformance over decades.

The Trade-offs of Private Indices

Despite the cost savings, private indices have trade-offs:

Lack of external standards: Public indices benefit from regulatory oversight, academic scrutiny, and market acceptance. Private indices have only the fund manager's own governance. If Vanguard makes a poor methodological decision, there's no external body of experts to question it. This creates a potential conflict of interest: Vanguard controls the index that its own funds track.

Transparency concerns: While Vanguard publishes its proprietary index methodologies, they are not subject to the same level of external review as public indices. An investor worried about index methodology quality might prefer the transparency and oversight of public indices.

Portability concerns: If an investor wants to leave a fund family and move to a competitor, moving to a fund tracking a different private index may involve transaction costs and market timing risk. Moving between public indices is cleaner because multiple managers track the same public index.

Benchmark comparisons: Professional investors often compare their performance to widely recognized public indices. A portfolio tracking Vanguard's proprietary total stock market index cannot easily be compared to a Russell-tracking or S&P-tracking benchmark because the indices have different compositions.

Talent and expertise: Public index providers employ teams of experts dedicated to index methodology and maintenance. They invest in research, technology, and governance. A fund manager's proprietary index competes for talent and investment budget with the fund manager's core business (asset management). This can lead to less rigorous governance of the index.

Other Fund Managers with Proprietary Indices

Vanguard is not alone. Other large fund managers have developed private indices:

BlackRock has developed proprietary indices for some of its iShares ETFs, though it also uses public indices from MSCI, Russell, and others. BlackRock's philosophy balances proprietary customization with the credibility of public indices.

Dimensional Fund Advisors has built its entire business model on proprietary indices designed to capture specific factor premiums (value, small-cap, momentum) that it believes the market has overlooked. Dimensional's indices are more specialized than broad-market public indices and reflect sophisticated investment research.

Schwab has developed proprietary indices for its ETF lineup, similar to Vanguard's strategy of reducing licensing costs.

These private index developers argue that their proprietary methodologies offer improvements over public indices. Dimensional claims that its factor-focused indices capture value and small-cap premiums more efficiently. BlackRock argues that its proprietary indices offer better diversification or lower turnover. However, independent verification of these claims is difficult because the indices are proprietary.

The Ongoing Competition Between Public and Private Indices

The competition between public and private indices is an ongoing dynamic in the passive investing industry. Public index providers argue that their indices offer transparency, governance, and market acceptance. They contend that proprietary indices, no matter how well-designed, lack the external credibility of a widely adopted standard.

Private index developers counter that they can innovate faster and make decisions that benefit their investors directly, without the committee-based governance that slows public index providers. They argue that proprietary indices, by reducing costs, already provide a concrete benefit that passive investors value.

Over time, this competition has led to improvements in both. Public index providers have become more responsive to investor feedback. Private index developers have become more transparent about their methodologies. The net result is that passive investors benefit from competitive pressure improving index quality and reducing costs.

How it flows

Next

Understanding index providers and their methodologies—both public and private—requires knowing which provider or index is most appropriate for different investment objectives and situations.