Misusing ESG Data: The Most Common Analytical Mistakes
How Do Investors Misuse ESG Data?
ESG data misuse is among the most common and consequential analytical mistakes in ESG investing. The form it typically takes: treating ESG scores as objective measurements comparable to financial ratios, relying on a single provider's methodology without understanding its assumptions, accepting estimated data as equivalent to reported data, and failing to account for temporal lags in annual ESG reporting. These errors propagate through ESG analysis — portfolio construction decisions, engagement priorities, fund selection — in ways that systematically distort conclusions. This article examines the specific analytical mistakes that arise from ESG data misuse and provides practical corrections for each.
ESG data misuse: treating ESG scores as precise, objective measurements equivalent to financial metrics — rather than as opinions based on incomplete, self-reported, estimated data with significant provider divergence — leads to portfolio construction errors, misguided engagement, and overconfident ESG analysis conclusions.
Key Takeaways
- ESG scores are opinions, not measurements. Berg-Koelbel-Rigobon (2022): 0.38-0.71 pairwise correlations between major providers. Two companies rated very differently by different providers cannot both be correctly rated.
- Estimated data in ESG databases is often not labeled as such — users don't know whether a specific metric reflects actual company disclosure or a model estimate. Scope 3 estimates have error margins of 50%+.
- Annual ESG scores are based on data published 6-18 months after the reporting period — controversies and improvements since the reporting period are not reflected.
- Large-cap bias: ESG scores systematically favor companies with comprehensive disclosure programs — independent of actual ESG performance. Small-cap and emerging market companies score lower partly due to disclosure quality, not actual performance.
- ESG score changes may reflect methodology revisions, not company behavior changes. MSCI has made significant methodology revisions that caused score changes unrelated to company behavior.
Mistake 1: Single Provider Reliance
The error: Using one ESG rating provider's scores as the definitive assessment of a company's ESG quality — making portfolio inclusion/exclusion, engagement priority, and ESG quality ranking decisions based on that single source.
The problem: As Berg-Koelbel-Rigobon established, major providers disagree substantially. If MSCI rates Company X as AA (leader) and Sustainalytics rates it as Medium Risk (average), at least one is wrong. Basing investment decisions on one provider means you're choosing which set of methodology assumptions to rely on — without knowing whether those assumptions are most appropriate for your analytical purpose.
The correction: For significant portfolio holdings, cross-validate across two or three providers:
- Where providers agree (both high, both low): higher confidence in the assessment
- Where providers disagree substantially: investigate the source of disagreement — it often reveals a genuine analytical question about the company that is worth investigating directly
Practical implementation: Most Bloomberg and FactSet terminals include ESG scores from multiple providers. The additional analytical step of comparing scores takes minutes for individual holdings and can reveal important divergence.
Mistake 2: Treating Estimated Data as Reported Data
The error: Using ESG database metrics without knowing whether the underlying data is company-reported or provider-estimated — treating both as equivalent quality.
The problem: ESG providers fill data gaps with industry averages, peer-group estimates, and proprietary models. These estimates have wide uncertainty bands (scope 3 emissions: ±50% for individual companies) that are not reflected in the precision of the score display.
Where estimation is most prevalent:
- Scope 3 emissions: most companies don't measure them; providers estimate using spend-based or physical activity methods
- Gender pay gap: before CSRD/ESRS S1 mandates, mostly estimated for companies that don't voluntarily disclose
- Supply chain labor practices: largely modeled from sector/geographic proxies
The correction: For your key analytical variables:
- Identify which metrics are directly reported vs. estimated
- For estimated variables, apply wider confidence intervals to conclusions
- For primary investment decisions, source directly from company disclosures when possible (CDP reports, sustainability reports, proxy statements)
How to tell: Some providers label data quality (reported, estimated, modeled) in their data export. Bloomberg ESG data includes disclosure scores and data quality indicators. If your provider doesn't label this, ask or assume key non-disclosed metrics are estimated.
Mistake 3: Ignoring Temporal Lag
The error: Using current ESG scores as if they reflect current company behavior — when they actually reflect behavior from 12-24 months ago, based on annual reporting cycles.
The problem: ESG scores are typically based on sustainability reports published in the current calendar year for the prior fiscal year. By the time the score is used (mid-year current), it may reflect behavior from 18-24 months ago.
Examples of harmful lag:
- A company involved in a major environmental incident in Q1 current year may still show high environmental scores based on last year's report
- A company that announced a major ESG commitment in Q3 current year won't show this in scores for another 12-18 months
- Governance changes (new independent board members, improved audit committee) are not immediately reflected in governance scores
The correction: Supplement annual ESG scores with real-time monitoring:
- Controversy monitoring (most data providers offer this as a separate feed)
- News monitoring for significant ESG events at major holdings
- SEC filings review for governance changes (proxy statements, 8-K reports)
- CDP annual questionnaire submissions (often ahead of sustainability report publication)
Mistake 4: Large-Cap and Disclosure Bias Unawareness
The error: Using ESG scores to compare small-cap vs. large-cap companies or developed market vs. emerging market companies — without accounting for the systematic disclosure quality bias that disadvantages smaller/EM companies.
The problem: Large companies have comprehensive sustainability reporting programs; small companies often don't. ESG providers score companies with more complete disclosures higher — partially independent of actual ESG performance. This creates a systematic large-cap premium in ESG scores that reflects disclosure quality, not necessarily ESG quality.
Practical consequences:
- An ESG-screened portfolio that uses minimum ESG score thresholds systematically underweights small-cap and emerging market companies — not because they perform worse on ESG, but because they disclose less
- Comparing an EM company's ESG score to a US large-cap's score without acknowledging disclosure differences is analytically misleading
The correction:
- Use sector-relative or region-relative ESG scores when comparing across different disclosure environments
- Supplement ESG scores with additional research for small-cap and EM holdings where data quality is lower
- Avoid hard ESG score cutoffs that effectively become disclosure quality thresholds
Mistake 5: Treating Score Changes as Company Performance Changes
The error: Interpreting an increase or decrease in an ESG score as indicating an improvement or deterioration in company ESG performance — when the change may reflect a methodology revision or updated data coverage.
The problem: MSCI and other providers periodically revise their ESG methodologies — changing the weight of specific indicators, updating scoring algorithms, or expanding coverage to new data types. These revisions cause ESG score changes that have nothing to do with company behavior.
2022-2023 examples: MSCI made significant methodology changes that caused many companies' scores to move substantially. Investors who interpreted these movements as company behavior changes made incorrect engagement decisions.
The correction:
- Monitor ESG provider announcements for methodology changes
- When a score changes significantly, check whether the company released new information or the provider changed methodology
- Morningstar and other independent ESG research services track major methodology changes
Mistake 6: Carbon Footprint Precision Overconfidence
The error: Reporting portfolio carbon footprint (WACI or absolute) to two or three decimal places as if the figure is precisely measured.
The problem: Portfolio carbon footprint calculations depend heavily on scope 3 emissions data — which is mostly estimated. For a portfolio with significant scope 3 exposure (most diversified portfolios), the true precision of the carbon footprint figure is ±30-50%.
The correct approach: Report carbon footprint with:
- Clear indication of which emission scopes are included
- Acknowledgment of the proportion of data that is estimated vs. reported
- A range rather than a point estimate for portfolios with high estimated data proportion
- Year-over-year comparison only when the measurement methodology is consistent
PCAF standards: The Partnership for Carbon Accounting Financials provides data quality scores (1-5) for different data sources — Level 1 (audited direct measurement) vs. Level 5 (sector estimates from revenue). Portfolios with mostly Level 4-5 data have very low precision in their carbon footprint figures.
Mistake 7: Not Reading ESG Methodology Documentation
The error: Using ESG scores from a data provider without having read or understood their methodology documentation.
The problem: Different providers make fundamentally different choices — some score companies relative to industry peers, others on absolute standards; some focus on disclosure quality, others on outcomes; some adjust for industry exposure, others use universal criteria. Using a score without understanding these choices means you don't know what question the score is answering.
The correction: For your primary ESG data provider:
- Read the methodology overview document (typically 20-50 pages; publicly available for most major providers)
- Understand: what is the primary scope (what does the score measure)? How is it weighted? Is it relative or absolute? How are data gaps handled?
Common Mistakes
Treating a higher ESG score as universally meaning a better company. A higher score means the company scores better on this specific provider's methodology — not that the company is objectively more sustainable.
Using ESG scores as hard pass/fail thresholds without methodology understanding. An ESG score threshold of 50/100 as a minimum has very different implications depending on which provider's methodology produced the score. Define thresholds after understanding methodology.
Comparing ESG scores from different providers as if they're measuring the same thing. MSCI ESG score is not comparable to Sustainalytics ESG risk rating — they use different scales, methodologies, and underlying questions. Do not treat them as equivalent.
Related Concepts
Summary
ESG data misuse is one of the most consequential and correctable analytical mistakes in ESG investing. The key corrections: cross-validate across multiple providers for significant holdings rather than relying on one source; distinguish reported from estimated data and apply appropriate confidence calibration; supplement annual ESG scores with real-time controversy monitoring to address temporal lag; use sector-relative and region-relative comparisons to account for disclosure quality bias against small-cap and EM companies; distinguish company behavior changes from methodology changes in score movements; and report portfolio carbon footprint with appropriate acknowledgment of data quality and precision limitations. ESG data is a valuable analytical input — but only when used with understanding of its limitations, not as a precise measurement equivalent to financial metrics.