Numbers that lie: Why benchmarks are a variable yardstick

In the second of our four-part series on private equity metrics, PEI examines how benchmarks are gamed from two different, relatively simple angles.

An illustration of two dice

“When a GP is presenting fund-level historical performance and talking about how they compare relative to some sort of arbitrary peer cohort, LPs will discount that presentation,” says Jennifer Choi, managing director of the Institutional Limited Partners Association.

Investor scepticism aside, most managers will be able to show you that their performance compares favourably with their peers. How is this possible? Benchmarking is gamed from two different, relatively simple angles.

First is the choice of comparable data set. There’s an assortment of private equity indices – both public and private market-related – such as Cambridge, Burgiss and PitchBook, that the industry can choose from. Within each are embedded different layers and ways of tracking portfolio and manager performance, each with their own flaws and limitations.

Some of these include data capture (or the inability to gather information on all PE funds in the market), survivorship bias (the exclusion of performance data for funds that no longer exist), non-reporting funds (how unavailable data is treated varies across providers) and frequent  modifications (some providers will adjust historical performance figures), according to a JPMorgan Asset Management report.

One executive from a private equity technology company – who subscribes to two prominent performance data providers – said there can still be huge disparities between them. GPs are at liberty to select whichever suits their story best.

Second is the choice of vintage year. What seems like a fixed data point is really anything but, and if a GP wants to place themselves into a different cohort for comparison, it can do so by changing its definition of vintage year. In its 2019 survey of institutional investors, eVestment asked how LPs define vintage year. While a majority opted for the date of the first drawdown, a little under a third chose the date of the first investment, while others opted for the date of the final close. Without a standard definition of vintage year, GPs have some optionality.

One often hears funds described as “top quartile” or “top decile”, but the definition is ambiguous, adds James Gelfer, a senior strategist at PitchBook Data. “For GPs, the peer group that you select is what’s important,” he says. “Oftentimes GPs will see themselves as having a certain strategy, while LPs might actually categorise the GP in a different basket of funds in their portfolio.”

What’s an LP to do? Using multiple benchmarks is generally advisable, with an awareness of the characteristics and limitations for each, market participants say. Comparing historical returns over a period is also the most straightforward approach.

To improve informational edge, LPs have in recent years begun deviating from the straightforward total value-to-paid-in and internal rate of return. Direct alpha, for example, is “more resistant to exploitation due to the external factor of a public market index”, argues PitchBook in its Benchmarks 3Q 2018 Report.

Direct alpha is directly calculated based on the series of PE cashflows that are capitalised by benchmark returns to the same single point in time, providing a better way to gauge the annualised returns of private market funds while accounting for the macro environment in which the fund is operating. When using direct alpha, LPs can also opt to use custom benchmarks that closely reflect the portfolio. Examples include industry-weighted S&P 600 for small-cap exposure or levered industry-weighted S&P 600 to reflect the financing actually used.

The challenge, notes Gelfer, is that nobody else uses it, making comparing fund performance more difficult. “There’s a push from many academic and industry professionals to go towards that, but it’s kind of fighting an uphill battle. For large pensions where alternatives is still a small component, it’s challenging to go several steps deeper on the complexity level of performance metrics. It’s a non-starter for many LPs.”