Performance anxiety: the quest for better benchmarks

Measuring and benchmarking performance has long been a headache for private equity investors. Could relative performance measures ease some of the pain – and help LPs make better asset allocation decisions? Amanda Janis reports

Private equity investors have been complaining about IRRs for over a decade. 

“You can’t eat IRRs,” was one of the first – and most memorable – such limited partner quotes Private Equity International ever reported. That came from Sandra Robertson – now CIO of the Oxford University endowment and then an investor for Wellcome Trust – during the 2001 EVCA annual meeting. 

If you go to the grocery store, what do they want – cash or IRR?

Réal Desrochers

Another favourite: “I always say to the board: If you go to the grocery store, what do they want – cash or IRR?” That was Réal Desrochers – then the private equity head for CalSTRS and now for CalPERS – speaking to PEI back in 2007.

The chorus of voices in favour of cash-on-cash or other absolute return measures like time-weighted returns (TWR) has grown louder over the years. Some of investors’ main gripes have been that IRRs are subject to valuation bias, and that different timing of cash flows can distort returns. So too can the sequencing of investments, with large wins or losses early on greatly impacting a fund’s IRR. As one investor put it, managers “know how to work their IRR”.

WHEN IS A 2x NOT A 2x?
Despite this, most LPs will still want to look at IRRs as well as multiples. “I always say you need both,” says Christian Diller, partner and co-founder at Montana Capital Partners. “IRR is a measure that says something about how long it takes to get your money back; the multiple gives an idea of how much it is.”

To illustrate his point, Diller explains that an investment returning 2x in 2006 or 2007 may have had a big IRR, but an investment returning 2x today would probably have a much lower IRR. “Right now it takes a little bit longer [to exit] and because of that IRRs are much lower than investments exited in 2006-2007.”

Investors also use both measures to benchmark performance against similar funds tracked by a number of commercial databases including Thomson One (formerly Venture Economics) and Cambridge Associates. 

“You definitely want to track how your fund is doing relative to other similar funds; that’s what people do with IRR and multiple of invested capital,” says Steven Kaplan, a scholar who works often on private equity performance issues. Kaplan, who is the Neubauer Family Distinguished Service Professor of Entrepreneurship and Finance at University of Chicago Booth School of Business, says choosing the right database provider to create a benchmark is key, however – because each data set has its own limitations.

Montana’s Diller agrees. “It’s good to use these indices as a benchmark or also to derive performance numbers for private equity. On the other hand, I would also certainly look at the composition of these indices because if you are a European investor, for example, a lot of these indices might not reflect what you have in your portfolio.”

Jenny Fenton, Altius Associates’ co-founder and COO, adds: “There’s a number of different indices these days and they all have their flaws. In addition to industry benchmarks, a lot of clients also have an internal benchmark, which is typically some sort of public market equivalent.”

The use of IRRs and multiples to track private equity fund performance is clearly entrenched among fund managers and investors. But should it be?

“It’s fair to say that neither one of them gets at what we really want to look at, which is some sort of time-weighted, dollar-weighted kind of measure,” says Josh Lerner, a private equity scholar who holds the Jacob H. Schiff Professor of Investment Banking chair at Harvard Business School. “IRRs are prone to getting distorted, because all it takes is the quick return of a small amount of money to make a big IRR; and multiples don’t really take advantage of the time value of money.”

Increasingly, LPs and advisors are also incorporating relative performance measures into their reporting repertoires. Unlike cash multiples, TWR or IRR measures, these equations factor in assumptions for reinvestments – and in one variation, even leverage – against a public market equivalent (PME) adjusted to mimic a private equity fund’s cashflows. 

Such tools have been advanced by various private equity scholars over the past 15 years. A number of PME calculations now exist, with one of the most frequently cited versions – the ‘PME-multiple’ – stemming from a paper first circulated in 2003 and published in 2005 by Chicago Booth’s Kaplan and Antoinette Schoar, Professor of Entrepreneurial Finance at MIT Sloan School of Management. 

Kaplan and Schoar studied 746 (largely realised) funds from Thomson One’s database between 1980 and 2001, showing IRR as well as a public market equivalent using the PME-multiple formula. Their formula discounted all cash outflows of the fund at the total return to the S&P 500 and using the same index’s total return, compared the result to the discounted fund’s cash inflows (net fees). A net result greater than one indicated outperformance. 

“The public market equivalent that Antoinette Schoar and I came up with is really the simplest measure and it’s easy to calculate. It’s basically a market-adjusted multiple of invested capital,” says Kaplan. “So it’s just saying: how would your fund have done if you compare it to how it would have done if you just put the money in the public market.”

Raj Mehmi, principal at Altius and head of its reporting and monitoring team, says there’s been increasing interest from their fund of fund and separate account clients in PME-type measures but most remain focused on IRR and multiples. “Although some of these new measures have been around for a few years, people are used to using the IRR measurement and some of our clients are only just starting to get used to PME,” he says. “So there’s a bit of a learning curve; it may be that there’s a bit of time lag before the measures gain wider use.”

Cambridge Associates is one of the groups trying to move PME measures into the mainstream, says Eric Johnson, a managing director who advises institutional investors on asset allocation and investment issues. Clients have been interested in it, he says, adding Cambridge plans to roll out its own brand of PME more broadly later this year. 

“It’s very useful context to use a PME measure of some sort to help answer those big broad allocation questions,” Johnson says. “A PME approach provides a sense of what the other environment or other opportunity cost was like at the time. It provides context and doesn’t supplant the existing vintage-based measures at all; it just helps them.”

Using the same sort of example as Diller, Johnson explains how PME can put IRR or a multiple of invested capital in better context. “Say starting from a certain point in time you’ve allocated to private investments or you could have put that money into something else. All of those sorts of existing measures for the private investments don’t change – you can still have a TVPI, an IRR, an MIRR … But what’s been missing for a lot of investors is: how do we know if 2x was good? A 2x total multiple [may look] really, really good versus other private investment strategies at that time or versus other vintage years. But it’s actually helpful to know what that 2x return is in the context of what you might have been able to get investing in public markets.”

This is a crucial point for private equity investors, says Kaplan. “Private equity LPs have been – I think – relatively unsophisticated, relative to the public sector, in looking for alpha and calculating how you’ve done relative to the public markets.” But that’s been changing, he adds. “LPs over the last five or so years have gotten more aggressive and sophisticated. Obviously they’re putting more pressure on contracts with GPs and I think more sophistication on performance measurement goes hand-in-hand with that.”

Even with additional or different equations, calculating returns relative to public markets isn’t straightforward. Academics have repeatedly undertaken studies on private equity performance versus public markets, producing varying results. 

“Honestly, there are so many approaches of how to benchmark things and different methodologies … In the end I’d say it’s not possible to exactly match private equity with public equity,” says Diller, citing private equity’s illiquidity. “Each approach is only an approach to get as close as possible to other asset classes, but it’s still very difficult to do that.”

The other issue market participants repeatedly point to is inconsistent data. The various commercial databases contain different sets of funds and use diverse methodologies for collection that may affect their data set.

“Given the discrepancies of the data, it’s not clear how much results [produced by certain studies] are [just particular to that] individual database,” says Lerner. He’s spearheading the creation of a global database for academic use alongside the Private Capital Research Institution; GPs including Kohlberg Kravis Roberts, The Carlyle Group and Apollo Global Management have already signed up to submit data.

The idea, Lerner says, is to create “a super-high quality database, where we’re getting the data from a combination of commercial providers as well as from the private equity groups themselves. We’ll avoid some of these difficulties you see where you have three studies with three separate results using three different databases, and try to be able to really converge a little more closely.”

The Institutional Limited Partners Association, meanwhile, is working with Cambridge Associates to create an ILPA-specific benchmark/database that will focus solely on the private markets investment performance data of funds in which ILPA members have invested. 

Cambridge’s Johnson says the partnership will help to expand the fund universe data to which people have access. “I think one of the key issues that comes up with so many people is composition of the universe. Almost by definition no one is going to have the 100 percent comprehensive private investment universe. So the question is how broad and representative can you make it, such that you are reducing the possibilities for database bias as much as possible and you’re getting as representative a picture as possible.”


Many are hoping that more and better data – as well as the use of various PMEs in research on benchmarking and performance – can shed more light on basic but core issues that strike at the heart of private equity’s conventional wisdom: not just performance relative to public markets, but also concepts like persistence, or the idea that top-performers will replicate performance with subsequent funds across different time periods/market cycles.

“Persistence is an issue that’s extremely hard to get a fix on,” says Lerner. “If one fund outperforms, does the next fund outperform? There have been any number of studies with quite different answers using different databases and certainly that’s an example of a pretty basic question – it’s not like it’s some hi-tech analysis – where you need to get the right data to really be able to answer that.”

Kaplan’s and Schoar’s work in 2005 with the Thomson One database found evidence of persistence in US buyout funds raised in the 1980s and 1990s (though a 2011 paper by Oxford’s Ruediger Stucke suggested flaws in the database). A new study Kaplan has undertaken with fellow scholars Robert Harris (University of Virginia Darden School of Business), Tim Jenkinson (Oxford’s Said Business School) and Stucke, uses a database from Burgiss Group of some 1,400 funds from more than 200 institutional investors. Their data remains preliminary, with a draft circulated in December. While it confirms the 2005 study’s original findings about vintages from the 80s and 90s, it suggests that may no longer be the case with the next decade of funds – meaning a GP’s track record may not actually be indicative of future performance.

“Getting some new data and looking at that is of great interest and what was interesting – and this is preliminary, we still have to make sure it’s right – is that with venture there is still a huge amount of persistence and with private equity we found that the persistence was actually quite a bit lower in the 2000s than it has been in the 1990s,” says Kaplan. “That means that you’d have to look at other criteria to figure out who is going to outperform.”
Despite all these advances, sources caution there is no single equation or academic study that will solve the industry’s issues around performance and benchmarking.

“There isn’t one be-all and end-all number that’s going to help you; you have to look at this through different prisms depending on the questions you’re looking to answer,” says Cambridge’s Johnson. Time-lag is also an issue, he says. “It’s very hard to know how things are going to work out until a lot of time has passed and you really have a full set of numbers. And then by the time you actually get all those numbers and you can start analysing them in some way, the world has changed. All investors in so many different asset classes have to be wary of chasing past performance.”

Measuring and benchmarking advances – and some of the core questions they can help answer – will continue to be a hot topic as academics and investors advance their thinking around these concepts. “There’s lots of discussion and lots of different views,” agrees Montana’s Diller. “But the funniest thing is that this discussion will never end: as long as you get performance numbers, it will change.”