Performance assessment: what we know and what we don't know

We hear it at every conference and read it in many article written on the topic: “Private equity is such an attractive asset class because it offers high returns!” Often this claim is supported with some chart or table that shows PE out performance based on a comparison between historic private equity and stock market returns. Sometimes, quietly, someone adds that “actually, it offers high returns if (and only if) youmanage to get into the top quartile of the market.”

HOW TO COMPARE A PRIVATE EQUITY PORTFOLIO TO PUBLIC MARKET INVESTMENTS
The assessment of private equity performance is challenging, as this illiquid asset class violates several fundamental assumptions of standard financial market theory.

There are theoretical and practical challenges with respect to the assessment of risk and return of buyout investments. For example, it is imposible to determine in absolute term whether a 17 percent IRR of a private equity fund should be considered good or bad performance per se. Instead the question has to be, “good or bad performance relative to what”?

One practical approach to this problem is the benchmarking of PE investments by comparing their performance to the returns of “similar” stock market investments. However, we have to realise that the two differ fundamentally in key determinants of investment risk, such as the average size and age of the company (especially relevant in the VC segment) and in the degree of financial leverage (especially relevant in the buyout segment). Therefore it is difficult to say whether a performance of, say 17 percent net IRR of a given private equity fund is good or bad compared to a 14 percent return to the S&P500 index over a comparable time period, as the answer to this question depends on the differences in risk, illiquidity etc. of the two investment alternatives.

CONSTRUCTING A MIMICKING PORTFOLIO
The basic idea behind a comparison of private equity performance with public market investments is fairly straightforward. One simply answers the question: How much (less) performance would a similiar investment in the public markets have generated? Things become tricky, however, when it comes to the exact definition of “similar” investments. First of all, the investments. First of all, the investments of the “mimicking” portfolio have to the “mimicking” portfolio have to match the private equity investments in terms of (a) the timing of their cash flows (investments and distributions). Second they have to mimic the systematic risk pattern of private equity. which is much more chalenging.

One pragmatic approach to the construction of a mimicking portfolio is to calculate the returns to a broad public market index, such as the S&P 500 or the MSCI Europe, assuming that the pattern of stock purchases and sales matches the timing of the private equity cash flows. This method underlies, for example, the “public market equivalent” calculations provided by Thomson Venture Economics and has also been used in a number of academic studies (see, for example Kaplan and Schoar, 2005). It explicitly assumes, however, that the risk pattern of private equity investments is identical to that of the chosen public market index. However, companies who receive venture capital financing are typically much smaller and younger and those who receive buyout funding typically have much higher financial leverage than the average company in a broad stock market index. Both these aspects influence the risk pattern substantially.

CAPTURING THE SPECIFIC RISK-PATTERN OF PRIVATE EQUITY
One possibility to correct for the differences in risk between private equity and public market investments is the matching of operating beats and financial leverage between the tow. This has been demonstrated in a number of recent academic studies. For example, Ljungqvist and Richardson (2003) and Phalippou and Zollo (2005) consider operating betas in their analysis of private equity investments, while Groh and Gottschalg (2006) benchmark the fully risk-adjusted performance of US buyouts by correcting for differences is operating risk and financial leverage.

The exact determination of the risk-adjusted mimicking portfolio reuires for each private equity transaction (a) the identification of a peer group of publicly traded companies with the same operating risk, (b) the calculation of the equity beatas for each of these “public peers”, (c) the unlevering of these beta factors to derive their operating or unlevered betas, (d) the determination of a market weighted average of these operating betas for every peer group, and (e) the relevering of these betas on the level of the private equity transactions at closing and exit. The unlevering and relevering procedures also require the specification of the risk, which is borne by the lenders, the risk of tax shields the equity investor can benefit from, as well as an applicable corporate tax rate.

With this data and the calculations the mimicking portfolio can be established as follows: An equal amount of equity as for every private equity transaction in invested in a representative markt portfolio and is levered up iwth borrowed funds to the same beta factor as the private equity investments at closing. The risk of the public market transaction is then adjusted every year, tracking the risk of the private equity investments. Therefore every position is liquidated annually, interest is paid, debt is redeemed and the residual equity is levered up again with borrowed funds (respectively funds are lent) to the preveailing beta risk of the private equity investments. This procedure is repeated until the exit date. A detailed description of the corresponding methodlogy can be found in Groh and Gottschalg (2006).

While this procedure is admittedly complex, it is the only way to accurately compare private equity to stock market investments. Any simplification in the comparison has to be expected to yield potentially misleading results, as shown is Groh and Gottschalg (2006).

WHY A PERFORMANCE ASSESSMENT OF THE PRIVATE EQUITY ASSET CLASS REMAINS DIFFICULT
The question of how private equity as an asset class performed compared to investments in the public markets has received much attention from practitioners in the public markets has received much attention from practioners and academics alike. Data limitations and the long investment cycle in private equity make it difficult to provide equity make it diffcult to provide a definitive answer to this question. This section reviews relevant methodological and empirical findings in the academic literature and explains and explains the partly conflicting results of the studies as due to different data characteristics and approaches to the treatment of risk.

Challenge I: finding an unbiased sample of the entire private equity universe
To assess whether returns of private equity investments are on average better than comparable stock market investments, one has to first of all compose a dataset on the returns of the entire universe of private equity investments – or at least a representative (unbiased) sample thereof. This first step constitutes a major challenge already in an industry that calls itself private and is famous for a lack of transparency.

Professional database service providers such as Thomson Venture Economics have over many years been able to compose what can be considered the most comprehensive commercially available database on private equity investments and fund returns. Still it is fair to assume that even this enormous database does not capture every single private equity investment ever made. Missing data becomes a problem as we cannot exclude the possibility that the missing deals perform substantially better or worse than those included in the database, which would make a performance assessment based only on the deals in the database inaccurate. A recent study by Phalippou and Gottschalg (2006) shows that to some extent such a bias is even present in the Thomson Venture Economics PE Fund performance database. At least for the early vintages, the funds included in the performance database seem to be less successful in terms of their exit success rate (% exited through IPO or M&A) than the larger universe of funds. This suggests that even the large sample of funds in the performance database may be upwardly biased by about 2 percent per annum.

Several academic studies draw on much smaller samples, sometimes with an obvious bias introduced by the way data was collected. While statistical methods to correct for these biases are able to make such data suitable for the analyses performed in the respective papers, the authors acknowledge that the biases still limit the generalisability of any performance assessments (Ljungqvist and Richardson, 2003; Cochrane, 2005; Groh and Gottschalg, 2006).

Challenge II: avoiding noisy valuations of residual values
Whenever private equity funds still have unrealised investments on their books, any performance measurement becomes unreliable. Absent market valuations for the corresponding companies, there is just no way to determine their fair valuation at a given point in time at a sufficient level of precision. Potentially, these investments are developing nicely and will soon lead to realisations at a value far beyond the residual value recorded on the books, Maybe the residual value which – particularly in the buyout segment – is often kept “as cost” is quite accurate as a proxy for the NPV of future returns. But it is also possible that the residual values belong to living dead investments that will never return a dollar to investors and should have been written off some time ago.

It is easy to see that the greater the percentage of residual values, the less precise the performance measurement for the corresponding sample of funds. One way out of this dilemma tis to look at funds with no or minimal residual values only. However, here we exchange one evil for another, as fully realised funds tend to be relatively better performers than same-age funds that still have many deals on the books. Consequently we increase the sample bias if we look at funds with no or minimal residual values only. However, here we exchange one evil for another, here we exchange one evil for another, as full realised funds tend to be relatively better performesrs than same-age funds that still have many deals on the books. Consequently we increase precision, but also increase the sample bias if we look at realised-only funds 1.

A practial way to deal with this challenge is to look at a sample of entire vintage years sufficiently mature funds (Ljungqvist and Richardson, 2003; Kaplan and Schoar, 2005; Phalippou and Zollo, 2005; Phalippou and Gottschalg, 2006), in the sense that these were raised so long ago that non-exited investments should be few and in any case unlikely to lead to substantial cash distributions. A detailed analysis of the residual value dynamics of mature funds performed in Phalippou and Gottschalg (2006) strongly suggests that these correspond for the most part to “living dead” investments and should consequently be written off. In any case, the residual value challenge limits all analysis to funds raised a long time ago and limits our ability to gain insights into more recent PE performance.

Challenge III: considering different risk patterns of private equity and stock investments
It is intuitive to see that private equity investments have very different risk characteristics compared to stock market investments. They can differ in key determinants of investments risk,such as the average size and age of the company and in the degree of financial leverage. lf the risk of investing in a portfolio of pre-IPO startups or mature, but highly leveraged businesses is substantially different from that of an investment in, say, the S&P500, it is clear cannot simply be compared. Instead a careful adjustment for the risk pattern of all PE investment has to comparable stock market investments can be made. Groh ahd Gottschalg (2006) show empirically that failure to consider risk in the comparison can yield highly misleading results.

So we know what it takes to fully answer the question of how private equity performance compared to stock market investments, or at least a sufficiently large and unbiased sub-sample thereof. This data should cover a time period until as recently as possible, while non-exited investments should be absent from the data or their residual values should be unambiguously determined. Finally, we do not only need cash flow data for these investments, but also relevant information to determine their risk pattern, i.e. at least the companies' age, size, sector of activity and the evolution of the capital structure over the time of the investment. Today, such a “first best” performance assessment is unavailable, as data limitation in the notoriously secretive private equity industry made it impossible to gather the necessary data.

PRIVATE EQUITY PERFORMANCE RELATIVE TO THE STOCK MARKET – A SUMMARY OF WHAT WE KNOW
The question of how private equity funds performed historically compared to investments in the public markets can still only be partially answered. To-date data of sufficient depth and breadth on private equity investments to conduct an accurate, unbiased and comprehensive performance comparison is simply unavailable. Nevertheless the findings from the various studies enable us to draw a number of important conclusions.

First of all, it becomes evident that that the general claim the private equity historically offered higher returns than the stock markets is as such unwarranted and has to be called into question. The majority of the broader studies report the net-of-fee performance of private equity to be either below (Phalippou and Zollo, 2005; Phalippou and Gottschalg, 2006) or not much different (Kaplan and Schoar, 2005) from those of stock market investments. The studies that de report substantial out performance either look at smaller samples from a (historically better performing) subsegment of the private equity market only (as the analysis of early US) buyouts by groh and Gottschalg 2006), look at investments made by funds that were slected by a sophisticated LP (such as Ljungqvist and Richardson 2003) or do not fully capture the ex-ante differences in risk between the two alternatives (Cochrane 2005). Each of these studies makes valuable and important contributions to the advancement of our understanding of hw private equity performance can be measured and benchmarked, and future research with more and deeper available date will be able to build upon these to lead to more generalisable performance measurements.

Importantly, none of these studies quantify the other less attractive features of private equity, such as the illiquidity and the unpredictable nature of cash flows, in the analysis. If these were to be considered, they would lead to an additional discount on private equity returns that would further decrease the relative performance of this asset class.

IMPLICATIONS
The first and most important conclusion from our discussion should be: “Beware of any “simple” performance comparison!” It is easy to come up with some data on private equity returns, some data on seemingly similar stock returns and draw whatever conclusion you want from the comparison. Getting the comparison right requires more representative broader and deeper data than is available to most, if not all. It is therefore important to understand where data comes from and how it differs from the overall population of private equity investments. A sample of deals that a large LP invested in may be biased, just as much as a set of deals that a large consultancy has seen or the survey respondents to an academic study. For many purposes, a such data is perfectly suitable and if necessary the biases can be dealt with statistically.

However, such samples do generally not allow definitive conclusions on whether PE outperforms the stock market or not. Furthermore, an accurate comparison requires complex methods to deal with uncertain evaluations fo residual values (Cochrane, 2005; Phalippou and Gottschalg, 2006) and to incorporate the role of risk in the comparison (Groh and Gottschalg, 2006). Paying attention to these factors seems crucial to avoid being misled by inaccurate numbers.

The second implication is that to the extent that we can something about the average historic performanceof private equity, there is little evidence that it is above the returns of truly comparable stock market investments. It may be time to get used to the thought that (at least in the 1980s and much of the 1990s), private equity has on average offered lower or at best similar returns than a cash-flow-matched and equally risky portfolio of public market investments.

True, top quartile performers outperform the S&P 500 even on a risk-adjusted basis, but is that comparison really fair? As one only meets top-quartile GPs – and LPs who fund top-Quartile GPs – one wonders where the rest of the industry is hiding. Maybe it is time to admit that private equity can create substantial value but it does not always. It can contribute to the creation or resurrection of superb businesses. Many players demonstrate this on a repeated basis, as the finding of performance persistence of the top performing GPs in several studies has shown (Kaplan and Schoar, 2005; Phalippou and Gottschalg, 2006). However, this cannot be said about all GPs and it seems important to face the fact that often private equity also destroys value and harms companies.

The most important challenge going forward is hence to refine our understanding of what works and what does not work in private equity. Only then one can make sure that the enormous potential of the private equity model to generate great returns and to benefit the companies the are acquired is being fully realised. Giving up the illusion of genuinely high private equity returns across the board may be the first step in the process.

The constant repetition of the (empirically unwarranted) claim the private equity offers great returns per se is in fact dangerous for the industry. It currently attracts additional capital in times when fund sizes and deal sizes are at a record high already. Importantly, it also attracts capital fromless experienced and less sophisticated investors who may end up backing the wrong GPs, based on the belief that even a random fund selection process would lead to the supposedly attractive average returns are not necessarily attractive and – more problematic for the industry – as long as less skilled GPs continue to get funded average future returns are less likely to increase than they otherwise would be.

Reference
Cocharne, John H. (2005): The Risk and Return of Venture Capital, in: Journal of Financial Economics, Vol. 75, PP. 3 – 52

Gompers, Paul A. and Lerner, Josh (1997): Risk and Reward in Private Equity Investments: The Challenge of Performance Assessment, in: Journalof Private Equity, Vol. 1, pp.5 – 12

Phalippou, Ludovic and Zollo, Maurizio (2005): Performance of Private Equity Funnds: Another Puzzle? INSEAD workign paper

Phalippou, Ludovic and Gottschalg, Oliver (2006): The Performance of Private Equity Funds, HEC working paper

Gottschalg, Loos and Zollo (2004) Working out where the value lies, European Venture Capital Journal, June 2004

Groh, Alexander and Gotschalg, Oliver (2006) The Risk-Adjusted Performance of US-Buyouts, HEC Working paper

Kaplan, Steven N. and Schoar, Antoinette (2005): Private Equity Performance: Returns, Persistence and Capital Flows, in: Journal of Finance, Vol. 60, pp. 1791 – 1823

Ljungqvist, Alexander and Richardson, Matthew (2003): The Cash Flow, Return and Risk characteristics of Private Equity, NBER Working Paper 9454