Benchmarking: private against private

In previous papers I have shown how private equity investors can benchmark the performance of their portfolio against a public market index, to measure the relative benefit of investing in this asset class. Another important issue often raised by investors is the relative performance of their portfolio with respect to the private equity industry itself, which is only partly addressed by the publication of private equity indexes. This paper reviews some of the issues with benchmark indexes and presents an easy method based on portfolio replication that provides a sensible way of benchmarking the performance of private equity portfolios.

Private equity investors face the issue of benchmarking their private equity returns against those of the industry to answer the simple questions: “How do we compare with the others?” As always the question is fairly obvious but the answer is not. Ideally an investor would like to compare the returns of his portfolio to the returns of an index representative of the asset class over the same period to assess the quality of his portfolio selection. Several service providers, including Venture Economics and Cambridge Associates, publish quarterly benchmark returns for buyout and venture capital, which provide estimates of how the entire industry has been performing over that period.

Whereas the computation and the reporting of these index returns – leaving aside all the issues related to stale net asset values – is definitely helpful for an overall assessment of the performance of the industry, there are clear limitations in using them as benchmarks for particular private equity portfolios. This is very well illustrated by the differences in the index returns reported for the various quarters, which can be of the magnitude of several percentage points (see Figure 1). The reason for the discrepancies is that the indexes are generated from different data sets.

Both Venture Economics and Cambridge Associates rely on data the majority of which is collected from clients they are advising. As a result, the corresponding portfolios, while containing a relatively large overlap of shared funds, are deemed to be different from an allocation perspective.

In addition, the quarterly returns of a private equity portfolio are particularly sensitive to two additional factors: the granularity of general partner allocation due to the low frequency and potential high impact of changing valuations of portfolio companies; and vintage year allocation due to the J-curve effect. As an extreme example, a young portfolio with a couple of fund investments from an investor entering the market has very little chances of achieving returns similar to a broadly diversified portfolio built over decades, even if the sector allocation is similar.

As a result, index returns can only legitimately be used as benchmarks for portfolios with allocations relatively close to the one used for the computation of the index. As aggregates of large institutional portfolios, these index portfolios are likely to be representative of the historical development of the total commitments to the industry, but do not necessarily match an investment programme run by a specific investor. Moreover, index returns are “pooled” returns and do not give any insights as to the potential dispersion of the achievable returns. Finally, they are time-weighted, which, due to the intrinsic nature of net asset value, is arguably not be the best way to assess the performance of a portfolio.

The standard way of benchmarking individual partnerships is to rely on the “Investment Benchmarks Reports” published by Venture Economics, which give a fairly detailed and complete view of how the various investment stages and vintages have developed over time. The only problem is that while comparing the returns of individual general partners is straightforward, it is almost impossible to deduce any meaningful ranking on a portfolio basis. All the data required is there, but what is missing is a tool that allows investors to replicate their portfolio and get the statistics corresponding to their particular allocation. Such statistics would provide a one to one basis for the benchmarking of individual portfolios.

The statistical properties of a given portfolio cannot be derived from the individual statistics of the various vintages, but have to be calculated based on a Monte Carlo simulation. A Monte Carlo simulation in this case consists of randomly assembling thousands of portfolios with allocations that are identical to that of the original portfolio. Once this is done, the analysis of the IRR statistics for these random portfolios helps rank the actual portfolio.

As an example, we consider a hypothetical client portfolio or “test portfolio” with a $200 million allocation split evenly between 20 unique limited partnerships, 10 of which are invested in buyout and 10 in venture capital over the vintages from 1994 to 2003. The IRR based on the reported cash flows and net asset values as per the end of first quarter 2004 is 28.8 percent, and the return over the last quarter is 7.0 percent. What can one say about this portfolio using portfolio replication?

Figure 2 represents the result of our analysis. Using the Venture Economics cash flow database as the data source, the simulator generates thousands of random portfolios with identical geography, investment stage and vintage allocation. Given the size of the database, there are between 10 to 100 matches for each single commitment, resulting in a universe of more than 1030 different random portfolios. At each individual step, the simulator builds the cash flow stream for the corresponding portfolio, computes the IRR and stores it. At the end of the simulation, the programme outputs the statistics of all IRRs. These statistics show the probability of achieving any given returns by randomly picking funds without applying any asset management skills. The 28.8 percent IRR achieved by our test portfolio belongs to the second quartile within its peer group, which indicates positive selection skills.

It is well known that IRR also has its shortcomings when used as unique measure of return. Benchmarking should not be reduced to the mere comparison of IRRs, but should include the analysis of additional cash measures. Aside from IRR, many investors rely on realised and unrealised multiples to assess their portfolio. The main advantage of the portfolio replication methodology is that once the simulation is set, it is simple to compute as many other quantities as one whishes to characterise the performance of a portfolio. Based on the cash flows and net asset values of the simulated portfolios, it is easy for instance to build the statistics of the ratios of distributions to paid-in capital (DPI) to benchmark the realised multiple of the actual portfolio.

Figure 3 represents the result of this analysis for the test portfolio. Again, the test portfolio turns out to belong to the second quartile with respect to distributions (DPI) and total value (TVPI), which represents the quality of the underlying portfolio and helps quantify it as well. The bottom quartile ranking with respect to net asset value (RVPI) is not a concern due to the overall good performance with regard to distributions and total value. By looking at different quantities such as absolute returns, speed of cash flow, percentage of realised and unrealised investments etc., one can get a much better picture of the quality of the portfolio, compared to what it could have been by investing with other general partners with the same investment profile. As such it provides a natural benchmark for selection skills as opposed to a benchmark against pooled returns of the entire industry.

As a further illustration of the difficulties involved in using index returns, Figure 4 provides a comparison of the actual returns of the test portfolio over five quarters and the corresponding benchmark index returns computed by averaging the returns of buyout and venture capital as reported by Venture Economics and Cambridge Associates. As the test portfolio is much less diversified than the benchmark portfolios, one expects the quarterly returns to show much greater volatility. Over the last quarter 2003, the test portfolio even reported a negative return whereas the overall industry was having its best quarter of the year. The 7 percent return over the first quarter 2004 is much higher than both benchmarks. No doubt the reasons for these discrepancies can be investigated at investment company level, but still the interpretation is difficult for benchmarking purpose.

It is also worth noting that the portfolio replication method can be applied to measure the statistics of quarterly time weighted returns as well, as represented in Figure 5. The broad dispersion of return achieved by the replications of the test portfolio over single quarters illustrates the reason why benchmarking against an index is challenging. By nature, short-term returns of private equity portfolios are very sensitive to large mark-ups or big realisations by single partnerships, which are likely to promote the portfolio to top quartile level for a given quarter, while prudent accounting or large write-downs will result in temporary below market returns.

Adding to the complication is the fact that a below/above market return on a quarterly basis does

not necessarily imply a bad/good return with respect to peers overall. Third quarter 2003 provides an example for this phenomenon: the 3.5 percent return of the test portfolio is smaller than the return of the indexes, but ranks top quartile against its peers.

This is why the interpretation of quarter performances in isolation remains difficult, and why time weighted returns have to be followed for several consecutive quarters (as in Figure 5) to develop some understanding of the performance. As the test portfolio ranks top quartile for three out of these five quarters, and in spite of a catastrophic first quarter 2003, the picture emerging from the analysis is one of a portfolio performing above average.

In summary, the simulation method is nothing but an extension of the standard private equity benchmarking technique already used at single partnership level to portfolios. The basic idea behind this type of benchmarking is to provide a comparison with peers, which is achieved by replicating the entire portfolio. As the statistics for a specific portfolio cannot be read off a table or a book, an individual analysis has to be run for each single portfolio with the help of a simulation tool. This analysis is one of the many quantitative techniques pioneered by Capital Dynamics for its clients, which assist in better understanding the drivers of performance for private equity portfolios.

Christophe Rouvinez is a partner at Capital Dynamics, a firm specialised in private equity asset management. Capital Dynamics has offices in Zug, Switzerland and New York.