Performance and risk statistics reported by hedge funds are unreliable. In fact, the only true way to tell the profit or loss on a hedge fund is to cash out the investment.

These are the findings of a working paper by Don Goldman, chief financial engineer for Measurisk, a risk evaluation firm in New York, which was filed with the Securities and Exchange Commission in early May just ahead of the SEC roundtable on hedge funds.

In the paper, "Seeing is Not Believing: Fund-of-Fund and Hedge Fund Risk Assessment and Transparency, Survival and Leverage," Goldman cites a number of problems with hedge fund data.

A chief problem, he said, is that there is no crystal ball when it comes to investment data and analysis. Any analysis of the financial markets assumes that history may repeat itself, but not exactly in the same pattern.

"All risk models and tools are flawed," Goldman said. "Yet without the models and tools, there is increased risk. With them, there is a danger of misinterpretation or misapplication, or too much trust resulting in laxity. A defective tool may be better than no tool at all - but that is all."

Return data, he warned, should be viewed with skepticism. Otherwise, investors may become complacent. For example, the Value-at-Risk (VaR) measure is frequently used to evaluate hedge fund risk, Goldman said. But VaR is defined as the amount you can lose, with a probability of 5%. In reality, investors could find themselves strapped with big losses, Goldman said.

"Five percent of the time you will lose more," he said. "And these statements are made under the assumptions that times are pretty normal. But when large losses occur, things are anything but normal." This is true with any risk measure, not just VaR, Goldman said.

One way around this weakness of probability-based measures is to use a risk measure like downside volatility, which requires no probability measure, Goldman said. "All criticism aside, VaR is generally a useful risk measure in the right hands, [provided that] it is interpreted in an intelligent way with adequate drilldown and tracking capabilities," he said.

Hedge fund data is also distorted by survivorship bias, as funds that fail are dropped from databases, perhaps artificially raising returns, Goldman continued. It has been estimated that the hedge fund failure rate ranges between 4% and 25% annually. So the total returns on the remaining funds and the Sharpe ratios have an upward bias, particularly with highly leveraged funds. But based on his analysis, higher leverage implies a fund is less likely to survive. If it does, it will have a higher Sharpe ratio. The Sharpe ratio is a measure of a fund's returns per unit of risk. The higher the Shape ratio, the better the return per unit of risk.

"One has to take the figures for money managed by hedge funds with some healthy skepticism," Goldman said. "The bottom line is that when you look at survivors, you see higher Sharpe ratios associated with higher leverage. Funds might look less risky than they are simply because they are still alive."

Data on the amount of assets under management is also questionable, as they don't just include capital under management but additional money belonging to the manager that he or she uses for leverage, Goldman said.

Returns, too, can be distorted by the fee structure, how security positions are marking to market, and how illiquid securities are priced. Many hedge funds, Goldman said, do not have the tools to price illiquid securities and must depend on dealers for pricing. Many funds-of-funds are not capable of separate pricing.

Another concern is that if hedge funds become popular, more dollars will flow into them and it could become difficult for hedge fund managers to generate stellar returns. As a result, the managers may take more risks to get better returns. In the past, Goldman said, analyses of major hedge fund blow-ups found that there were increased capital flows when the hedge funds' risk models broke down.

Copyright 2003 Thomson Media Inc. All Rights Reserved.

Subscribe Now

Access to premium content including in-depth coverage of mutual funds, hedge funds, 401(K)s, 529 plans, and more.

3-Week Free Trial

Insight and analysis into the management, marketing, operations and technology of the asset management industry.