Trust, but verify.
That is how compliance officers at quantitative investment management funds should monitor the activities of their quant experts.
If compliance officers didn't know that already, they certainly will in the wake of the whopping $242 million penalty the Securities and Exchange Commission imposed on quantitatively-driven fund manager AXA Rosenberg last month for a "coding error" in its risk models. Of the $242 million, $25 million was a civil fine and the remainder was to compensate investors for their financial loss.
"Quant managers need to ensure that their compliance policies and procedures are tailored to the risks of their model's strategies, and that compliance personnel are integrated into the development and maintenance of their investment models," said Rosalind R. Tyson, director of the SEC's Los Angeles Regional Office, at the time of the fine.
The implication: compliance personnel must oversee the development and maintenance of models to ensure that risks are not hidden from clients and that they are minimized.
Based in Orinda, Ca., AXA Rosenberg is owned by French insurance company AXA SA. AXA Rosenberg, neither admitted nor denied wrongdoing in settling the case, but refrained from future violations of the securities laws. The firm also agreed to hire an independent consultant in quantitative investment techniques to review its disclosures.
Just how involved compliance officers should be is a matter of some debate. "Compliance personnel typically do not have the quantitative background that would be required to be intimately involved with model development, warns Edward Pittman, counsel at the Washington, D.C., office of the law firm of Dechert LLP. "The SEC's presumption that they do is a bit unrealistic and confusing."
Pittman, who once served as CEO of a quantitative fund manager, also says that the SEC made no reference in its fine to the role of compliance officers at Axa Rosenberg and mentioned "only in passing" that a team of quants had been focused on unearthing the source of their model's underperformance for at least eight or nine months before they were informed of the coding error.
But others take a more hard lined approach. "Any compliance officer at a quantitative investment firm should have a good working understanding of quant modeling and the technology involved," says Jack Hewitt, a partner in the New York law firm of McCarter & English.
At the very least, compliance officers will have to put in place a risk mitigation and notification process that enables them to detect risks within quantitative models that the quant modeling developers or portfolio managers directly involved with the model might prefer to hide.
"Quantitative funds may have to re-assess their internal compliance policies and procedures," says Timothy Levin, a partner in the Philadelphia office of Morgan, Lewis & Bockius.
Pittman agrees. "Compliance officers should be able to rely on experts to develop the models and execute strategies but if problems are discovered that could affect the ability of the firm to comply with the law, compliance officers need to be involved in the process," he says.
Here is what the SEC said it found: A coding error was introduced in the model in April 2007. But the firm's chief executive officer didn't find out about it until November 2009, more than four months after it was discovered. The error wasn't disclosed to the SEC until March 2010 and to clients until April 15, 2010. That meant there was a lack of disclosure and apparent intentional cover-up.
Quantitative funds are the "black boxes" of investing-portfolios run by managers who generally try to generate profit with algorithms that they don't fully share with investors or even internally. Not only must the overall investment strategy be tested but it must also be properly translated into computer code by a programmer.
Therefore, there are two types of errors quant funds have to worry about: either an error in the model or actual investment strategy used and an error in the code. When errors are embedded in code, they may go undetected for weeks or even years and when discovered it could take weeks or months to determine the effect of the errors. As the clock ticks, so do the potential financial losses to investors.
An ounce of prevention can go a long way. First up: reducing the potential for a coding error and barring that testing that a model is working properly. "Most programming experts have a code buddy to ensure that the code is input correctly and then testing should take place periodically," says David Klein, senior quantitative analyst for SuperDerivatives, a New York-based valuations firm.
Such testing could help reveal if there were an error with either the code or the model itself.
Klein cites the following differences as examples: in a coding error, the code might need to calculate the average of four stocks by adding their returns and dividing by four; the programmer forgot to divide by four. By contrast, an error in the model is far more subtle. "It could be that to correctly approximate real world behavior, the fund manager needs to take the geometric average of returns instead of their arithmetic average,'' he says.
Testing should be conducted not only by the programmers and the quant managers involved in creating the model but also by independent risk assessment professionals who either work for the firm or are external consultants.
"The compliance manager doesn't need to understand how the code was written but he or she does need to understand why changes must be made to the model and what results are expected," says Levin.
When notified of any errors, the compliance manager must quickly move up the chain of command- the chief investment officer, the chief risk officer, chief executive officer, legal counsel, and other C-level executives.
Human resources executives must also be made aware of what is going on in the event an employee needs to be fired for non-performance or fraud. And last but not least the media department which is likely to want to know how to handle questions once a press release is issued if major financial losses are involved.
Hewitt compares addressing an error in code or a quant model to implementing a disaster recovery plan. "Each member of the team-officials from the programming, portfolio management team to the risk managers and other C-level executives is delegated a role to fulfill and a game plan to follow on corrective measures, notification and other actions," he says.
Just does every error merit escalation to senior management? "It's a judgment call," says Dechert's Pittman. "Coding errors are fairly routine and the SEC is emphasizing that material errors must be escalated. It then becomes significant for the firm to define just what it means by material.''
