Three days in August 2007 will live forever in algorithmic investing infamy. Numerous quantitatively driven stock funds leapt en masse off a cliff-just like lemmings.

From the 7th through the 9th, a few quant fund computer models started to unwind stocks, as the first footsteps of the credit crisis began to affect calculations. Some sales were to reduce risk. Others responded to margin calls. This spurred other models to respond by selling their stocks, and so on. The cascade of liquidation evaporated as much as 20% of fund values.

The culprit: investing models that were more similar, and hence much more correlated, than anyone realized, says Petter Kolm, director of the Mathematics in Finance Masters program at the Courant Institute of Mathematical Sciences.

"That event taught us that we have to do something that is more unique. We have to have our own models, our own strategies and factors," said Kolm, who is also a principal at the Heimdall Group.

Consider the Adaptive Quant Trading program developed by Vasant Dhar, head of the Information Systems Group and director for the Center for Digital Economy Research at the Stern School of Business at New York University. This futures-investing program is adaptive, designed to learn from market databases and extract hidden emerging patterns that are not easily recognized by humans.

"This system is based on a completely automated approach to model construction," says Dhar. "I don't construct the strategy. The strategy is constructed by the computer. No human [is] in the loop in the building of the strategy."

The AI asset manager has outperformed its benchmark since inception, earning 10.83% in 2011, compared to -3.64% for the NewEdge CTA Short Term Traders Index. It has earned 5.13% year-to-date, compared to -0.96% for the benchmark.

Dhar's investment philosophy is informed by this insight: patterns emerge before the reasons for them become apparent. This is particularly true for finance, he says.

"What makes machine-learning algorithms especially unique is that they can actually discover new, non-obvious ways of making money," says Spencer Greenberg, chief executive officer of machine learning asset manager Rebellion Research. "Since these algorithms learn from the data provided to them, they can find useful relations that even the designer of the system was not initially aware of."

Frank James, chairman and founder of James Investment Research, puts it this way: never rely on the common wisdom, just look at the data.

His firm uses stock-picking computer models for five mutual funds. The models utilize roughly 30 different investment factors, culled from more than 100 possible categories. Current factors used include statistics related to interest rates, interest rate changes and money growth, such as relative value and positive earnings.

The model also has proprietary approaches to analyzing market sentiment and management confidence, looking at figures such as open market purchases made by company executives using their own money.

"I have every confidence that if I select only the top 10% of the stocks picked from our model quarter-after-quarter, I will end up with very strong results," says James.

Model success, James says, depends on developing precise arrangements between investment statistics. That is why his firm relies on a technique called non-parametric statistics. This deals with data that has less-than-ideal parameters regarding distribution of results. He says more firms should use it because it helps with what he calls "dirty data."

"A lot of things in investing really don't have the precision you think they do," he says.

Where can machine learning help you? It depends on where your assets lie in the "spectrum" of investment theory, according to Professor Dhar at NYU. If an asset class, like mortgage-backed securities, has a lot of theory to inform your investment strategy, humans are still useful. In the case of MBS, there are good, clearly understandable mathematical relationships between all the factors, such as the effect of refinancing mortgages on interest rates being paid in or defaults on the flow of payments coming in.

However, at the other end of the spectrum, say stock or futures prices over very short intervals, Dhar says AI systems can be very useful.

"With these intelligent search systems, you can find things that humans find impossibly difficult to pinpoint," he says.

Greenberg of Rebellion Research says that asset managers will increasingly use machine-learning systems because they can process enormous quantities of data in seconds, detect subtle long-term statistical correlations between relevant variables and make predictions that are simultaneously based on thousands of distinct inputs. The human brain didn't evolve to handle these sorts of tasks.

"I expect we'll see more reliance on [artificial intelligence] over time, as competition leads quant funds to have an increased incentive to create algorithms that pick up on subtle patterns and behave in more intelligent and nuanced ways."

Kolm of the Courant Institute says that there are a growing number of asset and hedge fund managers experimenting with not only machine learning but other strategies such as "neural nets," which mimic the parallel processing of human neurons and "genetic programming," which imitates the biologic processes of mutation and replication to develop "survival of the fittest" algorithms. These esoteric techniques are still far from the mainstream, he said.

One major challenge plaguing AI asset management is the same that dogs human managers: the issue of attribution.

"You could have a really good model that underperforms because of just sheer bad luck, some outside event, act of God that is not amenable to being modeled and this leads to bad performance,'' Dhar said. "Or you could also have a crappy model that, also by luck, just happened to perform very well."