Wealthy investors say the prolonged economic downturn has become their top worry, according to research released Tuesday.

Four out of five millionaire investors surveyed by Spectrem Group say that the sluggish economy was their top concern in the first quarter compared to 70% in the same period a year ealier.

Similarly, 81% of mass affluent investors, whose net worth is between $100,000 and $1 million, cited it as a concern in the first quarter versus 74% a year earlier.

Only ultra-high-net-worth investors, with at least $5 million in assets, didn't list the slow pace of the recovery as their chief concern. Eighty-two percent of these investors said the national debt, and the contentious political environment (80%), beat out the economic downturn as their top concern (77%).

"With the unemployment rate remaining high, the continuing volatility in the stock market and gasoline prices, and a high-stakes presidential election looming, wealthy investors are anxious about the country's outlook," says George H. Walper, Jr., president of Spectrem Group. "You can expect a strong turnout among these voters."

Anxieties among the affluent about maintaining their own financial position and having enough money for retirement also increased from a year ago. In addition, a strong majority worry about tax increases.

But tax concerns are not translating into major investment changes. Just 40% of the wealthiest, 28% of millionaire and 25% of mass affluent investors are making investment changes because of possible tax increases.
"Overall, wealthy investors are more confident about their own finances, with roughly half optimistic that their personal financial situation will improve in 2012," Walper said.

Forty-nine percent of mass affluent investors, 52% of millionaires and 55% of ultra-wealthy investors expect their financial situation to grow stronger in 2012.

Matt Ackermann writes for Financial Planning.