If there is one industry that is known for issuing surveys, it's the mutual fund industry. Every week, mutual funds and other investment firms issue multiple surveys about investors' attitudes toward savings and preparedness for retirement-aimed at spurring people to save more and at giving financial professionals talking points for their clients.
Last week, for instance, Transamerica revealed that only 23% of women are very confident that they will be able to live a comfortable lifestyle in retirement, unchanged from 2004. AARP reported that 80% of investors in 401(k) plans don't know how much they are paying in investment, administrative and individual fees, or how these fees would impact their savings.
Inevitably, while the information is meant to be educational, it is also meant to get investors and financial planners to think about the asset management firm issuing the information, and potentially steer their business that way.
Even though such leading polling companies as Harris Interactive usually conduct the polls, just how good are these mutual fund, retirement and asset-allocation surveys that cross financial professionals' desks? Are they accurate reflections of consumers nationwide?
The only way to tell is to look closely at the survey's methodology, said Mathew Greenwald, president of Mathew Greenwald & Associates of Washington. Greenwald's company has conducted numerous surveys for financial services companies over the years.
The guts of a survey are its statistical design, its sampling procedures and questions. Good probability sampling selects people to respond to questions in proportion to the degree that they comprise the overall population. The quality of the interviews and questions put to respondents are of paramount importance.
Greenwald recommends that financial professionals who are interested in a survey call the sponsoring company and get detailed information about the survey's methodology. Only then can they determine if the survey is accurate and reflects the attitude of the investing public.
There are several variables to consider when evaluating a survey's results, according to Greenwald. Examples:
*Reasonable sample size. The sample size should be over 200, but 500 is better. A large sample size will more accurately reflect the attitudes of the population that served as the focus of the survey.
*The sample should not be skewed or distorted. For example, if the survey reveals consumer attitudes about investing, you don't want the sample to come from one area of the country in which people are more affluent than the rest of the country.
*How were the survey respondents contacted? Telephone interviews are best because the surveyor can get more specific information compared with written survey requests in which respondents are asked to check off items. In addition, respondents should be contacted through an automated "random digit dialing" program, rather than from specific lists. If names come from a list, the list could be biased. Be suspect of surveys conducted over the Internet, because there is no control over who answers the questions. Also be suspect when a publication reports surveys of its readership. The results may be biased due to the limited demographics of those who respond. But there also are some good readership surveys done by publications. Be wary of "push" surveys. These surveys are designed to elicit a specific type of response to prove a particular group's viewpoint.
*Look at the type of questions and the way they were asked. Check the methodology to see if a statistical analysis was conducted to insure that the questions were not highly correlated. Typically, survey firms conduct a "regression analysis" or "factor analysis" to insure the questions are independent from each other.
*How were the results interpreted? Does the person interpreting the results have a good understanding of the subject matter? What are the points being made? Based on your own experience, are their comments knowledgeable? Did you learn something from the survey results that you did not know beforehand?
*Consider the company that conducted the survey. Is the company a well-known entity that has a good reputation for conducting surveys?
*What is the sponsoring firm's goal in releasing the survey? Many insurance and investment companies have well-designed surveys that draw attention to people's financial behavior and attitudes. These surveys still could be helpful as long as you consider the benefits the company stands to derive from it.
*Was there a statistical test conducted to make certain the responses were not due to random chance? You want to know, for example, that 95 times of 100 times, for example, the responses represent 45% to 50% of the population's viewpoint on a specific issue derived from those surveyed.
*Ask the survey sponsor for past surveys so you can identify any important trends.
(c) 2007 Money Management Executive and SourceMedia, Inc. All Rights Reserved.
http://www.mmexecutive.com http://www.sourcemedia.com
