It's bad enough that 40% of workers in their 20s and 30s cash out their 401(k)s when they switch jobs, even though taxes and penalties decimate the balances to almost half, according to a CMI survey of 1,200 people in January commissioned by Fidelity.

Worse, even, a small percentage of 401(k) participants take out loans or hardship withdrawals from their retirement savings, which average only $122,000 in the first place, with a national median balance of $66,000, data from the Investment Company Institute and the Employee Benefit Research Institute shows.

Subscribe Now

Access to premium content including in-depth coverage of mutual funds, hedge funds, 401(K)s, 529 plans, and more.

3-Week Free Trial

Insight and analysis into the management, marketing, operations and technology of the asset management industry.