Retirees thinking of relocating abroad need to understand the pitfalls.

While warm climes, a lower cost of living and reduced taxes can hold great appeal, moving outside the U.S. for people with property and business interests is more difficult, according to Dean Deutz, a wealth solutions initiatives senior manager at RBC Wealth Management based in Minneapolis.

“What we see, especially with wealthy people, is that they go through an analysis and realize they could maybe move to a different country and save some tax money, but now they have all this complexity and see their family less,” says Deutz, who advises high-net-worth clients with assets over $5 million.

One difficult decision is determining whether clients should maintain their U.S. residency, especially if they are retaining their U.S. citizenship. Some clients may not want to give up ownership of their home, and so they will have to continue paying state taxes. Others may want to remain a U.S. citizen but move their primary residence abroad to try and leave the state income tax system. That can often save money in the long term, but is not as simple as just packing up and moving away.

In Minnesota, for example, you can give up your house, but you are not allowed to spend more than 183 days in the state unless you are paying income tax. And the state frequently audits those who change their residency by checking airline tickets, credit card records and other documentation.

“The other thing that we see complicate moving out of the U.S. or state tax system is business ownership,” Deutz says. “For some people, everything can line up, but they have a business they are a partial owner in and that brings them back into the state system.”

Exiting those business interests can be costly. Not only will a client need to pay more to bring on a tax attorney, but he or she will also have to relinquish or restructure their assets or businesses in the U.S.

One client Deutz is working with is considering moving his tax home to Costa Rica, which could save him $50,000 a year on taxes, but will cost $10,000 just to the state up front. “We’re still waiting to see if that’s going to work for him or not,” Deutz says. “He’s making a $10,000 bet to win $50,000.”

The decision to move involves more than finances, and advisors should also be prepared to help clients consider what the move would mean for their personal lives, including the likelihood of seeing less of their families.

“In our business, most of them end up deciding that the lifestyle, the family trumps the tax answer,” Deutz says.

Another consideration: Clients will likely have to drop their current advisor if they are living abroad. The SEC and FINRA have rules restricting trading on behalf of clients outside the U.S. Even for RBC Wealth Management, which has its parent company in Canada, an advisor based in the U.S. would not be allowed to continue serving a client who moved to Canada without additional and expensive registration.

For clients who are willing to forgo the tax advantages if they can still obtain the type of lifestyle that living abroad can offer, the most common solution is to become what Deutz terms “global retirees” who retain their residency and U.S. citizenship, but spend time in other countries.

“That’s what we see most often,” Deutz says. “It’s, ‘I have enough money to live where I want, and I’m going to do what I want rather than being driven by saving some dollars on taxes.’”

For more on retiring abroad see the full story here.