LPL Financial's 14,000-plus advisers nationwide experienced a network service outage for nearly 24 hours in the middle of last week’s unprecedented WannaCry global cyberattack that the company says was unrelated to its failure. The blackout revealed vulnerabilities that one prominent adviser called “embarrassing.”
“Last Friday at 2:38 p.m. ET, a construction crew damaged underground cables that provide critical network connectivity for LPL," the firm said in a statement. "Our backup connection experienced an unrelated technical issue and was unable to provide support as planned."
The full-day failure – starting on a weekday when the markets were open – exposed a critical weakness in the service its advisers and their clients depend upon. It also caused one expert to question LPL's explanation.
'WHOLE STORY'?
"It can't be the whole story," says Brian Edelman of Financial Computer of Bloomfield, New Jersey, who works with some of the country's largest financial firms on cybersecurity. "The fact is that they had an outage that shouldn't have happened. They are claiming it was because they had an internet line that went down. That in itself is an issue."
The IBD told its advisers and FINRA that its outage was unrelated to WannaCry. The malware attack, first discovered last Friday at about 4 a.m. ET, has encrypted hundreds of thousands of users' computers around the world with ransomware, according to the Department of Homeland Security. In this type of strike, hackers attempt to ransom their victims' data back to them by demanding payment for decryption keys. Various cybersecurity experts have said the incursions may have emanated from North Korea.
Regarding LPL's service loss, a spokesman for the company's communication and data service provider, Windstream, confirmed that workers severed a fiber optic cable line near LPL's offices in Charlotte, North Carolina.
By 9 a.m. Saturday, Windstream restored service to customers, the spokesman says. Four hours later, LPL started bringing its systems back online.
The cutting of cable lines should not have caused a systemwide failure, says Edelman, whose firm works with TD Ameritrade, Schwab, Fidelity, Pershing and UBS, among others. He does not provide services to LPL.
Large financial institutions maintain multiple data centers in different geographic areas to back up their services instantaneously in case of a problem in one area, he says.
'EMBARRASSING'
These centers are in constant communication with one another so that, when one or two of them fail, there is no interruption in vital services, such as during catastrophes like Hurricane Sandy in 2012, Edelman says. However, ransomware can cripple a network of data centers instantly due to their interconnectivity. The fact that all of LPL's systems failed makes Edelman believe the company was hacked or that LPL has underfunded its IT and cyber protections.
"You are spending millions and millions of dollars to back up your systems and you're trying to say someone tripped on a cord" that brought everything down?, he asks.
LPL did not provide an explanation of how damage in one spot could have foiled a broadly networked backup system, or how the secondary backup failure may have factored into the scenario Edelman described.
REDUNDANCY TESTING
One LPL adviser, John Hyland, managing director of LPL's largest office of supervisory jurisdiction, Private Advisor Group, with 621 advisers in Morristown, New Jersey, calls the outage "embarrassing."
"The little I know about technology is that a redundancy system should be tested constantly," Hyland says. "How does the redundancy plan fail at the very same moment that they severed the fiber optic line?"
Fortunately, he says, the interruption came at a "light" time -- although the markets were open and investors have been pushing indexes to record highs. Throughout the outage, LPL kept advisers informed about the problem and efforts to fix it, he says, and "nothing has bubbled up to me about advisers being upset."
Next time, perhaps LPL will be more prepared, he added. "When we have mistakes hopefully we all learn."
