James Grogan is the vice president of strategic alliances at SunGard Recovery Systems of Wayne, Pa. SunGard will be among the firms speaking on a panel about disaster recovery for mutual fund companies at the National Investment Company Service Association's 2002 Technology Forum this week.
Grogan discussed the current state of disaster recovery and business continuity plans for financial services firms following the events of Sept. 11 with Mutual Fund Market News' Andrew Brent. An edited version of their conversation follows.
MFMN: What is your role at SunGard?
Grogan: I'm the vice president of strategic alliances. There are technology alliances between the business continuity group within SunGard and partners that have products or services complementary to our products or services. So, for example, they often are partnerships with either hardware vendors or software developers.
MFMN: What do recovery services encompass for financial services firms?
Grogan: The business continuity that we offer includes the full spectrum of traditional hot site services, where companies would be able to come to our facilities, restore their data and programs, and resume operations in the event that they had any kind of a catastrophic outage at their own facility.
MFMN: When you say, "come to [your] facilities," do you mean have employees physically come there?
Grogan: Yes. We have facilities throughout North America, as well as in the U.K. and France. When a customer subscribes to a service, they subscribe to it based on the systems that they are running in their production data center at home. We would make available to them those same type of systems at our facility, for them to be able to resume their production operation.
MFMN: What about data backup?
Grogan: Typically, in the traditional mode, the customer is going to be responsible for backing up their own data on a daily basis and moving that to an offsite location. The logistics of that are often handled by someone local to where their data center exists. So if they're in the New York area, for example, they may have someone, either in New York or in North Jersey who has the vaults where their tapes are stored.
As an alternative to that, some customers have moved out of the traditional backup solution and moved towards high availability' protection of their data. In that case, we would have electronic copies of their data, which are written from the customer's production facility over telecommunications lines to the SunGard facility, already at our facility in the event the customer experienced an outage in production.
MFMN: How many of your financial services customers use that "high availability" protection?
Grogan: We've probably got, in round numbers, one hundred customers who are using high availability solution. But there are thousands of customers that we have, so those who use high availability are still a small percentage of the customer base.
MFMN: What causes firms to choose one method versus the other?
Grogan: In the business continuity industry, the traditional way that most businesses, probably 95% of all businesses today, do their backup is to tape on a nightly basis. In the event of a disaster, they would bring those tapes with them to their commercial recovery facility, such as the SunGard locations.
The leading edge of new technology, which 5% or so of our customers are using, would shorten the recovery time frame by using an electronic mechanism whereby the data is written both in their production systems and at SunGard throughout the business day.
For customers that have any kinds of concerns about the recovery time objective, they have to come up with a technology that is faster than the physical transit of those tapes and the time it takes to restore tapes at an alternate location.
MFMN: In terms of disaster recovery, what happened to financial services firms on Sept. 11? Were most firms adequately prepared in terms of data protection?
Grogan: Most financial services firms were not impacted so much in their data centers as they were in their end-user areas. Because of the cost of real estate in lower Manhattan, many of the data centers have already been moved to other locations, particularly in North Jersey. Jersey City has many of the data centers from financial institutions. So the data centers themselves, by in large, were not directly impacted.
Most of what was impacted by the attacks in New York City were the end user recovery areas.
Of course, those end users need to be connected to the information that was in the data centers. The biggest effort that they had as far as their recovery plan was to reestablish, in an alternate location, that end user recovery capability. They needed to replace PCs on desktops and the ability for their employees to work from another office environment when they were unable to get to any of the locations in lower Manhattan.
Most of the customers that were in the World Trade Center had to scramble and find space in hotel locations. One of the large brokerage houses took over one of the hotels in midtown Manhattan, for example, and they're still running out of that facility. Others were very quickly looking to get a lease for other available commercial real estate [in] the New York metropolitan area, particularly in North Jersey and southern Connecticut.
MFMN: What was the biggest challenge for firms that set up offices in these makeshift locations?
Grogan: Being able to secure the actual desktop equipment that would be needed for their employees, especially considering airplanes [were] not flying. Most of that equipment would have been shipped across the country by FedEx or other air express carriers, so it was difficult for them to acquire equipment in the timeframe that they were looking to replace it.
MFMN: How can firms plan to have equipment ready ahead of time?
Grogan: We have contracts in place for replacement equipment, but as I said, most people had the least focus on [equipment]. Most of the people in the financial community who struggled with their recovery, struggled because they had to [decide] on the fly where they were going to have their employees go, and how they were going to secure the equipment that they would then have installed at those locations.
So there were many issues surrounding acquiring the equipment, installing the equipment, finding the location, getting all of that up and running and tested, and communicating to their staff where their new office locations were going to be.
In addition to that, they had the physical security issues that surrounded the whole country at that time to make sure that wherever they were now sending their employees they had appropriate security precautions in place.
MFMN: If firms want to contract to have facilities available in case of a disaster, is there an ongoing fee? How does the fee structure work?
Grogan: That varies by contract. For those customers using equipment that is based in our data centers only, there is a daily usage fee.
MFMN: Since Sept. 11, has it been easier to convince firms to spend the necessary money for disaster recovery information backup, as well as alternative offices?
Grogan: I think people are paying a whole lot more attention to both aspects, particularly companies that have [one] large, centralized office.
MFMN: With the heightened firm awareness of the importance of disaster recovery plans, are you dealing with the same people at the companies you service and that have come to you, or has upper management become more involved?
Grogan: We're seeing more and more involvement by upper management, whether that's the chief operating officer, chief information officer, chief security officer or CEO. Every large company, over this period of time, has had to present to their senior executives where their business continuity plans stand. And certainly the satisfaction of those executives with the answers in the plans they were presented is going to be seen in how they set their priorities for moving forward.
MFMN: Have you gained new clients since Sept. 11?
Grogan: We've continued to gain clients, and [many existing] customers [have] upgraded their contracts.
MFMN: We've heard that not only are fund companies looking more closely at their own data backup but also at how well vendors are backing up data from other companies with which they do business. Is that true?
Grogan: That's a very good question. So much business is done with trading partners in an electronic format. At my data center, everything is mirrored so that the data is very well protected. Senior executives are pressing their business continuity architects to make sure that they are protected to the degree that they need both internally and with the external interfaces that their company has.
You can imagine, for example, that for a data center in the financial community, so much of what they do is [interdependent] -- based on receiving electronic feeds from the various exchanges around the world. All of those feeds are coming through vendor relationships. So if any of them were to fail - if your data center is up and running but you're not seeing any of the global trading activity coming in - your application is, in essence, idle when you need it to be working. Clearly the coupling between trading partners is something that is being looked at closely from a business continuity preparedness point of view.
