True business continuity means keeping your business running. If your people cannot get back to work and be productive, all of the resources that you expended to protect your data centers and keep them running was a waste of money. As I wrote in an earlier post, companies can contract with SunGard, Rentsys, IBM, HP, or a handful of other companies to either use space in a fixed facility, or to have a trailer driven or airlifted to a convenient location. What I didn’t discuss in that earlier post is the preparation by your vendor and your company when your contract is first signed, and then after you have declared a disaster and need to move in to a workspace recovery unit.
Like Chinese Food
You go into a Chinese restaurant and order from the menu. A few minutes later, your food shows up. Looks easy, right? What you don’t know is that well before the restaurant opened and all throughout the day, there is a substantial amount of prep work going on behind the scenes. Unlike a steak restaurant where you cut your meat and vegetables into bite-sized chunks at the table, hundreds of pounds of meat and vegetables need to be cleaned and chopped or diced in preparation for being tossed into the wok for cooking.
Similarly, there is a substantial amount of prep work that needs to be done before you can move into a workplace recovery center. After you sign your contract, you and your vendor will work together to develop the infrastructure your company needs to get your employees back to work. One of the first steps includes building and configuring the infrastructure to connect to your backup data center (which may be self-hosted or hosted by the same or a different recovery vendor). Unless your network or server configuration changes dramatically, maintenance is fairly easy.
Endpoint Preparation
One of the more involved aspects of workplace recovery is configuring your endpoints. In English, this means setting up the PCs that your workers will actually be using in the recovery center. The prep work involved in setting up your PCs is extensive and ongoing. The “gold master” (GM) configuration that you use for your desktops and laptops is a good starting point, but you might be using Dell PCs while your workforce continuity provider might supply Lenovo or HP computers. This means that in addition to testing your GM on your own computers, you also need to test it on the provider’s computers as well. And you can’t just stop there and burn a GM CD because Microsoft and Adobe are constantly releasing patches to fix bugs and close zero-day exploits. So whenever you patch your internal computers, you need to run the same tests on your provider’s computers and create a new GM.
Well I Declare!
When you declare a disaster, your vendor starts the provisioning process. Whether the workplace recovery center is fixed or mobile, this means setting up the required network connections within the vendor’s network so that the recovery center can connect with your data center. If you’ve signed up for mobile recovery services, one or more mobile recovery centers will be dispatched to your specified location. When they arrive, the trailers, generators, and satellite dishes will be deployed. After the mobile center is up and running, the IT experts begin the long process of setting up all of the endpoints.
Whether mobile or fixed, this process is the same. Each PC needs its internal hard drive erased to ensure that you company cannot gain access to any of the information that might have been on them from a previous deployment. If you want the drives erased to US department of defense standards, the number of wipes may increase to 10, 20 or even 30. If the drives are large, this can take days. After the drives are erased and formatted, one or more gold master CDs are used to lay down the operating system configuration required to fit your environment. If the GM is old, the computers may need to be booted then patched to the latest software, which can take more hours to days depending on how many computers need to be provisioned, and how out of date the GM CD is.
And the Point is?
Has your company built the time needed to image your PCs into your recovery time objective (RTO)? Do you have a gap if your RTO is substantially shorter than your recovery time capability (RTC)? Perhaps the answer lies in using virtual machines which are always running, and ready to be used when disaster strikes. Until you exercise your plan, you’ll never know!
