Results from the Disaster Recovery Preparedness 2014 Annual Report show almost three quarters of organisations worldwide have relegated their disaster recovery initiatives to the sidelines by not taking adequate steps to protect their data and IT systems. According to participants, poor planning, testing and technological deficiencies have led to instances in which more than $5M worth of critical applications failure, data centre outages and data loss have occurred.
At a time when application outages such as those experienced in the financial sector by the likes of HSBC and RBS are grabbing front page headlines, it’s very surprising that this many organisations haven’t gotten themselves enough into the game to sort out their disaster recovery strategies.
The study polled more than 1,000 organisations, from small businesses to large enterprises, to help them benchmark readiness for critical IT systems recovery in virtual environments. Results indicate that most companies are woefully unprepared to recover their critical business and IT systems in the event of a disaster.
- 64% of respondents surveyed say that their organisation’s disaster recovery (DR) budget is inadequate and underfunded.
- More than 60% report that they do not have a fully documented disaster recovery plan, and among the minority that does, 23% of respondents have never tested those plans.
- Approximately one-third say that they test their plan only once or twice a year, and more than 65% of those do not pass their own DR tests.
- 78% of respondents have experienced outages of critical applications, and of that group, 63% say that losses ranged from a few thousand dollars to over $5M.
- Of the respondents who have experienced outages, approximately 28% say their organisation lost datacenter functionality for weeks at a time in the worst instances.
I appreciate the difficulties organisations recovering their data from virtual environments. Leveraging only traditional backup and recovery software, it’s an extremely difficult environment in which to work. The servers have to be given new IP addresses and then turned on in exactly the right order.
Otherwise applications will fail, systems will jam and information will get lost. If you flick a switch in the wrong order, you have to go back and start all over again, something very trying in a real-life situation when the clocks are on, the CEO is screaming and the organisation’s reputation is in accelerating jeopardy.
Moreover, if data has been corrupted on the primary site, traditional migration and replication software will migrate and replicate the corrupt data to the second remote site, so there’s even less chance of a “straight-forward” recovery.
Of course, it’s easy to understand why companies shy away from regularly testing their backup and recovery plans. It can be extremely disruptive, so it has to be done at a quiet time (such as a bank holiday), which dramatically increases its cost. Note: Gartner suggests annual disaster recovery testing can reach or exceed $150,000 for many of its clients.
However, it’s worth noting that the majority of respondents surveyed acknowledge their disaster preparedness deficiencies and report that their organisation is now planning or revising its implementation strategy by:
- Building a comprehensive DR plan to recover applications, networks and business services, including primary and secondary sites.
- Defining recovery time objectives (RTOs) and recovery point objectives (RPOs) for critical applications to set proper expectations and assumptions for management and staff.
- Automating frequent recovery testing for critical applications to validate their recovery capabilities within specified RTOs/RPOs.
It will be interesting to compare these results with the 2015 survey next year. But there is consolation to be had in this year’s returns. Many top scoring organisations, for instance, are starting to fully document their disaster recovery plans and clearly identify critical applications together with their vital components so they understand their priorities for what to recover first.
These companies are learning that the characteristics of a good DR plan are much more than disparate lists of emergency phone numbers built into call trees. Successful disaster recovery in today’s multi-environment IT landscape must focus on applications and services, not individual files and servers. Recovery must be seen terms of such things as payroll, exchange, customer service centres and website availability.
IT providers at the forefront of the back-up and disaster industry have made already made that transition and are now developing the next generation of intelligent, multi-environment solutions where recovery is far more assured and reliable. These solutions can automatically test data validity and integrity every hour, and send out an immediate alert if they detect any deviations or corruptions.
It’s the kind of automated, round-the-clock, proactive readiness capability that will help companies take their recovery off the sidelines, get into the recovery game for real and start playing it safe with arguably the most important assets in their organisation.