WA council’s IT infrastructure runs on a single server

A recent audit conducted by the Auditor General of Western Australia has highlighted significant deficiencies in the disaster recovery capabilities of several local councils. The findings reveal a concerning lack of preparedness that could severely impact the councils' ability to maintain operations and deliver essential services in the event of an IT disaster.

One particularly troubling example uncovered by the audit involves an unnamed council that operates its entire IT system from a single physical server. This setup is inherently risky due to the absence of redundancy, making it highly vulnerable to a single point of failure.

The council’s disaster recovery plan, which stipulates a 48-hour server replacement timeframe, was found to be inadequate as it lacked specific details and was not supported by a comprehensive vendor agreement. The plan did not specify the server’s hardware specifications or outline the vendor's role in disaster recovery, leaving the council at risk of prolonged downtime and service disruption.

The audit examined the disaster recovery plans of six local councils and found that none were sufficiently prepared to manage IT disasters and fully recover key systems. Despite acknowledging the importance of disaster recovery planning, only one council's plan was considered adequate, and none had undergone proper testing. This lack of testing raises significant concerns about the effectiveness of the plans and the councils' ability to recover from a major IT incident.

Most councils had developed disaster recovery plans, but these plans often lacked critical details and were not comprehensive. In one case, a council had not documented any recovery plan at all.While all councils had experience in restoring individual data files from backups as part of daily operations, none had tested the full recovery of their IT systems. This lack of testing means councils cannot be confident in their ability to restore systems fully or ensure data consistency across applications in the event of a full system failure.

The audit revealed a heavy reliance on IT vendors for disaster recovery planning and testing, but most councils had not established detailed service agreements with their vendors. One council only had a verbal arrangement, which was formalized into a written agreement only after the audit. 

Caroline Spencer, the Auditor General of Western Australia, noted that although it was encouraging to see that councils recognized the importance of disaster recovery planning and had developed plans, none were fully prepared for a disaster scenario. Spencer emphasized the critical role of timely IT system recovery in mitigating financial and reputational losses and minimizing delays in service delivery to the public.

The audit’s findings underscore the urgent need for local councils to develop comprehensive and well-tested disaster recovery plans. To enhance their preparedness, the following steps are recommended: Develop Detailed Disaster Recovery Plans. Councils need to create comprehensive disaster recovery plans that include specific details about hardware and software specifications, defined roles and responsibilities, and clear recovery time objectives (RTOs) and recovery point objectives (RPOs).

It is essential for councils to establish formal agreements with their IT vendors.  testing of disaster recovery plans is crucial to ensure their effectiveness. These tests should simulate various disaster scenarios to identify potential weaknesses and areas for improvement. Staff should be regularly trained on disaster recovery procedures and the importance of these measures. This training ensures that everyone understands their role in the event of a disaster and can act swiftly and effectively.

Disaster recovery plans should be living documents that are regularly updated to reflect changes in the IT environment, emerging threats, and lessons learned from testing and real incidents.

The audit’s findings reveal significant gaps in the disaster recovery capabilities of Western Australian councils, posing a substantial risk to their operational continuity and service delivery. These deficiencies highlight the importance of comprehensive disaster recovery planning and the need for councils to take proactive steps to address these vulnerabilities.

By developing detailed disaster recovery plans, formalizing vendor agreements, conducting regular testing, and ensuring ongoing staff training and awareness, councils can significantly enhance their resilience and preparedness for IT disasters. These measures will not only protect the councils’ operations but also ensure that they can continue to deliver essential services to the public with minimal disruption.

The audit of Western Australian councils’ disaster recovery capabilities has revealed critical areas of concern that need urgent attention. The findings underscore the importance of comprehensive disaster recovery planning and highlight the need for councils to take proactive measures to enhance their preparedness.

By addressing these deficiencies, councils can ensure that they are better equipped to maintain operations and deliver essential services in the face of IT disasters, ultimately protecting public services and minimizing potential disruptions. The proactive steps recommended by the Auditor General provide a clear roadmap for achieving this goal and underscore the importance of ongoing vigilance and improvement in disaster recovery planning.