Summary of Lessons Learnt from AIMS

26 May 2023

In 2019, Te Rua Mahara o te Kāwanatanga Archives New Zealand initiated the Archival Integrated Management System (AIMS) Project to replace the previous system, Archway. Archway had reached the end of its life, posed security risks, and could no longer be supported.

In February 2022, we launched our new Collections Search tool. Several technical issues led to poor user and staff experience. Ongoing issues with poor functionality, bugs, and unplanned disruptions in service caused increasing levels of dissatisfaction from users and government agencies.

Since the system went live, we have been working to address immediate issues experienced by our users, and to understand how we might learn from the original project delivery. We have been supported by the Department of Internal Affairs, including establishing Incident Management Teams. Updates to Collections in January 2023 have resolved most of these issues and significantly improved the search function.

We have also engaged independent experts to inform the development of an integrated remediation plan for the system. In October 2022, ForeConsulting was contracted to undertake a lessons learnt review, which identified several critical areas of focus for successful implementation. These are provided in summary below, and the full report is released for reference. We are providing this information so that other agencies contemplating similar system upgrades can benefit from our experiences.

We are still responding to these lessons but have made progress in some key areas, including developing a business case to improve the system, streamlining business processes and supporting change management. Our response includes ensuring staff appointed to project governance roles are appropriately trained; setting up the next phase of the project, which has a focus on training, user engagement and change management; and commissioning independent quality assurance reviews for the next phase of the project.

Staff and external users should be appropriately represented, engaged, and consulted at the right times throughout the development of a project. Key user groups need to have opportunities to test a new product and provide feedback before go-live.

Risk management practices should be comprehensive, ensuring all risks and mitigations are well understood and reviewed regularly by the project team, stakeholders, and leadership. There should be clear criteria that a project needs to meet before the product goes live, including an assessment on whether performance testing is required.

Ensure vendor references are taken only from the most comparable or relevant projects.

Ensure vendor can provide support in a compatible time zone to optimise response times.

Ensure that Governance roles and accountabilities are clearly understood.

Undertake independent reviews to examine the project and provide quality assurance.

Ensure project sponsors are holders of a leadership position within the organisation and have control or influence over the business area or resources into which the project outcomes will be delivered.

Agree and document clear and detailed acceptance criteria well ahead of go-live and ensure go-live decision making aligns with the agreed acceptance criteria.

Ensure project resources remain engaged post go-live, to provide support for any issues.

Prior to go-live, undertake an independent review that examines the project in the same way as the Gateway Review Process.

Read the Lessons Learned report (PDF, 9.32MB)