Internal review of Governance of the Audit Working Paper Software Project

Internal review of Governance of the Audit Working Paper Software Project

Objective

To understand what elements contributed to how the audit working paper software (AWPS) project unfolded in order to improve outcomes for the next project and other similar projects for the Office of the Auditor General of Canada (OAG).

Context

The AWPS project was initiated in 2018 to replace the OAG’s audit working paper software. The project spanned several years during which significant changes affected the office, including the introduction of a formal project management governance framework.

The OAG purchased an off-the-shelf software solution and entered into a contract with the vendor to configure and customize the product to the OAG’s specifications.

The project team experienced ongoing challenges to configure the software with the OAG’s audit methodology and to deal with quality concerns. Despite these obstacles, the project team made slow but steady progress and expected to deliver a minimum viable product by the revised go-live dates. However, beginning in spring 2022, questions started arising about whether the intended outcomes of the project were attainable. An analysis was prepared in May 2022, and the decision to continue the project was made. Questions arose again in late August following a series of setbacks, and the project was eventually suspended and then cancelled in October 2022 when it was determined that its expected outcomes did not align with the desired future state of the organization, which had embarked on a digital transformation journey.

Oversight, communication, and collaboration

The quality and flow of communication was not always conducive to informed and timely decision making.

Oversight and challenge function roles were in place but not working effectively because of a lack of project management maturity, skills, and experience.

The project team had good collaboration at the working level. There were opportunities to strengthen collaboration at the more senior levels.

Recommendations

  • Management should provide the necessary resources to individuals who have strategic and project oversight roles so that they can effectively perform their roles according to their responsibilities and accountabilities, including for requirements related to performing an effective challenge function, providing and obtaining project briefings, and strategic thinking (the “how”).
  • The Chief Audit Executive should ensure that the observations stemming from project review engagements are openly and readily accessible office-wide.
  • The executive committee members should improve their ability to collaborate by holding each other accountable both individually and collectively.
  • Project sponsors should proactively leverage existing governance committees to clearly and appropriately communicate critical project health indicators and use these forums as the basis for collaboration and informed decision making throughout a project’s lifecycle.
  • Management should determine the threshold or conditions under which continued engagement with procurement to support contract monitoring may be required.
Maturing project management capacity

The Portfolio and Project Management team’s project management resources and guidance were evolving throughout the project lifecycle.

The project management plan was not completed with the necessary level of detail or rigour to adequately support the project.

Insufficient project planning created challenges for appropriate project execution.

Recommendations

  • The Portfolio and Project Management team should ensure that all project management methodology and lessons learned reports from previous projects are readily accessible office-wide and that it provides any necessary support to help project teams understand the guidance and tools that are available.
  • Project sponsors should ensure that key project artifacts, including the project management plan and associated plans therein, are completed with the appropriate level of detail to allow for meaningful use throughout the project lifecycle and are revised as needed.
  • Management should determine the skills required for project governance roles at the initiation of the project because these roles are assigned before preparing the project management plan.
  • Management should align the risk management methodology described in the Portfolio and Project Management team’s guidance with the project reporting templates and with the OAG’s Integrated Risk Management Framework.
  • Project sponsors should ensure that project risks are being assessed, managed, and reported in a manner that is consistent with the organization’s risk management framework, approach, and terminology. Discrepancies should be documented and communicated to all relevant stakeholders to ensure a common interpretation of risk information being presented.
Strategic assessment

Immediate needs drove the selection and configuration of the solution, lacking sufficient ongoing consideration of strategic factors and future needs.

Projects are not adequately incorporated in integrated planning activities and other corporate processes.

Recommendations

  • Project sponsors should periodically assess and confirm that the project objectives continually align with organizational strategic priorities and should adjust course when appropriate.
  • Contract business owners, supported by expert advice from the Procurement team, should ensure that contract requirements and the resulting procurement instrument align with how the project will be managed.
  • Management should assess if it needs to build internal capacity (the number of people and the skills available) in the area of procurement to address expected future needs in project procurement and assistance with contract monitoring.
  • Management should determine what supports are needed to assist project sponsors in identifying and costing their project resource needs and in determining what capacity and capabilities are available and what gaps may need to be addressed.
Management response

Management agrees with all of the recommendations and has developed a comprehensive management action plan to address them.