Monitoring Report on the System of Quality Control—2020–21 Fiscal Year

Report date: April 2022

1. Executive Summary

This report outlines the process for monitoring the Office of the Auditor General of Canada’s (OAG’s) system of quality control and presents the 2020–21 results of the evaluation conducted by the OAG’s Monitoring team. The monitoring process is designed to provide the OAG with reasonable assurance that the policies and procedures relating to the system of quality control are relevant, adequate, and operating effectively.

Overall, we found that the design of the OAG’s system of quality control was relevant and adequate and met the requirements of the Canadian Standard on Quality Control 1 (CSQC 1), issued by the Auditing and Assurance Standards Board. We also found that the OAG’s system of quality control operated effectively at the engagement level. In the practice reviews of engagements, the auditor’s reports were supported and appropriate. We did not identify any significant deficiencies.

However, there is an opportunity for the OAG to improve how its human resources policies and procedures provide it with reasonable assurance that it is managing its human resources risk effectively and that it has sufficient and appropriate information at the OAG level to make strategic human resources decisions. Such assurance would demonstrate that the OAG would continue to have sufficient personnel with the competence and capacities to address the challenges of the future. We noted a number of areas where the design and implementation of the human resources element in the OAG’s system of quality control could be improved. Nevertheless, the issues we identified did not affect the appropriateness of the reports issued by engagement leaders in the 2020–21 fiscal year.

Because we reviewed the OAG’s system of quality control for the 2019–20 and 2020–21 fiscal years concurrently, the needed improvements that were identified for the human resources policies and practices apply to both fiscal years. The detailed findings related to the needed improvements can be found in the 2019–20 monitoring report. The OAG is developing an action plan to address our findings and recommendations.

The Practice Review and Internal Audit (PRIA) team completed its inspections of financial audits and direct engagements for the 2020–21 fiscal year, and in those practice reviews, it concluded that all of the audit reports issued were supported and appropriate. However, in the direct engagement files it reviewed, it found 1 instance of non-compliance with the engagement performance element of the system of quality control. Because it concluded that the finding was an isolated case, it did not issue any formal recommendation.

In our view, the non-compliance does not indicate an irrelevant or inadequate system of quality control but relates to a set of unique circumstances that affected the implementation of the OAG’s system of quality control in that 1 direct engagement. The finding should contribute to continuous learning and improvement and represent an opportunity for senior management, the audit methodology team, and audit teams to learn from it.

2. Objective

The OAG must comply with the CSQC 1. This standard requires that a quality control system be established and maintained for all assurance engagements. It also requires the OAG to monitor compliance with quality control policies and procedures and report on its evaluation annually. We communicate the monitoring results to the Auditor General and management and recommend appropriate action where necessary.

The system of quality control is to provide reasonable assurance that the OAG and its personnel comply with professional standards and applicable legal and regulatory requirements, and that the audit reports that the OAG issues are appropriate in the circumstances.

3. Scope

The scope of the monitoring process includes assessing whether the policies and procedures relating to the OAG’s system of quality control are relevant, adequate, and operating effectively. We evaluated the OAG’s policies and procedures relating to the 6 elements of the system of quality control as identified in the CSQC 1 (see the “Elements and requirements of a system of quality control” section below).

The OAG’s monitoring process includes 2 distinct parts:

• Annual monitoring—An annual evaluation of the OAG’s compliance with its quality control policies and procedures that is conducted and reported on by the Monitoring team. This work does not include inspecting specific assurance engagement files.
• Practice review—A cyclical inspection of completed assurance engagement files, including financial statement audits and direct engagements (performance audits and special examinations), that is conducted and reported on separately by the PRIA team. The PRIA team selects a risk-based sample, ensuring that each engagement leader is selected at least once every 4 years.

Period of review

The monitoring process covered the period from 1 April 2020 to 31 March 2021.

Elements and requirements of a system of quality control

The following chart identifies the 6 elements of a system of quality control and the requirements for policies and procedures within each element, as outlined in the CSQC 1.

Monitoring procedures performed

To evaluate the design of the OAG’s system of quality control, we

• conducted interviews with the management employees who were responsible for areas under the 6 elements
• reviewed previous monitoring process results, the OAG’s audit methodology and system policies, internal risk assessments, and relevant external documentation that may have affected the system’s design
• interviewed selected staff at all levels, including regional office staff, using a questionnaire customized from one included in the Chartered Professional Accountants of Canada’s Quality Assurance Manual, to ascertain staff’s knowledge of, and compliance with, the OAG’s quality control policies

To evaluate the system of quality control’s operating effectiveness, we tested key controls, including any relevant changes to the system’s design.

Rating categories

As required by the CSQC 1, we rated any deficiencies found as follows:

• Systemic, repetitive, or significant deficiency—These matters require attention and prompt corrective action to comply with professional standards and legal and regulatory requirements and ensure that the reports issued are appropriate in the circumstances.
• Needs improvement—These matters are less significant; improvements are needed but do not indicate that the OAG’s system of quality control is insufficient to provide it with reasonable assurance that the system of quality control complies with professional standards and applicable legal and regulatory requirements, and that the reports issued are appropriate in the circumstances.

4. Findings

Needs improvement

Overall, we found that the design of the OAG’s system of quality control was relevant and adequate and met the requirements of the CSQC 1, issued by the Auditing and Assurance Standards Board. We also found that the OAG’s system of quality control operated effectively at the engagement level. In the practice reviews of engagements, the auditor’s reports were supported and appropriate. We did not identify any significant deficiencies.

Human resources element

We noted that some areas related to the design and implementation of the human resources element of the OAG’s system of quality control could be improved. Although the issues we identified are systemic and require corrective action, they did not affect the appropriateness of the reports issued by engagement leaders in the 2020–21 fiscal year.

Because we reviewed the OAG’s system of quality control for the 2019–20 and 2020–21 fiscal years concurrently, the needed improvements that were identified for the human resources policies and procedures apply to both fiscal years. The detailed findings related to the needed improvements can be found in the 2019–20 monitoring report for more insights.

More specifically, there was a lack of evidence at the OAG level that

• performance objectives were received by all professional staff and that performance discussions were occurring for all.
• staffing decisions were made according to up‑to‑date knowledge of the OAG’s current professional staff skills and capacities, the current gaps in competence and capability at the engagement level, and planned future staff needs
• succession planning decisions were made with medium- and long-term perspectives
• mandatory training and designation requirements for professional staff were effectively monitored
• rationales for staff departures were assessed on a timely basis to monitor retention risks

In the last few years, the OAG has reviewed and updated many of its human resources policies and procedures. However, it has not updated the mapping of this work to the CSQC 1 to ensure that it continues to address the standard’s human resources element. Also, the OAG has not identified the controls and related control owners that are needed to monitor the effectiveness of its human resources policies and processes and its compliance with the related CSQC 1 standard’s objective.

Nevertheless, the PRIA team completed its inspections of financial audits and direct engagements for the 2019–20 fiscal year and did not identify any significant findings at the engagement level.

To strengthen the design and implementation of the human resources element of its system of quality control, we recommended that the OAG should

• always keep the mapping of its human resources policies and procedures to the CSQC 1 up to date to ensure that the quality control system addresses the requirements for the human resources element
• clarify the information the OAG needs to collect, use, and report on from its system of quality control at the OAG level to give it the assurance that its human resources systems and practices are operating effectively on an ongoing basis
• review the form and content of documentation detailing the operations of the human resources element to ensure that the OAG demonstrates that it has sufficient personnel with competence, capabilities, and commitment to ethical requirements now and in the future, that human resources risks are effectively managed, and that the expectations of the current quality control standard and the upcoming Canadian Standard on Quality Management are met

Management agreed with our recommendations. The Human Resources team will work in consultation both with the managers responsible for implementing the different elements of the revised corporate planning process and with the methodology team responsible for implementing the new CPA Canada standards for quality management by the end of 2022. This work will provide senior management with assurance that human resource policies and procedures are aligned with the new standards’ requirements. The Human Resources team will work with the Finance team to collect sufficient and appropriate information about human resource risks and staffing sufficiency, competence, and capabilities. This information will be reported regularly to senior management.

The OAG is developing an action plan to address our findings and recommendations.

Engagement performance element

The PRIA team completed its practice reviews of financial audits and direct engagements for the 2020–21 fiscal year and concluded that all of the audit reports were supported and appropriate. However, in the direct engagement reviews, it found 1 instance of non‑compliance with the engagement performance element of the system of quality control.

The PRIA team identified the following most common or significant observations that came out of this review cycle:

• preparing and reviewing key documentation before the date of the audit report
• documenting audit conclusions against audit criteria and overall
• documenting consultations with internal specialists
• identifying a single engagement leader responsible for the overall quality of the engagement

The above observations were all present in the non-compliant file.

The objective of the system of quality control is to provide reasonable assurance that the OAG and its personnel comply with professional standards and applicable legal and regulatory requirements, and that audit reports that the OAG issues are appropriate in the circumstances.

We considered the information provided by the PRIA team and determined that the non-compliance does not indicate an irrelevant or inadequate system of quality control but relates to a set of unique circumstances that affected the implementation of the OAG’s system of quality control in that 1 direct engagement. The finding should contribute to continuous learning and improvement and represent an opportunity for senior management, the audit methodology team, and audit teams to learn from it.

The PRIA team discussed opportunities for learning and improvement with the direct-engagement assistant auditors general and the OAG methodology team and will discuss them with the direct engagement practice at an upcoming lessons-learned session.

5. Follow‑up on the 2019–20 Recommendations

As indicated above, the 2019–20 monitoring report was completed concurrently with the 2020–21 report. Therefore, we plan to follow up on the recommendations to strengthen the design and implementation of the human resources element of the OAG’s system of quality control in our 2021–22 monitoring report.

6. New Quality Management Standards

New quality management standards were issued in Canada in May 2021. The OAG continued its project to design a risk assessment process that complies with these new standards and other analyses of the implications of these standards.

7. Monitoring Team

This monitoring process was completed by a multidisciplinary team from across the OAG that was led by Annie Leclerc, Principal. She had the overall responsibility for the monitoring process, including conducting the process in accordance with professional standards.