Monitoring Report on the System of Quality Control—2018–19 Fiscal Year
Monitoring Report on the System of Quality Control—2018–19 Fiscal Year
Table of Contents
1. Executive Summary
This report outlines the process for monitoring the Office of the Auditor General of Canada’s (OAG’s) system of quality control and presents the 2018–19 results of the evaluation conducted by the OAG’s Compliance and Monitoring team. The monitoring process is designed to provide the OAG with reasonable assurance that the policies and procedures relating to the system of quality control are relevant, adequate, and operating effectively.
Overall, we found that the design of the OAG’s system of quality control was adequate and met the requirements of the Canadian Standard on Quality Control 1 (CSQC 1), issued by the Auditing and Assurance Standards Board. We also found that the system was operating as designed.
We identified an area for improvement and noted that the work of the teams who monitored the different parts of the process was delayed because of a number of factors, including personnel reasons. Timely reporting on the monitoring of the system and on the practice review results allows the OAG to take prompt corrective action. The teams involved are taking action to address delays.
2. Objective
The OAG must comply with the CSQC 1. This standard requires that a quality control system be established and maintained for all assurance engagements. It also requires the OAG to monitor compliance with quality control policies and procedures and report on its evaluation annually. We communicate the monitoring results to the Auditor General and management and recommend appropriate action where necessary.
Monitoring and reporting provide reasonable assurance that the OAG and its personnel comply with professional standards and applicable legal and regulatory requirements, and that the audit reports that the OAG issues are appropriate in the circumstances.
3. Scope
The scope of the monitoring process includes assessing the design, implementation, and operational effectiveness of the OAG’s system of quality control. We evaluated the OAG’s policies and procedures relating to the 6 elements of the system of quality control as identified in the CSQC 1 (see the “Elements and requirements of a system of quality control” section below).
The OAG’s monitoring process includes 2 distinct parts:
- Annual monitoring—An annual evaluation of the OAG’s compliance with its quality control policies and procedures that is conducted and reported on by the Compliance and Monitoring team. This work does not include inspecting specific assurance engagement files.
- Practice review—A cyclical inspection of completed assurance engagement files, including financial statement audits and direct engagements (performance audits and special examinations) that is conducted and reported on separately by the Practice Review and Internal Audit (PRIA) team. For each engagement leader, the PRIA team selects and inspects a completed assurance engagement at least once in every 4‑year period. The team’s results are included in the annual monitoring report.
Period of review
The monitoring process covered the period from 1 April 2018 to 31 March 2019.
Reliance on the international peer review
In 2017, the Auditor General requested that an international peer review be conducted to determine whether the OAG adhered to relevant legislation and professional standards applicable to the execution of its mandate. The review was also to determine whether the system of quality control for the direct engagement and annual financial statement audit assurance practices was suitably designed and effectively implemented. The review was conducted to provide the OAG with reasonable assurance that its work complied with all relevant professional standards. The review period was from 1 January 2017 to 31 December 2018.
We relied on this review in performing the monitoring process for the period from 1 April 2018 to 31 December 2018 and conducted our own work for the period from 1 January 2019 to 31 March 2019. The international peer review report and management’s action plan can be accessed from the External Reviews page of the OAG’s website. We did not duplicate its findings or management’s action plan in this monitoring report.
Management’s action plan will be followed up on next year.
Elements and requirements of a system of quality control
The following chart identifies the 6 elements of a system of quality control and the requirements for policies and procedures within each element, as outlined in the CSQC 1.
| Element | CSQC 1 requirements |
|---|---|
|
Leadership responsibilities for quality within the OAG |
|
|
Ethical requirements |
|
|
Acceptance and continuance of client relationships and specific engagements |
|
|
Human resources |
|
|
Engagement performance |
|
|
Monitoring |
|
Analysis conducted
To evaluate the design of the OAG’s system of quality control, we
- conducted interviews with the management employees who were responsible for areas under the 6 elements
- reviewed previous monitoring process results, the OAG’s audit methodology and system policies, internal risk assessments, and relevant external documentation that may have affected the system’s design
To evaluate the system of quality control’s operating effectiveness, we tested key controls, including any relevant changes to the system’s design.
Rating categories
As required by the CSQC 1, we rated any deficiencies found as follows:
- Significant deficiency—These matters require attention and prompt corrective action to comply with professional standards and legal and regulatory requirements.
- Needs improvement—These matters are less significant; improvements are needed but do not indicate that the OAG’s system of quality control is insufficient.
4. Findings
Overall, we found that the design of the OAG’s system of quality control was adequate and met the requirements of the CSQC 1. We also found that it was operating as designed, and we identified an area for improvement.
Needs improvement
The monitoring of the system of quality control, including practice review reports, provides value to the OAG when done on a timely basis. For the period under review, both the PRIA team and the Compliance and Monitoring team experienced delays.
Untimely reporting
The results of the monitoring of compliance with the system of quality control need to be communicated to the Auditor General and management annually. It is also important that PRIA provides its objective report on its findings and recommendations from practice reviews on a timely basis. Any delay in annual reporting or in reporting on practice review results affects the OAG’s ability to take prompt corrective action. We found that both the PRIA and Compliance and Monitoring teams experienced delays in completing work and issuing their reports because of a number of factors, including personnel reasons.
The PRIA team experienced delays in reporting its results. All 2018–19 individual financial statement practice reviews were completed and reported to engagement leaders by 31 March 2020. The majority of the individual 2018–19 direct engagement reports were completed and provided to engagement leaders by 30 April 2020. We acknowledge that the remaining 2018–19 direct engagement practice reviews were completed and reported to engagement leaders by 31 August 2020. The practice reviews did not identify any significant findings. The summary of findings from practice reviews for the 2018–19 fiscal year can be found on the Practice Reviews page of the OAG’s website.
The PRIA team has improved the timing of its reporting by using “on-time” reporting for the 2020–21 cycle. This allows management to take action sooner to inform the next cycle of audits. The team will publish observations on the OAG’s INTRAnet after the team and engagement leaders have accepted a minimum of 2 individual reports. After all practice reviews have been completed, a summary report will be published on the OAG’s website once it is approved by the Auditor General.
The Compliance and Monitoring team also experienced delays in reporting its results. The monitoring of the system of quality control is traditionally performed and reported on annually. The team acknowledges that the completion of this report is late. Planning for the monitoring process for the 2019–20 fiscal year is underway, and reporting is expected by July 2021. Work on the process for 2020–21 will begin immediately thereafter. The team is committed to re‑establishing timely monitoring and reporting.
Follow-up on the 2016–17 recommendations
We followed up on recommendations from the Monitoring Report on the System of Quality Control—2016–17 Fiscal Year. No significant deficiencies in the system of quality control were identified. The report noted 4 issues that needed improvement, but these issues did not affect the system from operating effectively. Of these, 2 were dealt with sufficiently during the 2016–17 reporting period, and no further work was conducted during the current monitoring process. The following chart provides more information on the other 2 issues.
| CSQC 1 element | Finding | Recommendation | Follow-up |
|---|---|---|---|
|
Human resources |
Although the review of the “realistic profile for audits” helped assess staffing needs, it was performed only once every few years. |
Audit practices should perform more frequent reviews of the sufficiency (capacity) of personnel and the appropriateness of key assumptions used. |
Completed. We found that the realistic profile had been reviewed. In addition, Human Resources now provides the audit practices’ management with monthly planning reports, which include information and data for improved planning. The practices also receive detailed annual information, and by tabling period, as relevant. |
|
Engagement performance |
The OAG’s archiving process of engagement files may not ensure documentation has been completed on or before the 60‑day completion date. |
The OAG should strengthen its controls for engagement documentation to make it possible to verify that the documentation has been completed on or before the 60‑day completion date. |
Completed. The Direct Engagement Practice Team added a step to the TeamMate methodology to remind teams to notify Records Management of the date of the report. This reminder allows Records Management to follow up on audits for which a notification email has not been received in order to verify that the audit has been closed within 60 days after completion. |
5. Compliance and Monitoring Team
Principal: Annie Leclerc
Director: Marianne Avarello
Audit Project Leader: Maxine Leduc
Senior audit professionals:
Jessica Semrau
Tony Tran
Mathieu Drainville