Introduction

1. The Office of the Auditor General conducts independent audits that provide objective information, advice, and assurance to Parliament, territorial legislatures, and Canadians. The Office has several product lines, including performance audits, annual audits, and special examinations. Performance audits and special examinations are referred to as direct report engagements.

2. A performance audit is an independent, objective, and systematic assessment of how well government is managing its activities, responsibilities, and resources. Performance audits contribute to a public service that is ethical and effective and a government that is accountable to Parliament and Canadians. Performance audits are planned, performed, and reported in accordance with professional auditing standards and Office policies.

3. Special examinations are a form of performance audit that is conducted within Crown corporations. The Office of the Auditor General of Canada audits most, but not all, Crown corporations. The scope of special examinations is set out in the Financial Administration Act. A special examination considers whether a Crown corporation’s systems and practices provide reasonable assurance that its assets are safeguarded, its resources are managed economically and efficiently, and its operations are carried out effectively.

4. The Canadian Institute of Chartered Accountants (CICA) establishes professional assurance standards that govern the conduct of audits. The Office establishes additional policies and procedures to guide its work. These are contained in an audit manual, various other audit tools, and a system of quality control (SoQC) that practitioners must follow. There is a product leader at the assistant auditor general level for each of the performance audit and special examination product lines. The primary functions of the product leaders are to provide leadership and oversight for the product line and to contribute to the quality of the individual audits.

5. The Practice Review and Internal Audit (PRIA) team conducted practice reviews of six selected direct report audits that were reported in 2013 and one that was reported in 2012. This practice review work was done in accordance with the monitoring section of The Canadian Institute of Chartered Accountants (CICA) Handbook—“Quality Control for Firms that Perform Audits and Reviews of Financial Statements, and Other Assurance Engagements (CSQC 1).” It was also done in accordance with the Office’s 2013–14 Practice Review and Internal Audit Plan, which was recommended by the Audit Committee and approved by the Auditor General. The plan is based on systematic monitoring of the work of all audit principals in the Office, on a cyclical basis.

6. This report summarizes the major observations related to the practice reviews of the selected direct report audits.

Overview

Objective

7. The objectives of practice review are to provide the Auditor General with assurance that

• audit principals (the practitioners) comply with professional standards, Office policies, and applicable legislative and regulatory requirements; and
• audit reports are supported and appropriate.

Scope and methodology

8. We planned and conducted seven practice reviews of direct report audits in the 2013–14 fiscal year, including six performance audits and one special examination audit. The reviews were conducted on six audit files that were reported during the 2013 calendar year and one that was reported during the 2012 calendar year. The latter had been rescheduled from the previous year due to exceptional circumstances affecting the availability of the audit team.

9. Our reviews included an examination of electronic (TeamMate) and paper audit files. We reviewed documentation related to the planning, examination, and reporting of the audits. We also interviewed certain audit team members, engagement quality control reviewers (EQCRs), and other internal specialists, as appropriate.

System of Quality Control elements and process controls reviewed

10. We focused our work on the selected elements (Appendix A) and process controls (Appendix B) of the System of Quality Control (SoQC) that we considered key or high risk.

11. We also looked at how the EQCRs carried out their responsibilities. EQCRs are management-level employees of the Office who are appointed to provide an objective evaluation for audits assessed as having a higher risk. Before the auditor’s report is issued, the appointed reviewer evaluates the significant judgments that the audit team made and the conclusions that it reached when it formulated the audit opinion. The EQCRs are an important element of the Office’s SoQC—they are involved in selected individual audits from the initial planning decisions to the closing of the audit file.

Rating system

12. We applied one of the following ratings to each selected SoQC element of the individual audits under review, as well as to the audit file overall:

• Compliant. Office policy requirements and applicable auditing standards were met.
• Compliant but needs improvement. Improvements are necessary in some areas to fully comply with Office policies and professional auditing standards.
• Non-compliant. Major deficiencies exist; there is non-compliance with Office policies or professional auditing standards.

13. After completing each practice review, we concluded on whether the audit opinion was supported and appropriate.

14. This report highlights the procedures performed, the observations and recommendations made, and management responses.

Results of the Reviews

Appropriateness of the audit reports

15. Overall, we found that in all seven files reviewed, the audit opinions were supported and appropriate.

Compliance with the system of quality control and process controls

16. Generally, the level of compliance with the System of Quality Control (SoQC) elements was very good. Three files were fully compliant with professional standards and Office policies, and two files had some opportunities for improvement noted. Two files were non‑compliant primarily in one area: a lack of evidence of the nature and extent of the practitioner’s review.

17. Of the seven files reviewed, three had been assigned an engagement quality control reviewer (EQCR). Two of these files had weaknesses in this area. In one file, the audit report date was prior to the EQCR final review. In another file, the EQCR was not engaged by the practitioner in a manner that allowed timely review during various stages of the audit.

Observations

18. We are unable to report statistically significant practice-wide observations in this report.^{Footnote 1} Such observations require a larger sample of reviews. The larger sample will be available for our 2014–15 report. The statistically significant practice-wide observations will cover the 2012–13 to the 2014–15 fiscal years. The observations included in this report relate only to the seven files we reviewed in the 2014 calendar year.

19. One area in particular—planning—was well done. The Audit Logic Matrix, audit programs, advisory committees, and other fundamental planning documents were prepared and well documented. Practitioners documented well their involvement in the files at this stage.

20. Likewise, post-tabling procedures for analyzing budgets and ensuring the files were closed within deadlines were also well done. This was the case in all seven files.

21. We found the elements of engagement management and consultations were generally well done. Documentation was usually easy to follow and properly reviewed. There were some areas where improvements were identified, but most files were compliant with standards and policies.

22. We did note non-compliance issues in some audit files during the examination and reporting phases. In two of the seven files, we noted that senior audit management had not documented timely reviews at the examination and reporting phases of the audit. These files lack evidence that the high‑risk sections of the files were reviewed. Practitioner review is a critical step in the audit process. It is the foundation of ensuring that observations and conclusions are valid. It has an impact on a number of our audit policies.

23. In two of the seven files, the About the Audit section had not been substantiated and reviewed. In both cases, we recommended that the teams substantiate the section and amend the audit files. In neither case were any errors noted in those sections.

Good practices

24. During our reviews of the audit files, we observed the following good practices:

• One audit team planned a small audit and streamlined the audit procedures in consultation with the Performance Audit Practice Team and with the assistant auditor general and product leader. This practice reduced the administrative burden while still allowing the team to meet OAG policies and audit standards.
• Another team met with Legal Services staff and proposed changes to procedures to assist with how audit teams handle allegations.

Other observations

25. We saw no documented Differences of Opinion in any of the seven files reviewed. Based on CICA 5030.40, OAG audit policy 3082 instructs practitioners to “ensure that the nature and scope of the difference of opinion and resulting conclusions are documented and implemented.” The practice review team notes that in the annual audit product line, differences of opinion occur and are documented as required. We find it unusual that no differences of opinion arose in the seven direct report files we reviewed.

26. In two audits we reviewed, we observed that multiple PX and DM drafts were issued. While we understand that the potential causes for issuing multiple drafts are numerous, we are concerned that the draft chapters may have been insufficiently advanced or reviewed prior to distribution.

Conclusion

27. For each of the seven direct report audits that we reviewed, the auditor’s report was supported and appropriate.

28. While the level of compliance with Office policies and professional audit standards is high, we observed some opportunities for improvement and two instances of non-compliance with requirements for documenting the nature, extent, and timeliness of the practitioner’s review.

29. We are making no recommendations to the audit practice or to the Professional Practices Group, as the nature and extent of our observations do not suggest systemic or pervasive issues.

Management’s response. Management thanks the Practice Review and Internal Audit team for its report and agrees that the observations do not reflect systemic or pervasive practice-wide issues. As a result, management will discuss the implications of the report with those affected.

Appendix A—System of Quality Control Elements

[Appendix A—text version]

Appendix B—System of Quality Control Elements and Process Controls Reviewed

Our review covers the following System of Quality Control (SoQC) elements:

Engagement performance. We reviewed whether the audit was planned, executed, and reported in accordance with Canadian generally accepted auditing standards, applicable legislation, and Office policies and procedures. We considered whether the Office meets its reporting responsibilities by having in place appropriate audit methodology, recommended procedures, and practice aids that support efficient audit approaches, producing sufficient audit evidence at the appropriate time.

As part of the conduct of the audit, we also reviewed audit file finalization. We determined whether audit files were closed within 60 days of the auditor’s report being given final clearance by the signatory and the financial statements being approved by the Board of Directors of the entity, or its equivalent, as required by Office policy.

We reviewed whether consultation was sought from authoritative sources and specialists with appropriate competence, judgment, and authority to ensure that due care was taken, particularly when dealing with complex, unusual, or unfamiliar issues. We also reviewed whether the consultations were adequately documented, and whether the audit team took appropriate and timely action in response to the advice received from the specialists and other parties consulted.

We reviewed whether the quality reviewer carried out, in a timely manner, an objective evaluation of

• the significant judgments made by the team,
• the conclusions reached in supporting the auditor’s report, and
• other significant matters that have come to the attention of the quality reviewer during his or her review.

We reviewed whether the work of the quality reviewer was adequately documented, and whether the audit team took appropriate and timely action in response to the advice received from the quality reviewer.

Human resources. We reviewed whether the adequacy, availability, proficiency, competence, and resources of the audit team were appropriately assessed and documented.

Ethics and independence. We reviewed whether the independence of all individuals performing audit work, including specialists, had been properly assessed and documented.

Leadership. We reviewed evidence of whether individuals working on the audit received an appropriate level of leadership and direction and whether adequate supervision of all individuals, including specialists, was provided to ensure that audits were carried out properly.