Report on a Review of the Financial Audit Practice—Financial Audits Completed in the 2017–18 Fiscal Year

Report on a Review of the Financial Audit Practice—Financial Audits Completed in the 2017–18 Fiscal Year

Introduction

1. The Office of the Auditor General of Canada (the Office) conducts independent audits and studies that provide objective information, advice, and assurance to Parliament, territorial legislatures, boards of Crown corporations, government, and Canadians. The Office carries out three main types of legislative audits: financial audits, performance audits, and special examinations. Performance audits and special examinations are referred to as direct engagements.

2. Financial audits include

In addition, the Office conducts audits that address specific needs of provincial and First Nations governments, such as the audit of the Canada Revenue Agency’s collection of income tax amounts on behalf of provincial governments.

3. The audits are conducted in accordance with Canadian Auditing Standards. The objective of financial audits is to provide an opinion on whether financial statements are presented fairly, in all material respects, in accordance with the applicable financial reporting framework. Where required, the auditor also provides an opinion on whether the transactions examined comply with all applicable laws and regulations.

4. The mission of the Practice Review and Internal Audit team is to enhance and protect organizational value by providing risk-based and objective assurance, advice, and insight. The team helps the Office accomplish its objectives by offering management recommendations based on the application of a systematic, disciplined approach to evaluating and approving the design and effectiveness of risk management, control, and governance processes.

5. The team helps the Office meet its obligations under Canadian Standard on Quality Control 1 issued by the Auditing and Assurance Standards Board. It does this by conducting inspections to determine whether engagement leaders (audit leaders) are complying with professional standards, Office policies, and applicable laws and regulations when conducting their audits. It also ensures that independent auditor’s reports are supported and appropriate.

6. The team performs its work in accordance with the Office’s most recent Practice Review and Internal Audit Plan, as recommended by the Audit Committee and approved by the Auditor General. The Plans is based on systematic, cyclical monitoring of the work of all engagement leaders in the Office.

7. To ensure that audits meet the standards of the Chartered Professional Accountants of Canada, the Office establishes policies and procedures for its work. These are outlined in the Office’s Annual Audit Manual, in its System of Quality Control, and in various other audit tools that guide auditors. The two assistant auditors general responsible for financial audits provide leadership and oversight of the Office’s financial audit practice and contribute to the quality of audits.

8. This report summarizes the key observations related to the practice reviews of selected financial audits completed in the 2017–18 fiscal year.

Overview

Objective

9. The objective of practice review is to provide the Auditor General with assurance that

Scope and methodology

10. The Practice Review and Internal Audit team conducted practice reviews of six financial audits and a limited practice review for one audit focusing on quality reviewer involvement in financial audits completed in the 2017–18 fiscal year. Our methodology requires that we review a selection of completed audits on a cyclical basis, including at least one engagement for each engagement leader over a four-year monitoring cycle. We used a random sampling approach to select the engagement leaders and their related files.

11. Our reviews included an examination of electronic (TeamMate) files as well as paper files, if applicable. We reviewed documentation related to the planning, examination, and reporting of the audits. We also met selected audit team members and other internal specialists to discuss issues.

12. We reviewed all files according to the System of Quality Control (Appendix A). We focused our work on the selected elements and process controls that we considered to be key or high risk (Appendix B) in the selected audits.

Rating

13. For each audit we reviewed, we rated the selected Quality Control element and process control as one of the following:

14. After completing each practice review, we concluded whether the independent audit opinion was supported and appropriate. We also concluded whether the audit file was compliant overall with Canadian Auditing Standards and Office policies.

Results of the Reviews

Appropriateness of the audit reports

15. Overall, we found that the independent audit opinions were supported and appropriate in all the files reviewed for which an opinion was issued.

Compliance with the System of Quality Control elements and process controls

16. In general, the overall level of compliance with the System of Quality Control elements was good. Three files complied in all material respects with the Canadian Auditing Standards and the Office’s annual audit policies. The remaining three files were compliant while were improvements needed. For more information, see the Observations section.

17. Our overall conclusion on a file is based on the review of all elements of the System of Quality Control. Consequently, a file can be non-compliant with one element of this system even though the overall conclusion is “compliant while improvements needed.”

18. For the limited review performed on the work of the quality reviewer, we concluded that the Engagement Quality Control Review element of the System of Quality Control was compliant while improvements were needed. For more information, see the Observations section.

Special consideration for this year’s cycle

19. When performing our reviews for this year’s cycle, the Practice Review and Internal Audit team paid special attention to possible gains of efficiency when audit teams performed their work and to good practices that may be beneficial for other teams. The team also looked at ways to gain efficiencies in its own processes. For example, we looked at ways to be more efficient in the review we performed this year on the independence process.

20. In last year’s summary report, the team recommended that Audit Services assess whether changes were required to the independence confirmation process or policy, or both. Audit Services has since made this assessment and decided to modify the Office’s policy to remove the team members’ obligation to confirm their independence before they start working on an audit.

21. The files we reviewed in the current cycle were completed before changes to the independence policy (Office of the Auditor General of CanadaOAG Audit 3031) were considered, approved, and in effect. We expect that an assessment of these files against the standard that was in effect at the time of the audit would yield observations similar to those identified in the last review cycle. Therefore, we have not assessed these files.

Observations

Good practices observed

22. In two files, the Practice Review and Internal Audit team observed good practices related to the documentation of the audit teams’ rationale as to why the significant audit risks identified had been reported or not reported to the Audit Committee. The team also observed that audit teams had documented their rationale as to why some low audit risks had been reported to the Audit Committee although this was not required by the Office’s methodology. We brought these practices to the attention of the Annual Audit Practice Team to consider in their upcoming methodology update.

Opportunities for efficiency

23. In some files, although the audit work was compliant with the Office’s methodology, the team identified opportunities for efficiency.

24. For example, in one file, the audit team had developed detailed procedures, mostly in the planning section, to address specific audit requirements for related services. However, the Office has audit templates which already include most of the detailed procedures developed by the audit team. Keeping this tailored audit program up to date would require extra effort from the audit team. The Practice Review and Internal Audit team believes that audit teams may gain efficiencies by using the Office’s audit templates. Using them would also minimize the risk of having a tailored audit program that is not in line with the current Chartered Professional Accountants of Canada handbook and Office methodology.

25. In another file, the Practice Review and Internal Audit team noted that an audit team had done an analytical review of all the financial statements’ line items that had a lead sheet. Although not required in their audit strategy, the audit team did this analysis to explain variances up to 20% of the Planned Materiality. The audit team informed us that they had done this to prepare for the performance analysis included in the Report Clearance Summary.

26. However, in the Report Clearance Summary, the audit team did this performance analysis at a level of 50% of materiality. This level was too high compared with what was required for the financial statements’ line items, which resulted in unnecessary work.

27. In two files, the Practice Review and Internal Audit team noted that the Office’s audit procedure templates were not filled out efficiently. These templates were developed to reduce the amount of documentation for audit teams. In one file, the audit team had over-documented their responses to the template questions. In another, the audit team had inserted a large number of documents as hyperlinks. Having too many details in templates means that it takes a lot of time to document and review. Also, having too many hyperlinks can lead to issues with the functionality of the hyperlinks.

Security of sensitive information

28. According to OAG Security Policy, regardless of the storage location (TeamMate or PROxI), all protected A or B audit working papers that the OAG originates must be designated as such. Consequently, audit team members must perform an assessment to determine the proper security labelling of any team-generated documentation. The Office Security Quick-Reference Card clearly indicates that if a document has to be categorized as “Protected,” this information must be placed on the cover page or on each page of the document, depending on the sensitivity of the information.

29. Even though documents are stored in an appropriate and secure container (TeamMate), there is still a risk that these unmarked documents could become vulnerable if removed from their secure environment by being printed.

30. The Practice Review and Internal Audit team noted that in five of the files it reviewed, the audit team had placed the wording “Protected A” or “Protected B” in the file name. Having this information in the file name may be a good practice, but it must be supported by having the classification written in the document. For two of these five files, the audit team had followed this good practice.

31. For three of these five files, the labelling in the file name was often not the same as the classification in the document. For example, we noted cases where “Protected B” was included in the file name, while no mention of a protected document was placed inside the document. Having mentioned “Protected” in the file name adds no value if not supported by the same information in the document.

32. We were informed that a number of these documents were “unclassified.” In our view, the reference to “Protected A” or “Protected B” in the file name led to misinterpretation and was misleading. Because there was no security classification information within the documents, we did not know whether the document owner was attempting to categorize sensitive information by adding “Protected A” or “Protected B” to the file name.

33. The Practice Review and Internal Audit team also noted that audit teams had added the word “Protected” to the file names of documents provided by the entity. Assessing whether a document should be categorized as “Protected” is the responsibility of the document owner. Audit staff should not make this assessment in place of the entity. However, if the audit staff members believe that a document was not properly categorized, they may wish to discuss the security classification with the entity.

34. We believe this is a systemic matter that requires prompt corrective action and/or changes in the Office’s procedures. The related recommendations are as follows.

35. Recommendation 1 to the Departmental Security Officer. The Departmental Security Officer should continue to provide audit staff with mandatory security information sessions and/or e-learning courses with specific examples adapted to the reality of audit work.

Management’s response. Agreed. The security team developed an e-learning course on the labelling of information. This course was launched in fall 2017 and is mandatory every three years. The security team also gives mandatory information sessions to every group every three years. In future, courses and information will be tailored, as needed, to adapt to the reality of audit work.

36. Recommendation 2 to the Financial Audit Practice. Engagement leaders should ensure that they and their audit staff have a good understanding of the Office’s security directives and that any working papers prepared by the audit team and stored in TeamMate are assessed against these directives and labelled according to the proper security level.

Management’s response. Agreed. Engagement leaders will review the Office’s security directives and discuss requirements for security labelling during audit kick-off meetings. When there is an auditor-prepared working paper, the application of appropriate security labelling will be verified by the reviewer of that working paper.

Independence Confirmation form

37. As previously mentioned (see paragraph 20), the Practice Review and Internal Audit team did not assess, in this cycle, whether Independence Confirmation forms had been completed and approved before team members started working on an audit. However, we did assess the completeness and disclosure of the Independence Confirmation forms.

38. All the forms the Practice Review and Internal Audit team reviewed were properly documented. The period covered by the Independence Confirmation forms were all in line with the period in the audits under review. However, in one file we found that two forms were missing for individuals who met the definition of an engagement team member. In this case, we asked the engagement leader to reopen the audit file to ensure that independence was assessed and documented for those two individuals. We also asked the engagement leader to inform the Principal of Practice Review and Internal Audit if any conflicts were identified.

39. For the file that had two missing Independence Confirmation forms, the Practice Review and Internal Audit team concluded that the file was not compliant with the Ethics and Independence element of the System of Quality Control.

Supervision and review

40. In one file, the Practice Review and Internal Audit team noted that the audit team did not use the version of the Summary of Uncorrected Misstatements (SUM) template currently available in the Office’s financial audit site of the INTRANnet and in TeamMate, thereby risking non-compliance with the current Office methodology. Furthermore, there was an error with one number reported as a credit instead of a debit. In our view, it is important that the SUM be properly completed because errors may lead to an incorrect audit conclusion.

41. Because the error did not affect the audit results, the Practice Review and Internal Audit team concluded that for this file the Supervision and Review element of the System of Quality Control was compliant, while improvements were needed.

42. However, when reviewing the Summary of Uncorrected Misstatements template, we noted that functionalities in the template did not address all the requirements of the Office’s methodology. In our view, this may bring some risk to an audit if an audit team relies only on the template functionalities.

43. Section 9015 of the Annual Audit Manual, Evaluating the effect of uncorrected misstatements, indicates, “If the Probable Overall Misstatement is greater than 75% of overall materiality, the engagement team must consult with Annual Audit Practice Team.” It also outlines that the audit team should do the following:

Use the SUM to assist in determining if the financial statements are free of material misstatement by:

44. We noted that the template currently calculates only the evaluation of uncorrected misstatements against equity. As a result, the audit team may not be compliant with the requirements of the Office’s methodology if they rely only on the automated calculation made in the Summary of Uncorrected Misstatements template.

45. Recommendation 3 to the Audit Services. Audit Services should assess whether changes are required to the Summary of Uncorrected Misstatements template or related procedures to help the audit team evaluate the effect of uncorrected misstatements, according to the Office’s methodology.

Management’s response. Agreed. Audit Services will assess whether changes to the Summary of Uncorrected Misstatements template could further assist audit teams in complying with Office policy and applying the guidance expressed in OAG Audit 9015 for evaluating the effect of uncorrected misstatements.

Engagement documentation

46. In one file, the Practice Review and Internal Audit team noted that many documents were included in an audit section with no support to the audit step requirements. All documents included in an audit file should be linked to an audit procedure. For that file, we concluded that for the Engagement Documentation element of the System of Quality Control, the audit was compliant while improvements were needed.

Engagement Quality Control Review

47. As previously mentioned (see paragraph 10), the Practice Review and Internal Audit team had to perform an additional review of an audit file to ensure that at least one file under review in the current cycle had a quality reviewer assigned to it. In doing so, we ensured that we could assess the Engagement Quality Control Review element of the System of Quality Control. As mentioned, for this file, we focused only on reviewing the quality reviewer’s work and the audit team’s interaction with that person.

48. The Practice Review and Internal Audit team noted that for the execution and reporting phase of the audit, the evidence of the quality reviewer’s involvement in the audit was very limited in the TeamMate file. We also noted that the quality reviewer’s final approval was not done in a timely manner. We concluded that for the Engagement Quality Control Review element of the System of Quality Control, the file was compliant while improvements were needed.

Conclusion

49. Some of the financial audits the Practice Review and Internal Audit team reviewed required that an independent auditor’s report be issued. In these cases, we concluded that the reports were supported and appropriate.

50. The Practice Review and Internal Audit team concluded that overall, three files were compliant and three files were compliant, while improvements were needed.

51. For the limited practice review for one audit focusing on quality reviewer involvement, the Practice Review and Internal Audit team concluded that for the Engagement Quality Control Review element of the System of Quality Control, the file was compliant while improvements were needed.

Appendix A—System of Quality Control Elements

Diagram showing objectives, levels, and elements of the System of Quality Control
Text version

This diagram shows three sides of a cube, each side depicting aspects of the System of Quality Control.

The top of the cube shows the objectives of the System of Quality Control:

  1. Compliance with professional standards and applicable legal and regulatory requirements; and
  2. Reports issued are appropriate in the circumstances.

The right side of the cube shows the two levels of the System Quality Control:

  • Firm level (Canadian Standards for Quality ControlCSQC 1)
  • Engagement level (Canadian Auditing StandardCAS 220 or Canadian Standard for Assurance EngagementsCSAE 3001)

The front of the cube shows the elements of the System of Quality Control:

  • leadership,
  • ethics and independence,
  • acceptance and continuance,
  • human resources,
  • engagement performance, and
  • monitoring

Appendix B—System of Quality Control Elements and Process Controls Reviewed

Our review covers the following System of Quality Control elements:

Leadership. We reviewed whether the engagement leaders ensured that the audits were carried out in compliance with Office policies, professional standards, the System of Quality Control, and applicable laws and regulations.

Ethics and independence. We reviewed whether the engagement leaders ensured that the independence of all individuals performing audit work, including specialists, had been properly assessed and documented.

Acceptance and continuance. For initial or recurring engagements, we reviewed whether engagement leaders assessed whether the team had the necessary competence, capability, time, and resources; whether the team complied with relevant ethical requirements; and whether the team considered management’s integrity.

Human resources. We reviewed whether the engagement leaders assessed the audit team’s adequacy, availability, proficiency, competence, and resources, and whether they documented their assessments.

Engagement performance

Within the engagement performance element, we also assessed the following:

Other Canadian Auditing Standards requirements and Office policies

We reviewed whether engagement leaders ensured that the audit was planned, executed, and reported in accordance with Canadian Auditing Standards, applicable legislation, and Office policies and procedures.

We also considered whether the Office met its reporting responsibilities by having in place appropriate audit methodology, recommended procedures, and practice aids to support efficient audit approaches and to produce sufficient audit evidence at the appropriate time.