Report on a Review of the Financial Audit Practice—Financial Audits Completed in the 2015–16 Fiscal Year

Report on a Review of the Financial Audit Practice—Financial Audits Completed in the 2015–16 Fiscal Year

Introduction

1. The Office of the Auditor General of Canada (the Office) conducts independent audits and studies that provide objective information, advice, and assurance to Parliament, territorial legislatures, boards of crown corporations, government, and Canadians. The Office carries out three main types of legislative audits: financial audits, performance audits, and special examinations. Performance audits and special examinations are referred to as direct report engagements.

2. Financial audits include audits of the financial statements of the Government of Canada, the three northern territories, Crown corporations, and other organizations. They are performed in accordance with Canadian Auditing Standards. The objective of financial audits is to provide an opinion on whether financial statements are presented fairly, in all material respects, in accordance with the applicable financial reporting framework. Where required, the auditor also provides an opinion on whether the transactions examined comply with all applicable laws and regulations.

3. The mission of the Practice Review and Internal Audit team is to enhance and protect organizational value by providing risk-based and objective assurance, advice, and insight. The team helps the Office accomplish its objectives by offering management recommendations based on the application of a systematic, disciplined approach to evaluating and approving the design and effectiveness of risk management, control, and governance processes.

4. The team helps the Office meet its obligations under the Chartered Professional Accountants of Canada Canadian Standard of Quality Control 1 by conducting inspections to determine the extent to which engagement leaders are complying with professional standards, Office policies, and applicable legislative and regulatory requirements when conducting their audits, and to ensure that independent auditors’ reports are supported and appropriate.

5. The team also performs its work in accordance with the Office’s most recent Practice Review and Internal Audit Plan, as recommended by the Audit Committee and approved by the Auditor General. The Plan is based on systematic, cyclical monitoring of the work of all engagement leaders in the Office.

6. To ensure that audits meet the standards of Chartered Professional Accountants of Canada, the Office establishes policies and procedures for its work. These are outlined in the Office’s annual audit manual, in its System of Quality Control, and in various other audit tools that guide auditors through the required steps. The four assistant auditors general responsible for financial audits provide leadership and oversight of the financial audit practice in the Office and contribute to the quality of individual audits.

7. This report summarizes the key observations related to the practice reviews of selected financial audits completed in the 2015–16 fiscal year.

Overview

Objective

8. The objective of practice review is to provide the Auditor General with assurance that

• financial audits comply with professional standards, Office policies, and applicable legislative and regulatory requirements; and
• independent auditors’ reports are supported and appropriate.

Scope and methodology

9. The Practice Review and Internal Audit team conducted practice reviews of four financial audits and limited practice reviews for two audits focusing specifically on quality reviewer involvement in financial audits completed in the 2015–16 fiscal year.^{Footnote 1} Our methodology requires that we review a selection of completed audits on a cyclical basis, including at least one engagement for each engagement leader over a four-year monitoring cycle. We used a random sampling approach to select the engagement leaders and their related files. With respect to the two practice reviews of the quality reviewer, we also used a random sampling approach to select the audit files.

10. Our reviews included an examination of electronic (TeamMate) files as well as paper files, if applicable. We reviewed documentation related to the planning, examination, and reporting of the audits. We also interviewed quality reviewers, selected audit team members, and other internal specialists, as appropriate.

11. We reviewed all files selected in terms of the System of Quality Control (Appendix A). We focused our work on the selected elements and process controls that we considered to be key or high risk (Appendix B) in the selected audits.

Rating

12. For each audit reviewed, we rated each selected System of Quality Control element and process control as one of the following:

• Compliant. Performance is satisfactory, with minor improvement possible; the audit file is in compliance with Canadian Auditing Standards (CAS) and Office policies in all significant respects.
• Compliant and improvement needed. Improvements are necessary in some areas to fully comply with Canadian Auditing Standards and Office policies.
• Non-compliant. Major deficiencies exist; the audit does not comply with Canadian Auditing Standards and/or Office policies.

13. After completing each practice review, we concluded whether the independent audit opinion was supported and appropriate. We also concluded whether the audit file was compliant overall with Canadian Auditing Standards and with Office policies.

Results of the Reviews

Appropriateness of the audit reports

14. Overall, we found that the independent audit opinions were supported and appropriate in the four files reviewed.

Compliance with the System of Quality Control elements and process controls

15. In general, the overall level of compliance with the System of Quality Control elements was good. One file complied in all material respects with the Office’s annual audit policies and Canadian Auditing Standards. The remaining three files were compliant and improvement needed. Please refer to the Observations section for details.

16. It is important to note that our overall conclusion on a specific file is based on the review of all elements of the System of Quality Control. Consequently, it is possible to be non-compliant with one element of the System of Quality Control even though the overall conclusion is compliant and improvement needed.

17. For the two limited reviews performed on the work of the quality reviewer, we concluded that the Engagement Quality Control Review element of the System of Quality Control was compliant. The work performed was completed in a timely manner and was well documented. Also, the audit teams dealt with the quality reviewers’ comments appropriately and to the quality reviewers’ satisfaction.

Observations

Security of sensitive information

18. For the current practice review cycle (for both financial audits and direct engagements), we have assessed security of sensitive information as a risk worthy of special attention. The OAG Security Policy states that “The Office is responsible for safeguarding the information and assets that it controls, including sensitive information that it creates and receives.”

19. According to OAG security policy, regardless of storage location (TeamMate or PROxI), all protected audit working papers (that the OAG originates) must be designated as such.

20. In performing our reviews, we found working papers in three audit files that were not designated as protected. We believe these working papers should have been designated as a minimum as “Protected A.” For some of these working papers, the source of information used by the auditors to prepare their audit working papers had been designated as “Protected.” This supports our observation that the working papers containing the same protected information should have been designated as such. Our review did not identify any documents that we believe should have been labelled higher than “Protected B.”

21. Even though the documents were stored in an appropriate and secure container (TeamMate), there is still a risk that these unmarked documents could become vulnerable if removed from their secure environment by being printed or emailed to other users.

22. We concluded that the Engagement Documentation element of the System of Quality Control was non-compliant in these three files. We believe this is a systemic matter that requires prompt corrective action and/or changes in the Office’s procedures. The related recommendations are as follows.

23. Recommendation 1 to the Financial Audit Practice. Engagement leaders should ensure that audit staff are aware of the Office’s security policy, and that any document stored in TeamMate be assessed against the policy and be labelled according to the proper security level.

Management’s response. Agreed. Engagement leaders will communicate the Office’s security policy and labelling requirements for audit documentation at an upcoming meeting of the annual audit engagement leaders and directors. Further, team audit planning meeting agendas will be updated to include a discussion of security labelling requirements, effective immediately.

24. Recommendation 2 to the Annual Audit Practice Team. The Annual Audit Practice Team should make the required changes to Office methodology to assist auditors in assessing the documentation against the Office’s security policy and label information according to the proper security level.

Management’s response. Agreed. The Annual Audit Practice Team, in cooperation with IT Services, will assess the most efficient and effective way to assist auditors in labelling audit documentation with an appropriate security label and deploy as appropriate agreed changes in future methodology or software updates.

25. Recommendation 3 to the Departmental Security Officer. The departmental security officer should develop mandatory security information sessions and/or e-learning courses with specific examples adapted to the reality of audit work and with particular attention to audit working papers.

Management’s response. Agreed. The departmental security officer will work with the Professional Development team and other stakeholders to determine

• the right solution (for example, awareness sessions, training one-on-one, e-learning sessions, or other) to help individuals assess and label information according to the proper security level; and
• an implementation calendar.

Information included in the entity’s annual report

26. We also concluded that one audit file was non-compliant with the supervision and review element of the System of Quality Control. When performing a review of other information included in the entity’s draft annual report, the audit team did not notice that the independent auditor’s report reproduced in the report was not the same as the original signed by the signatory. We looked at the entity’s website and noticed that the published independent auditor’s report was also not the proper one. The engagement leader took prompt action, and the entity corrected the situation.

27. With the introduction of Smart Documentation, the specific audit step to remind audit teams to perform work to ensure that the signed independent auditor’s report has been properly reproduced in the entity’s annual report has been removed. We consider this incident to be an isolated case. However, to avoid a similar situation in future, we are making a recommendation.

28. Recommendation 4 to the Annual Audit Practice Team. The Annual Audit Practice Team should reinstate the procedure in the TeamMate library to remind audit teams to ensure that the independent auditor’s report, as well as the audited financial statements included in the entity’s annual report, have been accurately reproduced.

Management’s response. Agreed. The Annual Audit Practice Team anticipates revising the relevant audit procedures in conjunction with the issuance of updated procedures received from our strategic alliance partner.

Annual Audit Manual, Section 2103: Performance Materiality

29. In Section 2103 (Performance Materiality), the OAG Audit Manual states: “If the haircut chosen is 50 percent of overall materiality OR the aggregate of the haircut and expected errors in the context of substantive tests of details is greater than 50 percent of overall materiality, the Engagement Leader is required to consult with Annual Audit Practice Team before proceeding with the proposed audit approach.”

30. During our review, we noticed that audit teams had not properly documented and/or calculated the percentage outlined in the policy stated above. We were informed by the Annual Audit Practice Team that readers had not properly interpreted the policy’s requirement. The practice team clarified that it is the expected errors for the planned substantive test of details that must be added to the haircut, not the sum of all expected errors from all substantive tests of details planned throughout the audit files.

31. We noted that in two of the files we reviewed, the calculation was neither performed nor documented according to the practice team’s interpretation of the policy. However, as part of our file review, we performed the calculation and concluded that the two files were in compliance. We consider this issue to be systematic, and believe the related Office policy needs clarification.

32. Recommendation 5 to the Annual Audit Practice Team. The Annual Audit Practice Team should clarify the wording of the referenced policy to facilitate a consistent interpretation, and should consider the use of technology to facilitate the calculation by audit teams.

Management’s response. Agreed. The Annual Audit Practice Team will ensure the above referenced policy is clarified or withdrawn as part of the fall 2016 methodology update. We will assess whether auditor compliance with the clarified policy would be further aided by procedure or template changes and if we conclude that to be the case, we will modify procedures or templates to assist auditors in complying with the clarified policy.

Independence confirmation

33. For the element Ethics and Independence, we found that two files were compliant, with improvement needed. In one file, we noted that a key internal specialist had not completed the required independence confirmation for the work performed on the review of key audit documents. In another file, the independence confirmation was signed with a generic sign-off. As such, we were not able to determine whether the required person had signed the declaration. In our view, these were isolated incidents.

Signing off on the planning phase

34. Office procedures require that at the end of the planning phase, the engagement leader should-sign off on the Audit Planning Template as well as the audit step, “Engagement Leader review and sign-off—Planning,” to demonstrate evidence of approval of the audit strategy and engagement planning. In one of the files, sign-off of the template and audit step was only performed after the year-end field work was completed.

35. We have concluded in this case that the file was compliant, with improvements needed. The rationale to support our conclusion is that we were able to see evidence of the engagement leader’s involvement in the planning of the audit despite the official sign-offs not being done in a timely manner. We consider this to be an isolated incident.

Consultation

36. In another case, an audit team consulted the Office’s Legal Services for clarification of an authority matter. In its Report to the Audit Committee—Audit Results, the audit team reported the matter as Legal Services’ conclusion. In our view, it is not appropriate to attribute the conclusion to a specialist in a way that distances the engagement leader from the conclusion.

Considerations for the practice

37. Although we have not observed an issue in the files we reviewed, we observed that the use of the special Smart Documentation developed for Small and Less (S&L) Complex Audits could result in a risk of non-compliance with some Canadian Auditing Standards (CAS) requirements. The OAG Audit Manual includes the following disclaimer: “Engagement Leaders should realize that the S&L library may not provide auditors with all relevant audit considerations. Therefore, when using the S&L library, one assumes that the auditors applying these procedures and related tools understand the CAS requirements and other related explanatory materials and office methodology enough to complete a CAS compliant audit in accordance with office policy.”

38. For example, the notion of unpredictability, while included in the regular Smart Documentation, has been removed in the special Smart Documentation for Small and Less Complex Audits. At this time, we are not able to report on all the differences between the two sets of documents. This is a matter that will require consideration by the Professional Practice Group.

Conclusion

39. For all of the financial audits we reviewed that required issuing an independent auditor’s report, we concluded that the report was supported and appropriate.

40. We concluded that one file was compliant, and three were compliant and improvement needed. For the two limited reviews of quality reviewer involvement, we concluded that both were compliant.

Appendix A—System of Quality Control Elements

Appendix A—text version

This diagram shows three sides of a cube, each side depicting aspects of the System of Quality Control.

The top of the cube shows the objectives of the System of Quality Control:

1. Compliance with professional standards and applicable legal and regulatory requirements; and
2. Reports issued are appropriate in the circumstances.

The right side of the cube shows the two levels of the System Quality Control:

• Firm level (CSQC 1)
• Engagement level (CAS 220 or S5030)

The left side of the cube shows the elements of the System of Quality Control:

• leadership,
• ethics and independence,
• acceptance and continuance,
• human resources,
• engagement performance, and
• monitoring

Appendix B—System of Quality Control Elements and Process Controls Reviewed

Our review covers the following System of Quality Control elements:

• leadership,
• ethics and independence,
• acceptance and continuance,
• human resources, and
• engagement performance.

Leadership. We reviewed whether the engagement leaders ensured that the audits were carried out in compliance with Office policies, professional standards, the System of Quality Control, and applicable legal and regulatory requirements.

Ethics and independence. We reviewed whether the engagement leaders ensured that the independence of all individuals performing audit work, including specialists, had been properly assessed and documented.

Acceptance and continuance. For initial or recurring engagements, we reviewed whether engagement leaders assessed that the team had the necessary competence, capability, time, and resources; that the team complied with relevant ethical requirements; and that it considered management’s integrity.

Human resources. We reviewed whether the engagement leaders assessed the audit team’s adequacy, availability, proficiency, competence, and resources and whether they documented their assessments.

Engagement performance

Within the engagement performance element, we also assessed:

• Supervision and review. We reviewed whether engagement leaders ensured that the audit files had documentation regarding who reviewed the audit work performed, the date, and the extent of the review.
• Consultation. We reviewed whether the engagement leaders ensured that appropriate consultations took place in a timely manner, when required.
• Engagement quality control review. We reviewed whether the quality review was carried out in a timely manner and whether the quality reviewer performed an objective evaluation of the significant judgments made by the team, the conclusions reached in supporting the auditor’s report, and other significant matters.
• Differences of opinion. If differences of opinion occurred, we reviewed whether the engagement leaders followed the Office’s established processes for addressing them.
• Engagement documentation. We reviewed whether engagement leaders properly addressed the confidentiality, safe custody, integrity, accessibility, retrievability, and retention of documentation, and whether the final assembly of the engagement file was completed on a timely basis (that is, the 60-day rule).

Other Canadian Auditing Standards requirements and OAG policies

We reviewed whether engagement leaders ensured that the audit was planned, executed, and reported in accordance with Canadian Auditing Standards, applicable legislation, and Office policies and procedures.

We also considered whether the Office met its reporting responsibilities by having in place appropriate audit methodology, recommended procedures, and practice aids to support efficient audit approaches and to produce sufficient audit evidence at the appropriate time.