Report on a Review of the Direct Report Audit Practice—Direct Report Audits Completed in the 2014–15 Fiscal Year

Report on a Review of the Direct Report Audit Practice—Direct Report Audits Completed in the 2014–15 Fiscal Year

Introduction

1. The Office of the Auditor General of Canada (OAG, or the Office) conducts independent audits and studies that provide objective information, advice, and assurance to Parliament, territorial legislatures, boards of Crown corporations, government, and Canadians. The Office has three main product lines: annual audits, performance audits, and special examinations. Performance audits and special examinations are referred to as direct report engagements.

2. A performance audit is an independent, objective, and systematic assessment of how well government is managing its activities, responsibilities, and resources. Performance audits contribute to a public service that is effective and a government that is accountable to Parliament and Canadians. Performance audits are planned, performed, and reported in accordance with professional auditing standards and Office policies.

3. Special examinations are a form of performance audit that is conducted within Crown corporations. The Office audits most, but not all, Crown corporations. The scope of special examinations is set out in the Financial Administration Act. A special examination considers whether a Crown corporation’s systems and practices provide reasonable assurance that its assets are safeguarded, its resources are managed economically and efficiently, and its operations are carried out effectively.

4. The Practice Review and Internal Audit team conducted practice reviews of selected direct report audits. This practice review work was done in accordance with the monitoring section of the Canadian Standard on Quality Control 1—Quality Control for Firms that Perform Audits and Reviews of Financial Statements, and Other Assurance Engagements, issued by the Chartered Professional Accountants of Canada. We also performed our work in accordance with the Office’s most recent Practice Review and Internal Audit Plan, which was recommended by the Audit Committee and approved by the Auditor General. The Plan is based on systematic, cyclical monitoring of the work of all engagement leaders in the Office.

5. To ensure that audits meet the standards of the Chartered Professional Accountants of Canada, the Office establishes policies and procedures for its work. These are outlined in the Office’s audit manual, its System of Quality Control, and various other audit tools, which guide auditors through a set of required steps. There are three assistant auditors general responsible for the direct report product line. They provide leadership and oversight for the product line and contribute to the quality of the individual audits.

6. This report summarizes the key observations related to the practice reviews of the selected direct report audits completed in the 2014–15 fiscal year.

Overview

Objective

7. The objective of practice reviews is to provide the Auditor General with assurance that

• direct report audits comply with professional standards, Office policies, and applicable legislative and regulatory requirements; and
• audit reports are supported and appropriate.

Scope and methodology

8. We planned and conducted eight practice reviews of direct report audits that were completed in the 2014–15 fiscal year. We used random sampling to select the engagement leaders and their related files.

9. Our reviews included an examination of electronic (TeamMate) and paper audit files. We reviewed documentation related to the planning, examination, and reporting of the audits. We also interviewed selected audit team members, engagement quality control reviewers, and other internal specialists, as appropriate.

10. We reviewed all files selected according to the System of Quality Control (Appendix A), focusing our work on the selected elements and key process controls that we considered key or high risk (Appendix B).

Rating

11. For each direct report audit under review, we rated each selected System of Quality Control element and process control as one of the following:

• Compliant. Office policy requirements and applicable auditing standards were met.
• Compliant but needs improvement. Improvements are necessary in some areas to fully comply with Office policies and professional auditing standards.
• Non-compliant. Major deficiencies exist; there is non-compliance with Office policies or professional auditing standards.

12. After completing each practice review, we concluded whether the audit opinion was supported and appropriate.

Results of the Reviews

Appropriateness of the audit reports

13. Overall, we found that the audit reports were supported and appropriate in the eight files reviewed.

Compliance with the System of Quality Control and process controls

14. Generally, the level of compliance with the System of Quality Control elements was good. All eight files complied in all material respects with the Office’s performance audit or special examination audit policies, and professional standards. We found areas that needed improvement in at least one System of Quality Control element in all files. Also, we noted that one file was non-compliant with one System of Quality Control element and another was non-compliant with a specific Office policy requirement.

15. There are no observations related to the work of the engagement quality control reviewer. Six of eight files selected for review had an engagement quality control reviewer assigned to them.

Observations

16. Except as specifically noted, the following observations were assessed by the Practice Review and Internal Audit team as “Compliant but needs improvement.” The observations are listed in order of their frequency. We noted one systemic item, which is described in the following paragraph.

17. In six of eight files, we found that Independence Confirmation forms were prepared and dated based on the date that the auditor joined the team, as opposed to the period covered by the audit. This puts the audit at risk for an independence issue between the start of the period covered by the audit and the time that the auditor joined the audit team.

18. In two files, practice review found that several Office specialists charged time to the audit but had no Independence Confirmation form completed. This puts the audit at risk for an independence issue.

19. In two files, we noted that the audit reports were not properly dated. The audit report should be dated after all high-risk areas have been reviewed by the engagement leader and the engagement quality control reviewer. As well, if the entity provides additional evidence, it ought to have been received, reviewed, and reflected in the updated report. Also, in one of these reports, the period covered by the audit was not properly reported. The period covered by the audit reflects the start and end dates of the scope of the work to be completed. The other file did not identify the start of the period covered by the audit in key planning documents, but this date was included in the final audit report.

20. In all eight of the audits we reviewed, we found that the electronic files had been archived within the 60-day deadline. However, in two files, we found that the paper files were not archived until significantly later. In one of these cases, it appears that the paper file was not complete. In one other file, we noted that the paper audit file was not archived at all. This element was rated as non-compliant according to the Office’s System of Quality Control for this audit.

21. In two files, we found that controlled documents were not returned to the Office subsequent to report tabling. Although the teams did document the situation, they did not follow proper procedure to follow up on these documents. In another case, we found that a team maintained a controlled document register but did not include it in the audit file. Although all of the controlled documents were returned in this instance, there was no documentation in the file indicating that this was the case.

22. An audit report conclusion was reworded after the deputy minister’s (DM) draft was issued. We found no documentation in the audit file about the significant professional judgment made to reach the conclusion.

23. During the course of the practice review, it became apparent that a team had relied on secondary evidence but did not identify it as such during the planning stage. If they use secondary evidence, teams should demonstrate its relevance, reliability, and validity.

24. In the above-noted audit, the engagement leader exercised significant professional judgment regarding the use of secondary evidence. We found no documentation within the file demonstrating why and how the practitioner obtained assurance. This observation was assessed as being non-compliant with Office policy.

25. In one electronic file, we found that many items were included in the audit file that either were not necessary or were blank templates. As well, in the paper files, we noted items that had no evidence of sign-off or review.

Good practices

26. We observed three examples of good practices that other teams in the Office should be aware of. In all cases, items were documented in the audit file in a pragmatic manner that supported the audit approach and was efficient in doing so, not burdening the team with documentation requirements.

27. In one case, we noted that prior to issuing the principal’s (PX) draft, the team had decided not to complete a small segment of originally planned audit work. OAG Audit Policy 7021—Evaluate the sufficiency and appropriateness of audit evidence addresses the issue when an engagement leader cannot complete the work originally planned due to an inability to obtain sufficient appropriate audit evidence. In this case, the team found they did not have enough time and resources to do the work; based on their updated awareness of the situation, they concluded that this activity might be better suited to follow-up audit work in the future. This was very well documented in a succinct and clear note to file.

28. In another case, the team consulted the Office Internal Specialist for Internal Audit, who provided them with a template to document their ability to rely on internal audit. This document is well designed and assisted the team in identifying whether and how they could rely on Internal Audit. In the end, the team decided not to rely, but this approach assisted the team in clearly completing the task and also ensured that they met Chartered Professional Accountants of Canada (CPA) General Assurance and Auditing, Section 5050—Using the Work of Internal Audit in Assurance Engagements Other than Audits of Financial Statements and Other Historical Financial Information.

29. In a third case, the individual in the engagement leader position changed during the audit. A note was included in the audit file explaining the role of the assistant auditor general and the new engagement leader.

Opportunities for efficiency

30. Practice review noted a situation that audit teams should consider in an effort to make their audits more efficient.

31. In one audit, we found too much documentation for the substantiation of several paragraphs. The goal of substantiation is not to provide every item of evidence in the audit file to prove a point. The purpose is to provide the strongest evidence so a reviewer has confidence that there is sufficient appropriate evidence. OAG Audit Policy 7060—Substantiating the chapter provides guidance that teams should revisit. Auditors preparing evidence for substantiation should be taught to “think like a reviewer.”

Recommendations to the Professional Practices Group

32. Observation. Audit teams do not appear to understand that the date in section H of the Independence Confirmation form relates to the beginning of the period covered by the audit. Although the policy is clear, six of the eight files we reviewed had this date wrong. According to the dates included, engagement team members had not confirmed their independence for the full period covered by the audit. This may have resulted in team members with conflicts of interest that were not identified and remediated.

33. Recommendation 1. The Direct Engagement Practice Team should remind engagement leaders of the technical requirements when completing the Independence Confirmation forms, through principal forums or other similar venues.

Management’s response. Agreed. This issue was discussed at the PX/DX Discussion Group on 26 May 2015 and at the PA Forum on 16 June 2015. Staff were reminded how to properly reflect the period covered by the audit in the Independence Confirmation form.

34. Observation. Presently, there is no mechanism in place that demonstrates the date of receipt when paper audit files are brought to Information and Records Management for archiving. The date recorded is the date the file is entered into the archival system. Also, the current policy refers to the need to finalize a file. Practice review is unsure of whether a file is finalized when it is delivered to Information and Records Management or when it is physically archived. Note: This observation applies to direct report and attest audits.

35. Recommendation 2. The Professional Practices Group (PPG) and Information and Records Management should develop a procedure that accurately tracks the date a file is delivered for archiving. Also, the PPG should consider revising the policy to clarify whether the 60-day count culminates with the delivery or archiving of the audit file.

Management’s response. Agreed. The Professional Practices Group (PPG) will work with Information and Records Management to ensure that the date a paper file is received by Information and Records Management is consistently recorded. PPG will further ensure that guidance on finalizing paper audit files is clear concerning when the 60-day count culminates.

36. Observation. There is a requirement for direct report audits to disclose the sources of criteria during planning and reporting. This is Office policy that is based on Chartered Professional Accountants of Canada (CPA) General Assurance and Auditing, Section 5025.66—Standards for Assurance Engagements Other than Audits of Financial Statements and Other Historical Financial Information—Reporting Standards. While the policy and standards require the sources of criteria, the Office’s special examination templates do not include a space for them. The Office’s performance audit templates do include such a space.

37. Recommendation 3. The Direct Engagement Practice Team (DEPT) should update the relevant templates so the sources of criteria are included. The DEPT should ensure that engagement leaders are aware of the requirement.

Management’s response. Agreed. At the present time, the recommended criteria for special examinations date back to 2006 and are not sourced. The Direct Engagement Practice Team is currently updating, in collaboration with Internal Specialists, the recommended criteria to be used in special examinations and their sources. This new information will be part of the 2015 Annual Methodology Update scheduled for November 2015. As part of the update, the practice team will update the reporting template for special examinations.

38. Observation. OAG Audit Policy 3062—Engagement leader responsibilities for audit quality states that “the responsibilities of the practitioner shall be discharged by the audit principal, who is the engagement leader for all engagements . . .” The policy also specifies that “in those circumstances where an audit principal is not assigned to the engagement, the assistant auditor general shall assume the role of engagement leader.”

39. It has come to our attention that the Office has audits that are or were being managed by a director who was in the role of an engagement leader. In the audit file we reviewed, it was well documented how the assistant auditor general and the director would cover the role of engagement leader. However, this practice is not consistent with Office policy for direct report and annual audits.

40. Recommendation 4. The Professional Practices Group should update OAG Audit Policy 3062—Engagement leader responsibilities for audit quality to make it possible for the Auditor General to name a staff member other than a principal as the engagement leader.

Management’s response. Agreed. Management will review and revise, as appropriate, the wording of the above-noted policy in response to the recommendation.

Conclusion

41. We concluded that each of the eight direct report audits we reviewed were compliant overall with Office policies and professional audit standards, and that the audit opinions were supported.

42. While the level of compliance with Office policies and professional audit standards is good, we observed that improvement is needed in some areas. We made four recommendations to the Professional Practices Group.

Appendix A—System of Quality Control Elements

Appendix B—System of Quality Control Elements and Process Controls Reviewed

Our review covers the following System of Quality Control elements:

• engagement performance,
• resources,
• independence, and
• leadership and supervision.

Engagement performance. We reviewed whether the audit was planned, executed, and reported in accordance with generally accepted Canadian auditing standards, applicable legislation, and Office policies and procedures. We also considered whether the Office met its reporting responsibilities by having in place appropriate audit methodology, recommended procedures, and practice aids to support efficient audit approaches and produce sufficient audit evidence at the appropriate time.

As part of the conduct of the audit, we also reviewed audit file finalization. We determined whether audit files were closed within 60 calendar days of issuing the final assurance engagement report (for special examinations) or within 60 days of the date of tabling (for performance audits).

We reviewed whether consultation was sought from authoritative sources and specialists with appropriate competence, judgment, and authority to ensure that due care was taken, particularly when dealing with complex, unusual, or unfamiliar issues. We also reviewed whether the consultations were adequately documented, and whether the audit team took appropriate and timely action in response to the advice received from the specialists and other parties consulted.

We reviewed whether the quality reviewer carried out, in a timely manner, an objective evaluation of

• the significant judgments made by the team,
• the conclusions reached in supporting the auditor’s report, and
• other significant matters that came to the attention of the quality reviewer during his or her review.

We reviewed whether the work of the quality reviewer was adequately documented, and whether the audit team took appropriate and timely action in response to the advice received from the quality reviewer.

Resources. We reviewed whether the adequacy, availability, proficiency, competence, and resources of the audit team were appropriately assessed and documented.

Independence. We reviewed whether the independence of all individuals performing audit work, including specialists, had been properly assessed and documented.

Leadership and supervision. We reviewed evidence of whether individuals working on the audit received an appropriate level of leadership and direction and whether adequate supervision of all individuals, including specialists, was provided to ensure that audits were carried out properly.