Executive Summary

Based on the work performed in the 2015–16 fiscal year monitoring exercise, we concluded that there was reasonable assurance regarding the following

• The policies and procedures for the Office of the Auditor General of Canada’s system of quality control were relevant and adequate. We did not identify any deficiencies.
• The system of quality control operated effectively at the Office level. We identified three Category 3 (isolated) deficiencies that did not prevent the system of quality control from operating effectively. We did not identify any Category 1 (serious) or Category 2 (systemic, repetitive, or significant) deficiencies.
• The system of quality control operated effectively at the engagement level, as all files reviewed were either compliant or compliant with improvement needed. We identified a number of Category 3 deficiencies, three Category 2 deficiencies, and no Category 1 deficiencies. The reports the Office issued were appropriate in the circumstances for the engagements Practice Review and Internal Audit reviewed.

The scope of the monitoring exercise included assessing the design and implementation of the Office’s system of quality control. Assessing the design addressed how relevant and adequate the system of quality control policies and procedures were. Assessing the system’s implementation addressed its operational effectiveness.

Table 1 describes the areas for improvement related to the operational effectiveness of the system of quality control at the Office level.

The Practice Review and Internal Audit team identified deficiencies, presented findings of non-compliance on specific elements of the system of quality control, and made recommendations related to the operating effectiveness at the engagement level in both the Annual Audit and the Direct Engagement practices. You can find the recommendations in the following reports on the Office’s Internet website:

• Report on a Review of the Direct Engagement Audit Practice—Direct Engagement Audits Completed in the 2015–16 Fiscal Year, and
• Report on a Review of the Financial Audit Practice—Financial Audits Completed in the 2015–16 Fiscal Year.

Findings from reviews of direct engagement audits. The Practice Review and Internal Audit team identified and made recommendations on the following Category 2 (systemic, repetitive, or significant) findings:

• Date of report—One file was dated prior to the quality reviewer finishing his review. In this case, the quality reviewer continued to review documents after the PX draft was sent. In another file, we noted that the engagement leader documented his review of high-risk areas after the date of the audit report. (Paragraph 41)
• Review of high-risk substantiation—In one of these three files, the documentation of that review was completed one day after the PX draft was sent to the entity. In another of these three files, the engagement leader did not document which paragraphs they considered to be high-risk paragraphs. (Paragraph 52)

Findings from reviews of annual audits. The Practice Review and Internal Audit team identified and made recommendations on the following Category 3 (isolated) findings:

• Security of sensitive information—Even though the documents were stored in an appropriate and secure container (TeamMate), there is still a risk that these unmarked documents could become vulnerable if removed from their secure environment by being printed or emailed to other users. (Paragraph 21)
• Performance materiality—We noted that in two of the files we reviewed, the calculation was neither performed nor documented according to the practice team’s interpretation of the policy. However, as part of our file review, we performed the calculation and concluded that the two files were in compliance. We consider this issue to be systematic [sic], and believe the related Office policy needs clarification. (Paragraph 31)

Introduction

1. The Canadian Standard on Quality Control (CSQC 1) from the Chartered Professional Accountants of Canada requires that a quality control system applicable to all assurance engagements be established and maintained.

2. For the Office, this provides reasonable assurance that the Office and its personnel comply with professional standards and applicable legal and regulatory requirements, and that the audit reports the Office issues are appropriate in the circumstances.

3. Monitoring compliance with quality control policies and procedures is meant to evaluate

• whether the Office observes professional standards and applicable legal and regulatory requirements;
• whether the Office has appropriately designed and effectively implemented the system of quality control; and
• whether the Office has properly applied its quality control policies and procedures, so that reports the Office issued are appropriate in the circumstances.

4. The Office is required to communicate the results of the monitoring process annually to the Auditor General and to management, and to recommend appropriate remedial action where necessary. This report fulfills that requirement.

5. The Office’s monitoring process is divided into two distinct parts:

• Annual monitoring (policy monitoring)—This is a yearly evaluation of how the Office complies with its system of quality control policies and procedures (the objective of this monitoring exercise), but excludes the inspection of specific engagement files.
• Practice review (completed file monitoring)—This is a cyclical inspection of completed engagement files. A completed assurance engagement is inspected for each engagement leader at least once in every four-year period. Practice review results are taken into consideration in the annual monitoring exercise and are reported separately.

6. The annual monitoring process assesses deficiencies found according to the following categories:

• Category 1—Serious: These matters require immediate corrective action to comply with professional standards and legal and regulatory requirements.
• Category 2—Systemic, repetitive, or significant: These matters require prompt corrective action or changes in the Office’s policies and procedures.
• Category 3—Isolated: These matters require consideration but do not show that the Office’s system of quality control is deficient or that the engagement reports it issued were inappropriate.

7. Category 1 and 2 deficiencies reflect major weaknesses that could prevent the Office from achieving CSQC 1 objectives.

8. This report reflects the two distinct parts of the monitoring process:

• a detailed report on the Office’s annual monitoring, and
• the summary results from the Report on a Review of the Direct Engagement Audit Practice—Direct Engagement Audits Completed in the 2015–16 Fiscal Year, and the Report on a Review of the Financial Audit Practice— Financial Audits Completed in the 2015–16 Fiscal Year.

9. The monitoring exercise covered the period from 1 January 2015 to 31 March 2016. We completed the monitoring work on 31 October 2016.

Assistant Auditor General: Stuart Barr
Principal: Gregg Ruthman
Director: Stéphane Rivest

Findings, Recommendations, and Responses

Design of the system of quality control—Adequacy and relevance

10. Overall, we found that the Office of the Auditor General’s design of the system of quality control was adequate and relevant in meeting the requirements of the Canadian Standard on Quality Control (CSQC 1).

The system of quality control addressed all required CSQC 1 elements

11. We found that the Office’s system of quality control included policies and procedures that addressed the six required elements of CSQC 1.

12. We reviewed the results of previous assessments from the 2014 monitoring processes, the Office’s audit methodology and policies, and the detailed crosswalk that maps the CSQC 1 requirements to the audit methodology and policies.

The Office documented its system of quality control policies and procedures, and communicated them to staff

13. We found that the Office documented its system of quality control policies and procedures, and communicated them to staff.

14. We reviewed how the Office documented and communicated its system of quality control (that is, how available and accessible it was) to ensure that it included a description of the quality control policies and procedures and the objectives they are designed to achieve.

15. The Office documented and communicated the system of policy control policies and procedures mainly through its two product-line manuals (annual audit and direct engagement), procedure libraries for each product line, and the INTRAnet.

16. The manuals describe the quality control policies and procedures and the objectives they are designed to achieve.

17. The system of quality control manual clearly states

• that each auditor has a personal responsibility for quality, and is expected to comply with the policies and procedures;
• that audit teams are responsible for carrying out quality control procedures that apply to the assurance engagement; and
• that audit teams are responsible for providing the Office with relevant information to ensure that the system of quality control functions properly.

The system of quality control was up to date

18. We found that the system of quality control was up to date. The Office has developed a system for monitoring and maintaining the system’s methodology, training, tools, and support.

19. We reviewed new developments in professional standards and regulatory and legal requirements, as well as improvements, updates, and corrections to the Office’s existing system of quality control policies and procedures. We reviewed whether the policies and procedures reflected, where appropriate, these required changes.

20. The Office updated its methodology on both an annual basis for major updates and on an ad hoc basis for more urgent updates.

21. Competent staff were responsible and accountable for monitoring and maintaining methodology. Staff performed these activities in a timely manner, and monitored and took corrective action as needed.

22. The Office has a process for monitoring observations from provincial institute practice inspections and the Canadian Public Accountability Board to determine if there are opportunities to improve the system of quality control to help its practitioners avoid the issues observed in other firms.

23. The Office’s INTRAnet shared announcements of methodology changes (methodology updates, standards interpretations, and notices), providing targeted communications to auditors and a historical reference of the changes made to audit methodology.

24. The Office developed a publication model to make it easier to maintain audit methodology. This model clearly defined the roles and responsibilities for monitoring and maintaining methodology, ensuring the accuracy and integrity of published quality control policies and procedures.

25. A coordinator logged all changes to methodology and also retained related information on consultations and approvals.

26. The Annual Audit Practice Team is responsible for a weekly process that monitors for and identifies upcoming changes proposed by Canadian and international standards setters. The team communicated upcoming changes identified as affecting the system of quality control to the central system of quality control team.

27. The Office entered into a strategic alliance with PriceWaterhouseCoopers whereby the Office had rights to PriceWaterhouseCoopers’ audit methodology and updates. As a result, PriceWaterhouseCoopers updated the Office on the changes it had made to its audit methodologies. The Office used this information to update its own system of quality control policies and procedures.

28. The Office monitors the activities of standards-setting bodies for legislative auditing. This results in formal and informal discussions with practice teams and monitoring applicable professional standards. Several members of management participated in the activities of standards-setting bodies.

29. The Office consults Legal Services annually on changes to legislation and regulations that affect the methodology.

30. Engagement leaders monitored changes in enabling legislation and the operational laws and regulations of the entities that they audited. The assistant auditors general and the principals responsible for entities developed strategic relationships with senior members within the portfolio of entities that they audited.

The Office responded to recommendations made by previous monitoring and other reports

31. We found that the Office responded to the recommendations from previous monitoring and other reports.

32. We reviewed observations from provincial institute practice inspections for the attest product line to determine if the system of quality control could be improved and if we could help practitioners avoid the issues identified in other firms. We did not note any requirement to improve the Office’s system.

33. We reviewed responses to previous recommendations from Office monitoring and provincial institute practice inspections.

34. Previous monitoring reports. The Monitoring Report on the System of Quality Control—2014 identified no serious deficiencies in the system of quality control. The report did note one isolated issue. This issue was determined not to affect the effectiveness of the system’s operation; this issue was as follows:

• The Office uses specialist skills in the course of its work by means of advisory committees that comprise external experts and internal specialists. The Office has identified and annually reviews the functional areas for internal specialists. While we note that identifying internal specialist areas is done well, we found that the Office does not have a formal process or procedures in place to select and appoint the internal specialists.

35. The Executive Committee approved the submission to the Auditor General of the Monitoring Report on the System of Quality Control—2014. The Executive Committee accepted the recommendations in the report and agreed to take the necessary steps to address the issues.

36. The Assistant Auditor General of Audit Services implemented the use of criteria for internal specialist selection and appointment in fall 2016.

Operational effectiveness of the system of quality control at the Office level

37. Overall, we found that the system of quality control operated effectively at the Office level.

The Office promoted an internal culture of quality

38. We found that the Office promotes an internal culture of quality through clear, consistent, and frequent messages, and rewards high-quality work.

39. We reviewed actions and messages that emphasize the requirement to perform work that complies with professional standards and issue reports that are appropriate in the circumstances. We also assessed whether the Office provided enough resources to develop, document, and support its quality control policies and procedures.

40. Senior management actions and messages. The Office’s vision and values are clearly stated and communicated, as well as the Office’s Code of Values, Ethics and Professional Conduct. Awards programs recognize staff members who promote the Office’s values, including product management and quality. The Office’s orientation training program includes an e-learning session to provide participants with a better understanding of the Office’s purpose, culture, and role in government. This e-learning is mandatory for all new hires and ensures that the culture of quality is made clear to all staff. Sharing the results of practice review activities with staff, including recommendations, helps promote a culture of quality and continuous improvement. During the period under review, the assistant auditors general of the audit practices and the Assistant Auditor General of Audit Services reviewed these results. The Chief Audit Executive presented the results at meetings of the principals and directors of attest audits and direct engagements.

41. The Office’s appraisal, promotion, and compensation processes require showing that employees meet the Office’s quality standards, and know and apply the system of quality control.

42. Senior management responsibilities for quality. The Office clearly assigns the roles and responsibilities for the elements of the system of quality control to senior management, who have the appropriate authority to fulfill their related duties.

43. The Auditor General assumes ultimate responsibility for the Office’s system of quality control. The Auditor General appoints the Assistant Auditor General of Audit Services, who is assigned operational responsibility and has an appropriate combination of education, professional qualifications, experience, and skills to fulfill this function’s duties. The Assistant Auditor General has the necessary authority to fulfill these responsibilities.

44. Sufficient resources to support the system of quality control. The Office has enough resources to develop, document, and support the system of quality control. This includes the resources and processes for monitoring new developments in professional standards and integrating changes identified in the monitoring of audit methodology in a way that ensures consistency and completeness. Audit Services is the operational centre for the system of quality control, and has resources from two product-line practice teams—the Annual Audit Practice Team and the Direct Engagement Practice Team. The practice teams conduct the following activities:

• monitor for new developments in professional standards, laws, and regulations;
• coordinate a common look and feel for methodology;
• provide quality assurance and advice; and
• ensure the accuracy and integrity of published methodology, including TeamMate procedures.

Office staff understood and complied with relevant ethical requirements

45. We found that all staff required to complete an annual Declaration of Conflict of Interest form for the 2015–16 fiscal year did so.

46. We found that the Internal Specialist, Values and Ethics assessed exception reports initiated in the 2015–16 fiscal year, and applied appropriate safeguards where necessary.

47. We found that the Office had an annual process for evaluating and managing rotation requirements.

48. We reviewed the processes for annual confidential declarations, the identification of threats to independence in exception reports, and job rotation analysis and actions.

49. Annual confidential declarations. To demonstrate their understanding of these fundamental principles and compliance with Office protocols, employees must read, understand, and adhere to the Office’s Code of Values, Ethics and Professional Conduct. Adhering to ethical requirements includes signing an annual Conflict of Interest form (annual confidential declaration) and assurance engagement reports on independence before beginning work on any assurance engagement. If employees identify threats to compliance with ethical requirements or independence, they must complete an Exception Report to help resolve the threat.

50. The Principal of Human Resources emails independence requirements annually to staff and the Office maintains an automated mandatory annual process that requires staff to declare their independence. The system sends the request to all users and tracks progress from the request initiation, to printing, delivery to Human Resources, and ultimately to Records Management. The Principal of Human Resources generates reports that track the progress and completion rate. The system automatically sends reminders to staff who have not completed the declaration. For the 2015–16 fiscal year, all staff members who were required to complete an annual declaration did so.

51. Exception reports. Staff members are required to promptly notify the Office of any circumstances or relationships that create threats to independence. If the threat is significant, the employee is required to initiate an Exception Report, which identifies the threat, and documents its impact and the appropriate action required to eliminate the threat or reduce it to an acceptable level. The Internal Specialist, Values and Ethics, reviews the report objectively and assesses the proposed safeguards, which may include additional actions to reduce the threat to an acceptable level. These safeguards reflect the individual’s level of influence on an audit and may include the following:

• increasing the level of supervision of the individual on the audit,
• segregating the individual from certain audit lines of enquiry, or
• removing the individual from the audit.

52. The Internal Specialist, Values and Ethics, assessed all exception reports initiated in the 2015–16 fiscal year and applied appropriate safeguards where required.

53. Job rotation. The Office’s objectivity may be threatened or appear to be threatened if senior personnel and quality reviewers, where applicable, continue to work with the same entity for a prolonged period. Staff rotation is often achieved through promotion or staff turnover; however, the responsibilities of senior personnel with signing authority are less likely to change unless a policy requires staff rotation. The Office job rotation policy requires that each year, Audit Services identify those senior personnel requiring job rotation to the assistant auditors general of the applicable audit practice for consideration and approval. The Executive Committee must approve exceptions to the job rotation policy for assistant auditors general, assistant auditors general of the applicable audit practice must approve exceptions for engagement leaders, and the Assistant Auditor General, Audit Services, and the Auditor General must approve exceptions to quality reviewer rotation requirements. Exceptions are granted only if appropriate safeguards exist.

54. We found that the Office performed an appropriate job rotation analysis. For the situations where extensions were required, the Office had appropriate safeguards to eliminate or reduce the familiarity threat to independence to an acceptable level and approved exceptions appropriately.

The Office fulfilled acceptance and continuance requirements

55. We found that the Office had processes to ensure that audit staff adhered to the principles of acceptance and continuance and applied them to all assurance engagements.

56. We reviewed Executive Committee records of decisions, and interviewed Legal Services to determine whether audit staff had followed acceptance and continuance processes at the Office level, and whether the Office had identified and resolved any threats of familiarity with an entity.

57. For a legislative audit office such as the Office of the Auditor General, many assurance engagements are required by legislation; the Office conducts other engagements at its discretion. For discretionary audits, the Office refers all requests for appointment by order-in-council or under the Financial Administration Act to Legal Services to determine whether the Office has the authority to conduct the engagement.

58. At the engagement level, for both discretionary and statutory audits, engagement leaders perform and document acceptance procedures for all new engagements. For statutory audits, if the Office decides it needs to waive or decline a statutory appointment, were that option available, the engagement leader prepares a briefing note and presents it to the Executive Committee for review. Legal Services may analyze whether there is a professional, legal, or regulatory requirement to remain as auditor or whether the Office should report the withdrawal, cancellation, or postponement, and the justification for that decision, to others outside the Office.

59. We found that, during the 2015–16 fiscal year, the Office completed the required Office-level acceptance and continuance procedures and reviewed all threats or acceptance and continuance actions.

The Office ensured it had sufficient staff with competence, capabilities, and commitment to ethical principles

60. We found that the Office assessed the competencies and capabilities it required at the team and group levels. It also developed a global staffing profile, as well as a People Management Framework, that helped to assess staffing needs. However, we found that the Office did not have a current recruitment strategy.

61. We reviewed documentation and carried out interviews on the following:

• recruitment (assessing staffing needs),
• use of specialist skills,
• assigning of professional personnel (reviewed the mandate of the Audit Resource Planning and Career Management team),
• staff training and professional development, and
• performance management.

62. Assessing staffing needs. The Office has an annual process to determine professional staffing needs at the Financial Audit Trainee and Performance Audit Trainee levels.

63. The goal of the Financial Audit Trainee and Performance Audit Trainee programs is to recruit, train, and retain employees with the general competencies required to become good financial and performance auditors. The Financial Audit Trainee program recruits university students from accounting programs to fill the permanent needs in the Office’s financial auditing operations. The Performance Audit Trainee program recruits students who have master’s degrees from a Canadian university. The Office has determined that these requirements provide the required general competencies and capabilities.

64. During the course of these two- or three-year programs, trainees must demonstrate that they meet additional specific Office competencies, which include delivering products according to the system of quality control.

65. In addition to assessing students’ needs, the Audit Resource Planning and Career Management team reviewed group profiles at the Audit Professional level (AP1–3) and analyzed Audit Professional-level staff to help identify the competencies and capabilities the Office requires at this level.

66. The Office reviewed its Office-level governance and senior-level functions, so it could redefine these roles and responsibilities to eliminate duplicating functions and increase efficiencies. As part of this review, the Audit Resource Planning and Career Management team took part in an initiative called the “realistic profile for audits.” This initiative established the Office’s current staff profile on a global level. The Office will review this profile annually for any significant changes.

67. The People Management Framework and related Human Resources policies and procedures did not have a formal recruitment strategy.

68. Recommendation. Human Resources should develop a formal recruitment strategy that helps the organization effectively recruit sufficient staff with competence, capabilities, and commitment to ethical principles.

Management’s response. The Office has recently drafted a Resourcing Strategy to address the internal and external staffing pressures that have developed in the period after the Strategic and Operational Review.

The Resourcing Strategy is in the consultation phase and we expect it to be finalized, translated, and posted on the INTRAnet by 31 March 2017.

69. Consultations with specialists. The Office used specialists’ skills in the course of its work by means of advisory committees made up of external experts and internal specialists. The Office identified and reviewed the functional areas for internal specialists. Although we noted that identifying internal specialist areas was done well, we found that the Office did not have a formal process in place to select and appoint internal specialists.

70. Recommendation. Audit Services should set criteria for internal specialists for evaluating the qualifications of persons appointed as internal specialists. Such criteria might include requirements for related designations and/or experience, for educational background and training, and for maintaining professional competencies and experience and the capacity to provide advice in the area of specialty.

Management’s response. The Assistant Auditor General of Audit Services implemented the use of criteria for internal specialist selection and appointment in fall 2016.

71. Assigning professional personnel. The Office had a process for assigning professional personnel to audit engagements. Directors worked with the Audit Resource Planning and Career Management team and the Resource Manager to assess and document the assignment of appropriate staff with the necessary competencies to the assurance engagements under their responsibility.

72. The Principal is responsible for validating that the staff mix of the engagement team, specialists, and any audit experts collectively have the appropriate competencies and capabilities to meet the requirements of audit and assurance standards.

73. Office policy requires that before the engagement team completes the planning or survey phase of an assurance engagement, the Principal must assess the team to be satisfied that the team, the specialists, and any experts who are not part of the audit team, collectively have the appropriate competence and capabilities, and assign roles and responsibilities. The Principal uses the Engagement Team Competency and Resource Assessment form to document this assessment, which also captures the strategies needed to ensure that

• the Principal assigns the appropriate staff with the necessary competencies, and
• there are adequate resources and time available for the assurance engagement under the Principal’s responsibility.

74. While Principals are responsible for the resource planning of audit teams, the Audit Resource Planning and Career Management team supports them in the following ways:

• managing and monitoring the corporate resource planning tool, Retain, a resource database that contains information by auditor and audit product;
• analyzing resource needs to support assistant auditors general in audit practices, senior Office committees, and audit teams;
• assisting staff transfers and sharing among groups; and
• identifying possible solutions for critical resource requirements.

75. The Audit Resource Planning and Career Management team may be consulted

• by resource managers or principals when an audit team is looking for a resource or for an assignment or transfer for their auditors, and
• by auditors who are looking for an assignment or a transfer.

76. We found that the process to assign professional personnel was appropriate and operated effectively within each practice.

77. Training and professional development. The Office had invested in training and professional development for its product and people management. It had

• assessed training and professional development needs,
• developed a professional development business plan to address gaps and opportunities and add to its value proposition,
• created budgets, and
• dedicated resources to training and professional development.

78. In its learning vision, the Office states that it is committed to building and promoting a learning culture that adds value to its work for Parliament and Canadians and supports the lifelong learning of Office employees.

79. For a few years, the Office has invested heavily in renewing the audit training curriculum, methodology, and tools. The Office used a training needs analysis that assesses training and professional development needs by competency and skill level, developed a professional development business plan to address gaps and opportunities and add to its value proposition, created training and professional development budgets, and dedicated resources to training and professional development.

80. The Office has developed the Leadership Program, which focuses on people management, to meet the professional development needs of the Office’s leaders and assist continuous learning in this area. Leadership is a key component of the system of quality control. The program follows a multi-dimensional approach that includes formal training, interactive knowledge-sharing events, practical tools and resources, and coaching, as well as support services to resolve issues.

81. The Office has a vision for learning that focuses on continuous learning beyond the classroom. One of the key elements of this vision is emphasizing on-the-job coaching and offering on-the-job experiences that are relevant to staff. The Office’s Professional Development team’s role is to give staff the best formal training possible and to help managers provide feedback and coaching as staff experiment with newly acquired skills.

82. Professional Development does an annual scan of the training and professional development environment by consulting with product leaders, reviewing training evaluations, and consulting with accounting firms (that is, PricewaterhouseCoopers and Deloitte) on what is happening in the industry. Based on the results, Professional Development updates training and professional development initiatives.

83. Performance management. In accordance with professional standards, the Office had an annual performance management system in place that required managing products to a high level of quality.

84. The Office has a process for performance management in place that includes goal setting, competencies, ongoing feedback, assessment processes, corrective actions, training and development, and career planning. All active staff receive mandatory annual performance appraisals.

85. The performance appraisal process includes assessing values and competencies, and being required to manage products to a high level of quality according to standards. For any quality-related issues that management identifies, Human Resources helps to remedy the situation through coaching and mentoring, more frequent follow-ups, training, and other appropriate corrective actions.

86. We found that during the 2015–16 fiscal year, the Office completed performance appraisals for most active staff. Human resources monitors and follows up on performance appraisals to ensure that they are completed for all active staff.

The Office encouraged the reporting of complaints and allegations about how it conducted its work

87. We found that the Office encouraged the reporting of complaints and allegations about the conduct of its work.

88. We reviewed documentation and interviewed Legal Services about complaints and allegations it received about failing to comply with professional standards and/or the Office’s system of quality control for work the Office performed during the period under review.

89. The Office’s policies—OAG Audit 1012 Audit Quality and OAG Audit 1091 Complaints and Allegations—meet the CSQC 1 requirements for addressing complaints and allegations. The Office communicates these policies to all employees by means of the INTRAnet.

90. The Office receives external and internal complaints through a public inbox managed by the Communications team. Complaints are then tracked in a database, and the Auditor General or the Chair of the Audit Committee and whomever is appointed as investigator addresses and investigates them. The targeted response time on all issues is 90 days.

91. The Executive Committee receives a quarterly status report on all closed and outstanding complaints and allegations. For the period under review, the Office received no complaints or allegations either internally or externally about how it conducted its audits regarding the system of quality control.

The Office communicated the results of the monitoring process

92. We found that the Office communicated the results of the monitoring process.

93. We reviewed the publications of the monitoring reports on the Office’s Internet website and the corporate messages to all staff announcing the Executive Committee’s approval of the Monitoring Report on the System of Quality Control.

94. The Office’s policy, OAG Audit 1012 Audit Quality, meets the CSQC 1 requirements for communicating the results of the monitoring process. The Office communicates this policy to all employees by means of the INTRAnet.

95. The results of the monitoring process are published on the Office’s Internet website and include the following reports

• Monitoring Report on the System of Quality Control—2014,
• Report on a Review of the Annual Audit Practice—Annual Audits Completed in the 2014–15 Fiscal Year, and
• Report on a Review of the Direct Report Audit Practice—Direct Report Audits Completed in the 2014–15 Fiscal Year.

96. The Office communicates the approval by the Executive Committee of the Monitoring Report on the System of Quality Control to all staff.

The Office addressed cases in which the results of monitoring procedures showed that reports may have been inappropriate or that procedures were omitted during the performance of the engagement

97. We found that the Office complied with the requirements to address cases in which the results of the monitoring procedures showed that a report may have been inappropriate or that procedures were omitted during the performance of the engagement.

98. We reviewed the Protocol for Practice Reviews and Internal Audits and the Practice Review Programs.

99. The Office’s Protocol for Practice Reviews and Internal Audits defines the process in cases where the monitoring procedures’ results show that a report may be inappropriate or that procedures were omitted during the performance of the engagement.

The Office monitored changes to standards and updated methodology in a timely fashion

100. We found that the Office monitored changes to standards and updated methodology in a timely fashion.

101. We reviewed the Monitoring of Emerging Standards Process and the annual deployment of changes to standards into methodology.

102. The Office’s process for maintaining the system of quality control and the Audit Methodology and ensuring it is up to date met the CSQC 1 requirements.

Office specialists were consulted when required

103. We found that audit teams consulted the various Office specialists and documented the extent of their consultations as required by the system of quality control.

104. We reviewed consultation data and details from audit teams of various specialists.

105. The Office’s policy, OAG Audit 3081 Consultations, defines the importance of consultations within the conduct of audits so that it can reduce the risk of error and improve how audit teams apply professional judgment. The policy also defines the process for consultations and the requirements for documenting the consultations.

106. We found that audit teams consulted Office specialists when dealing with complex, unusual, or unfamiliar issues.

107. We also found that information on the extent, details, and conclusions of consultations with internal specialists were not available.

108. Recommendation. Audit Services should establish and communicate requirements and guidelines for internal specialists to maintain summary information on the nature and extent of consultations with internal specialists.

Management’s response. By June 2017, Audit Services will define and communicate documentation requirements and guidelines for the documentation of summary information on the nature and extent of consultations with internal specialists.

Operational effectiveness of the system of quality control at the engagement level

109. Overall, we found that the system of quality control operated effectively at the engagement level. In the practice reviews of engagements, the auditors’ reports were supported and appropriate.

110. Four of the deficiencies that the Practice Review and Internal Audit team noted represented systemic deficiencies requiring prompt corrective action.

Engagements complied with professional standards for quality control

111. We found that the Practice Review and Internal Audit team made recommendations about engagement performance and presented findings of non-compliance and compliance with improvement needed on specific elements of the system of quality control in the Annual Audit and the Direct Engagement practices.

112. We found that four of the deficiencies that Practice Review and Internal Audit noted represented systemic deficiencies requiring prompt corrective action.

113. We reviewed practice review audit program and the reports on the reviews of the practices for the 2015–16 fiscal year to determine whether they complied with the system of quality control at the engagement level. Practice Review and Internal Audit reviewed four attest audit files, two limited scope attest audit files, and six direct reporting engagement files in the following areas:

• supervision and review,
• engagement quality control review,
• differences of opinion,
• engagement documentation, and
• ethics and independence.

114. Supervision and review. Ensuring that team members complete the Office’s assurance engagements to the highest quality requires the Office to adequately supervise team members and to review audit work and documentation. Supervision is important to ensure that engagement teams are organized and that the quality of the work produced during the engagement is monitored for quality. Review is important to ensure that

• team members performed the work according to professional standards,
• the work supports the conclusions reached,
• the evidence obtained is sufficient and appropriate, and
• team members achieved the objectives of the engagement procedures.

115. Engagement quality control review. Quality reviews objectively evaluate the significant judgments the engagement team made and the conclusions reached in formulating the assurance engagement report. The Office assigns quality reviewers to each annual audit of entities that issue or have securities outstanding in public markets. The Office also assigns quality reviewers to other assurance engagements based on the assurance engagement’s level of risk. Quality reviewers have the technical qualifications to perform the role, as well as sufficient and appropriate experience and authority.

116. The Office has a process for selecting and appointing quality reviewers. The Office selects quality reviewers based on the engagement’s level of risk. Audit Services receives risk assessment input from each audit team and prepares a risk assessment for all engagements using selection criteria outlined in the methodology. It is normally recommended that engagements assessed as high risk be selected for a quality review. Low- to medium-risk audits are not normally assigned a quality reviewer.

117. For engagements selected for a quality review, Audit Services appoints a quality reviewer based on specific criteria. Audit Services consults the assistant auditors general annually to review the list of audit engagements for its group and whether its senior personnel are available for quality review assignments. Once Audit Services compiles all risk assessments, it consults the product leaders on the recommended quality reviewer selection and appointment.

118. Differences of opinion. During the course of an assurance engagement, the team, those consulted about the assurance engagement, and the engagement leader and quality reviewer may have differences of opinion. Audit team members have the right to form their own conclusions on significant matters in the areas of the assurance engagement for which they are responsible, and ensure that their views receive adequate consideration. Teams should not date an assurance engagement report until team members resolve all differences of opinion.

119. Engagement documentation. This component addresses the confidentiality, safe custody, integrity, accessibility, retrievability, and retention of engagement documentation and the completion of the final assembly of engagement files on time.

120. Ethics and independence. This component addresses the audit staff independence requirement to the engagement.

121. The detailed observations from monitoring the operational effectiveness of the system of quality control at the engagement level can be found in the following reports on the Office’s Internet website:

• Report on a Review of the Direct Report Audit Practice—Direct Report Audits Completed in the 2014–15 Fiscal Year, and
• Report on a Review of the Annual Audit Practice—Annual Audits Completed in the 2014–15 Fiscal Year.

Engagement team competencies were assessed and documented

122. We found that that the engagement leader assessed and documented engagement team competencies.

123. We conducted a detailed review of the 2015–16 fiscal year practice review files (eight direct reporting engagements and six attest audits) to determine team competencies were assessed and whether the assessment was thorough enough.

124. Before the planning/survey phase was completed, the engagement leader assessed the team to be satisfied that members, specialists, and others collectively had the appropriate competence and capabilities; this leader documented the assessment.

Engagement teams undertook and documented appropriate consultation

125. We found that the engagement teams performed and documented appropriate consultation.

126. We conducted a detailed review of the practice review files for the 2015–16 fiscal year to determine the nature of the consultations undertaken and whether they were sufficient.

127. Audit teams consult with internal and external specialists and senior Office staff when dealing with difficult or contentious matters or other matters requiring specialized knowledge or experience. Before the date of the assurance report, both the individual seeking consultation and the party consulted agree to the nature and scope of consultations, and the conclusions resulting from them. The teams then carry out the conclusions resulting from consultations.