Executive Summary

The Canadian Standard on Quality Control (CSQC 1) from the Chartered Professional Accountants of Canada (CPA Canada) requires that the Office of the Auditor General of Canada (OAG or the Office) establish and maintain a quality control system applicable to all the Office’s assurance engagements. This provides reasonable assurance that the Office and its personnel comply with professional standards and applicable legal and regulatory requirements, and that audit reports issued by the Office are appropriate in the circumstances.

The Office is required to establish a monitoring process designed to provide it with reasonable assurance that the policies and procedures relating to the system of quality control are relevant, adequate, and operating effectively.

The Office is required to communicate the results of the monitoring process annually to the Auditor General and management, and to recommend appropriate remedial action where necessary. This report fulfills that requirement.

We reviewed the Office’s system of quality control (SoQC) for the period from January to December 2014. The scope of the monitoring exercise included assessing the relevance and adequacy of the design and the operational effectiveness of the system of quality control. The monitoring work focused on each element of the system, as well as on one other higher-risk area: whether the consultation by the engagement team was adequate.

Overall, based on this evaluation, the Office’s system of quality control is relevant and adequate, and is operating effectively. We did not identify any serious deficiencies.

However, we noted one area for improvement related to the operational effectiveness of the system of quality control. Nevertheless, this weakness did not affect the adequacy or relevance of the system or the appropriateness of the audit reports issued. We made the following recommendation in this area:

The Professional Practices Group should set criteria for internal specialists for evaluating the qualifications of persons appointed as internal specialists. Such criteria might include requirements for related designations and/or experience, for educational background and training, and for maintaining professional competencies and experience and the capacity to provide advice in the area of speciality.

Management’s response. For the 2016–17 fiscal year, the Assistant Auditor General of the Professional Practices Group will set criteria for internal specialists.

Recommendations in other reports. The Practice Review and Internal Audit team made recommendations for engagement performance and also presented one finding of non-compliance on a specific element of the SoQC in the Direct Engagement practice. You can find the recommendations in the following reports on our Internet site:

• Report on a Review of the Direct Report Audit Practice—Direct Report Audits Completed in the 2014–15 Fiscal Year, and
• Report on a Review of the Annual Audit Practice—Annual Audits Completed in the 2014–15 Fiscal Year.

Introduction

Background

1. The Canadian Standard on Quality Control (CSQC 1) from the Chartered Professional Accountants of Canada (CPA Canada) requires that the Office of the Auditor General of Canada (OAG or the Office) establish and maintain a quality control system applicable to all the Office’s assurance engagements, including annual audits, performance audits, and special examinations. The objectives of CSQC 1 are to provide reasonable assurance that the Office and its personnel comply with professional standards and applicable legal and regulatory requirements, and that the audit reports issued are appropriate in the circumstances.

2. A system of quality control consists of policies designed to achieve the objectives established by CSQC 1 and the procedures necessary to implement and monitor compliance with those policies. Table 1 describes each of the elements of CSQC 1.

Focus of monitoring

3. CSQC 1 requires the Office to establish a monitoring process that will assess whether it is complying with its system of quality control and professional standards on an ongoing basis, report on whether a quality control system has been appropriately designed and carried out, and provide recommendations to correct the deficiencies identified.

4. The objective of monitoring is to provide reasonable assurance that the policies and procedures that make up the Office’s system of quality control (SoQC) are relevant, adequate, and operating effectively. According to CSQC 1, monitoring is an ongoing consideration and evaluation of the OAG’s system of quality control (“policy monitoring”) including inspection of at least one completed engagement for each engagement leader on a cyclical basis (“completed file monitoring”).

5. Monitoring is used as a tool for management to evaluate the Office’s system of quality control and the results of monitoring are reported according to the requirements of CSQC 1. The monitoring report is intended for management purposes. The Office is required to communicate the results of the monitoring process annually to the Auditor General and to management, and to recommend appropriate remedial action where necessary. This report fulfills this requirement.

6. Monitoring is performed as an active part of the OAG’s system of quality control. The purpose of monitoring compliance with quality control policies and procedures is to evaluate

• adherence to professional standards and applicable legal and regulatory requirements;
• whether the system of quality control has been appropriately designed and effectively carried out; and
• whether the OAG’s quality control policies and procedures have been properly applied, so that reports issued by the OAG are appropriate in the circumstances.

7. The results of monitoring enable management to determine whether key components of internal control continue to function over time, and to identify and communicate internal control deficiencies in a timely manner to those parties responsible for taking corrective action, and to others, as appropriate. The findings and recommendations help to ensure that the Office manages its risks properly over time. They can also help to standardize practice among practitioners and ensure that they use best practices consistently.

8. The scope of the monitoring exercise included assessing the design and implementation of the Office’s system of quality control. Assessing the design addressed how relevant and adequate the SoQC policies and procedures were. Assessing the system’s implementation addressed its operational effectiveness.

9. This monitoring exercise reviewed the system of quality control for the period from January to December 2014.

10. Our monitoring work focused on each element of the system of quality control, as well as one other area considered to be of potential concern: whether the engagement team’s consultations were adequate. Table 2 provides an overview of the monitoring coverage of quality control elements as required by CSQC 1.

11. The Office’s monitoring process is divided into two distinct parts, as required by CSQC 1:

• Annual monitoring (“policy monitoring”)—A yearly evaluation of how the Office is complying with its system of quality control policies and procedures (the objective of this monitoring exercise) but excluding the inspection of specific engagement files.
• Practice review (“completed file monitoring”)—A cyclical inspection of completed engagement files. A completed assurance engagement is inspected for each engagement leader at least once in every four-year period. Practice review results are taken into consideration in the annual monitoring exercise and are reported separately.

12. This report includes the summary results from the Practice Review team’s engagement file inspections (“completed file monitoring”). The annual summary report from file inspections follows a separate review and communication process that includes public disclosure. Although CSQC 1 requires that file inspections be part of the overall monitoring process, it does not require that the results of monitoring the system of quality control and the results from engagement-level file inspections be communicated at the same time.

13. The annual monitoring process assesses deficiencies found according to the following categories:

• Category 1—Serious: Matters that require immediate corrective action to comply with professional standards and legal and regulatory requirements.
• Category 2—Systemic, repetitive, or significant: Matters that require prompt corrective action or changes in the Office’s policies and procedures.
• Category 3—Isolated: Matters that require consideration but that do not show that the Office’s SoQC is deficient or that the engagement reports issued are inappropriate.

14. For the purposes of this report, Category 1 and 2 deficiencies refer to major weaknesses that could prevent the Office from achieving the objectives of CSQC 1.

Findings, Recommendations, and Responses

Design of the system of quality control—Adequacy and relevance

15. Overall, we found that the Office of the Auditor General’s design of the system of quality control (SoQC) was adequate and relevant in meeting the requirements of the Canadian Standard on Quality Control (CSQC 1).

16. The Office has established a system of quality control, which comprises policies and procedures that address the six elements required by CSQC 1. The Office documents and communicates the SoQC to staff to ensure that its staff consistently performs high quality work. To ensure the system’s continued relevance, the Office has a process in place to monitor and maintain its quality control methodology.

The system of quality control addressed all required CSQC 1 elements

17. We found that the Office’s system of quality control included policies and procedures that addressed the six required elements of CSQC 1.

18. This finding is important because CSQC 1 requirements are designed to enable the Office to achieve reasonable assurance that

• the Office and its staff comply with professional standards and applicable legal and regulatory requirements, and
• reports issued by the Office are appropriate in the circumstances.

19. Applying these requirements properly is expected to ensure that quality work is performed by the Office and its staff consistently and according to professional standards.

20. We made no recommendation in this area.

21. The Office conducted extensive work during the Office’s Renewal of Audit Methodology (RAM) project to ensure that it had addressed all CSQC 1 requirements, with extensive guidance from PricewaterhouseCoopers (PwC), crosswalks that map our work to professional standards, and consultations. Work done for the 2012 and 2013 monitoring processes validated this finding.

22. We reviewed the results of previous assessments from the 2012 and the 2013 monitoring processes and the Office’s audit methodology and policies.

The Office documented its system of quality control policies and procedures, and communicated them to staff

23. We found that the Office documented its system of quality control policies and procedures, and communicated them to staff.

24. This finding is important because performing high quality work consistently requires policies that are documented and communicated and procedures that are supported by Office leaders who are committed to quality. Proper documentation and communication supports the development of an Office culture that fosters a positive attitude toward quality, complies with professional standards, and monitors the Office’s controls.

25. We made no recommendation in this area.

26. The SoQC policies and procedures are documented and communicated mainly through the Office’s three product-line manuals (annual audit, performance audit, and special examination), procedure libraries for each product line, and the INTRAnet.

27. The manuals include a description of the quality control policies and procedures, and the objectives they are designed to achieve.

28. The SoQC manual clearly states

• that each auditor has a personal responsibility for quality, and is expected to comply with the policies and procedures;
• that audit teams are responsible for carrying out quality control procedures that apply to the assurance engagement; and
• that audit teams are responsible for providing the Office with relevant information to ensure that the system of quality control functions properly.

29. We reviewed how the Office documented and communicated its system of quality control (that is, how available and accessible it was) to ensure that it included a description of the quality control policies and procedures, and the objectives they are designed to achieve.

The system of quality control was up to date

30. We found that the system of quality control was up to date. The Office has developed a system for monitoring and maintaining the system’s methodology, training, tools, and support.

31. This finding is important because the Office can only have reasonable assurance that its quality control policies and procedures are relevant if they are up to date. CSQC 1 requires that the Office analyze new developments in professional standards and applicable legal and regulatory requirements, and reflect them in the Office’s policies and procedures. Maintaining up-to-date quality control methodology entails

• monitoring the key sources that affect the methodology; and
• integrating changes identified in a way that ensures consistency and completeness throughout policies and procedures, TeamMate libraries and templates, guidance, and checklists.

32. We made no recommendation in this area.

33. The Office updated its methodology on both an annual basis for major updates and on an ad hoc basis for more urgent updates.

34. Competent staff were responsible for and accountable for monitoring and maintaining methodology. Staff performed these activities in a timely manner, and monitored and took corrective action as needed.

35. A process is in place to monitor observations from provincial institute practice inspections and the Canadian Public Accountability Board to determine if there are opportunities to improve the system of quality control to help our practitioners avoid the issues observed in other firms.

36. The Office’s INTRAnet captured and displayed announcements of methodology changes (Methodology Updates, Standards Interpretations, and Notices), providing targeted communications to auditors and a historical reference of the nature of the changes made to audit methodology.

37. The Office developed a publication model to make it easier to continue to maintain audit methodology. This model clearly defined the roles and responsibilities for monitoring and maintaining methodology, ensuring the accuracy and integrity of published quality control policies and procedures.

38. A coordinator logged all changes to methodology and also retained related information on consultations and approvals.

39. The Annual Audit Practice Team (AAPT) is responsible for a weekly process designed to monitor and identify upcoming changes proposed by Canadian and international standard setters. An upcoming change identified as affecting the SoQC was communicated to the central SoQC team.

40. The Office entered into a strategic alliance with PwC whereby the Office had rights to PwC’s audit methodology and updates. As a result, the Office received updates on the changes PwC had made to its audit methodologies. The Office used this information to update its own SoQC policies and procedures.

41. The Office monitors the activities of standard setting bodies for legislative auditing. This results in formal and informal discussions with practice teams and monitoring applicable professional standards. Several members of OAG management participated in activities of standard setting bodies.

42. Legal Services is consulted annually on changes to legislation and regulations that affect the methodology.

43. Portfolio assistant auditors general and engagement leaders monitored changes in enabling legislation and the operational laws and regulations of the entities that they audited. Entity assistant auditors general and principals developed strategic relationships with senior members within the portfolio of entities that they audited. If an entity becomes aware of an initiative to change legislation or regulations that may have had an effect on audit methodology, the Office was informed.

44. We reviewed new developments in professional standards and regulatory and legal requirements, as well as improvements, updates, and corrections to the Office’s existing system of quality control policies and procedures. We reviewed whether these required changes were reflected, where appropriate, in the policies and procedures.

The Office responded to recommendations made by previous monitoring and other reports

45. We found that the Office responded to recommendations made by previous monitoring and other reports.

46. This finding is important because identifying any weaknesses in the Office’s system of quality control is crucial for taking timely corrective action.

47. We made no recommendation in this area.

48. Area reviewed: Previous monitoring reports. The 2013 monitoring report identified no “serious” deficiencies in the system of quality control. The report did note one “isolated” issue. This was determined not to affect the effectiveness of the system’s operation. This area was as follows:

• The Office uses specialist skills in the course of its work by means of advisory committees that comprise external experts and internal specialists. The Office has identified and annually reviews the functional areas for internal specialists. While we note that identifying internal specialist areas is done well, we found that no formal training and development were provided for internal specialists to maintain their skills and any related designations.

49. The Executive Committee approved the submission to the Auditor General of the Monitoring Report on the System of Quality Control—2013. The Executive Committee accepted the recommendations in the report and agreed to take the necessary steps to address the issues.

50. Area reviewed: Provincial institute practice inspections. The Office underwent four provincial practice inspections of its financial audit practice in 2014. None of the inspections identified any reportable deficiencies with the system of quality control.

51. We reviewed observations from provincial institute practice inspections for the attest product line to determine if the system of quality control could be improved and if we could help practitioners avoid the issues in other firms. We did not note any requirement to improve the Office’s SoQC.

52. We reviewed responses to previous recommendations from Office monitoring and provincial institute practice inspections.

Operational effectiveness of the system of quality control at the Office level

53. Overall, we found that the system of quality control (SoQC) is operating effectively at the Office level.

54. A system of quality control consists of policies designed to achieve the objectives of the Canadian Standard on Quality Control (CSQC 1), and the procedures needed to carry out those policies. This is important because it enables the Office to for determine if there are any deficiencies and to recommend appropriate remedial action. To help operational effectiveness at the Office level, it is important to promote an internal culture of quality so that the Office of the Auditor General and its staff understand and comply with relevant requirements.

The Office promoted an internal culture of quality

55. The Office promoted an internal culture of quality through clear, consistent, and frequent messages, and rewarded high-quality work.

56. This finding is important because the culture or tone set by senior management is critical in creating an environment for quality work. Promoting an internal culture of quality establishes this as a core value that every employee is expected to have. Frequent communications to staff reinforce the commitment to quality, as does assigning responsibilities for quality to senior members who have enough and appropriate experience and ability, and the necessary authority to assume those responsibilities.

57. We made no recommendation in this area.

58. Area reviewed: Senior management actions and messages. The Office’s vision and values are clearly stated and communicated, as well as the Office’s Code of Values, Ethics and Professional Conduct. Awards programs are in place to recognize staff members who promote the values of the Office, including product management and quality. The Office’s orientation training program includes a session to provide participants with a better understanding of the Office’s purpose, culture, and role in government. The course is mandatory for all new hires and ensures that the culture of quality is made clear to all staff. Sharing the results of practice review activities with staff, including recommendations, helps promote a culture of quality and continuous improvement. In 2014, the product leaders and the Assistant Auditor General of the Professional Practices Group reviewed these results and the Chief Audit Executive presented them at meetings of the principals and directors of attest audits and performance audits.

59. The Office appraisal, promotion, and compensation processes require showing that Office quality standards are met, and knowing and applying the system of quality control. Training sessions for independence are also included in the Office’s orientation training.

60. Area reviewed: Senior management responsibilities for quality. The roles and responsibilities for the elements of the SoQC are clearly assigned to senior management, who have the appropriate authority to fulfill their related duties.

61. The Auditor General assumes ultimate responsibility for the Office’s system of quality control. The Assistant Auditor General of the Professional Practices Group, appointed by the Auditor General, is assigned operational responsibility and has an appropriate combination of education, professional qualifications, experience, and skills to fulfill the duties of this function. The level of the position provides the Assistant Auditor General with the necessary authority to fulfill these responsibilities.

62. Area reviewed: Sufficient resources to support the system of quality control. The Office has enough resources to develop, document, and support the system of quality control. This includes the resources and processes for monitoring new developments in professional standards and integrating changes identified in the monitoring of audit methodology in a way that ensures consistency and completeness. The Professional Practices Group is the operational centre for the system of quality control and has resources from three product-line practice teams—the Annual Audit Practice Team, the Performance Audit Practice Team, and the Special Examination Practice Team—and the Audit Quality Team. The practice teams conduct the following activities:

• monitor for new developments in professional standards, laws, and regulations;
• coordinate a common look and feel across practice teams;
• provide quality assurance by the practice teams; and
• ensure the accuracy and integrity of published methodology, including TeamMate.

63. We reviewed actions and messages that emphasize the requirement to perform work that complies with professional standards and issue reports that are appropriate in the circumstances. We also assessed whether enough resources had been provided to develop, document, and support the Office’s quality control policies and procedures.

Office staff understood and complied with relevant ethical requirements

64. We found that all staff who were required to complete an Annual Confidential Declaration for 2014 did so.

65. We found that the Internal Specialist, Values and Ethics, assessed exception reports initiated in 2014 and applied appropriate safeguards where necessary.

66. We found that the Office had an annual process in place to evaluate and manage rotation requirements for the Annual Audit practice in 2014. This process was not in place in the Direct Engagement practice but the process started in 2015.

67. This finding is important because Office employees maintain public confidence in the impartiality and objectivity of the Office by avoiding and preventing situations that could give rise to a conflict of interest, or even the appearance of or potential for a conflict of interest.

68. There should be an annual analysis of rotation requirements in the Direct Engagement practice, as in the Annual Audit practice. However, because this process started in 2015 and will be completed in 2016, we made no recommendation in this area.

69. Area reviewed: Annual confidential declarations. To show they understand these fundamental principles and their compliance with Office protocols, employees must read, understand, and follow the Office’s Code of Values, Ethics and Professional Conduct. Following ethical requirements includes signing an Annual Conflict of Interest Report (“Annual Confidential Declaration”) and assurance engagement reports on independence before beginning work on any assurance engagement. If employees identify threats to compliance with ethical requirements or independence, they must complete an Exception Report to help resolve the threat.

70. Independence requirements are emailed annually to staff and the Office maintains an automated mandatory annual process that requires staff to declare their independence. The system sends the request to all users and tracks progress from the request, to printing, to delivery to Human Resources, and ultimately to Records Management. It then generates reports that track the progress and completion rate. The system automatically sends reminders to staff who have not completed the declaration. In 2014, all staff members who were required to complete an annual declaration did so.

71. Area reviewed: Exception reports. Staff members are required to promptly notify the Office of any circumstances or relationships that create threats to their independence. If the threat is considered to be significant, the employee is required to initiate an Exception Report. This identifies the threat and documents its effect and the appropriate action required to eliminate the threat or reduce it to an acceptable level. The Internal Specialist, Values and Ethics, reviews the report objectively and assesses the proposed safeguard. This may include further actions to reduce the threat to an acceptable level. These safeguards reflect the individual’s level of influence on an audit and may include the following:

• supervising the individual on the audit at an increased level,
• segregating the individual from certain audit lines of enquiry, or
• removing the individual from the audit.

72. The Internal Specialist, Values and Ethics, assessed all exception reports initiated in 2014 and applied appropriate safeguards where required.

73. Area reviewed: Job rotation. The objectivity of the Office may be threatened or appear to be threatened if senior staff and quality reviewers, where applicable, continue to work with the same entity for a prolonged time period. Normally, staff rotation occurs automatically through promotion or staff turnover; however, the responsibilities of senior personnel with signing authority are less likely to change unless a policy requires rotation. The Office job rotation policy requires the Principal of Human Resources to annually identify those senior personnel requiring job rotation to be considered by the Executive Committee. The Executive Committee must approve exceptions to the job rotation policy and grant them only if appropriate safeguards exist. Rotations in the regions can present special challenges. They may thus require more lead time and more consultations among senior management.

74. Although the Office policy called for the Principal, Human Resources, to perform job rotation analysis, in fall 2014 the analysis was done by the Professional Practices Group for the Annual Audit practice because it has the requisite expertise, knowledge, and data to perform a multi-year analysis and rotation forecast. For the Direct Engagement practice, job rotation analysis was performed by individuals on an engagement-by-engagement basis. However, it should also have been performed at a practice level. The Office rotation policy was changed in 2015 to transfer responsibility for job rotation analysis from the Principal, Human Resources, to the Professional Practices Group. The process will now include an analysis for both practices.

75. In one instance in 2014, an Assistant Auditor General was assigned to an entity for more than seven years in a senior role. In that case, the Executive Committee approved an extension of the term and documented the existing safeguards, thus decreasing risk to an acceptable level.

76. We reviewed the process for annual confidential declarations, how threats to independence in exception reports were identified, and job rotation analysis and actions.

The Office fulfilled acceptance and continuance requirements

77. We found that the Office had processes in place to ensure that the principles of acceptance and continuance were observed and applied to all of its assurance engagements.

78. This finding is important because acceptance and continuance procedures provide the Office with valuable information for performing risk assessments and carrying out reporting responsibilities.

79. We made no recommendation in this area.

80. For a legislative audit office such as the Office of the Auditor General, many assurance engagements are required by legislation, whereas others are conducted at the discretion of the Office. For discretionary audits, all requests for appointment by order-in-council and/or under the Financial Administration Act are referred to Legal Services to determine whether the Office has the authority to conduct the engagement.

81. At the engagement level, for both discretionary and statutory audits, engagement leaders perform and document acceptance procedures for all new engagements. For statutory audits, if the Office decides it needs to waive or decline a statutory appointment, were that option available, the engagement leader prepares a briefing note and presents it to the Executive Committee for review. Legal Services may analyze whether there is a professional, legal, or regulatory requirement to remain as auditor or whether the Office should report the withdrawal, cancellation, or postponement, and the justification for that decision, to others outside the Office.

82. During 2014, the required Office-level acceptance and continuance procedures were completed and all threats or acceptance and continuance actions were reviewed.

83. We reviewed Executive Committee records of decisions, and conducted interviews with Legal Services to determine whether audit staff had followed acceptance and continuance processes at the Office level and whether the Office had identified and resolved any threats of familiarity with an entity.

The Office ensured it had sufficient staff with competence, capabilities, and commitment to ethical principles

84. The Office assessed the competencies and capabilities it required at the team and group levels. It was also developing a global staffing profile as well as a People Management Framework that will help to assess future staffing needs.

85. The Office used specialist skills in the course of its work by means of advisory committees made up of external experts and internal specialists. The Office identified and annually reviewed the functional areas for internal specialists. Although we noted that identifying internal specialist areas was done well, we found that the Office did not have a formal process in place to select and appoint internal specialists.

86. The Office had a solid process in place for assigning professional personnel to audit engagements. Principals worked with the Audit Resource Planning and Career Management team and the Office staff scheduler, Retain, to assess and document the assignment of appropriate staff with the necessary competencies to the assurance engagements under their responsibility.

87. The Office had invested in training and professional development for its product and people management. It had

• assessed training and professional development needs,
• developed a professional development business plan to address gaps and opportunities and add to its value proposition,
• created budgets, and
• dedicated resources to training and professional development.

88. In accordance with professional standards, the Office had an annual performance management system in place that required managing products to a high level of quality.

89. The Office did not have a current Recruitment and Retention strategy. The latest strategy was developed for the 2006–2009 period and was still in effect for 2014. Considering the environment during 2014, this was appropriate.

90. In 2014, the Office did not have a Succession Plan. On 22 April 2015, the Executive Committee approved processes for succession planning and performance management at the Office. For succession planning, Human Resources will conduct interviews with the Auditor General, assistant auditors general (AAG), and service leaders to decide on resources for critical positions. We will follow up on this initiative in the report on the 2015 monitoring activities.

91. These findings are important because employees are the Office’s most important asset and its biggest cost. The Office can only issue reports that are appropriate in the circumstances if its staff has the competence, capabilities, and commitment to ethical principles to perform assurance engagements according to professional standards and ethical requirements.

92. Recommendation. The Professional Practices Group should set criteria for internal specialists for evaluating the qualifications of persons appointed as internal specialists. Such criteria might include requirements for related designations and/or experience, for educational background and training, and for maintaining professional competencies and experience and the capacity to provide advice in the area of speciality.

Management’s response. For the 2016–17 fiscal year, the Assistant Auditor General of the Professional Practices Group will set criteria for internal specialists.

93. Area reviewed: Assessment of staffing needs. The Office has an annual process in place to determine professional staffing needs at the Financial Audit Trainee (APS) and Performance Audit Trainee (APD) levels. Each year, the Audit Resource Planning and Career Management team (ARPCM) reviews the number of APSs and APDs the Office has, analyzes potential departures, meets with assistant auditors general to discuss group requirements for students, and prepares an analysis of APS and APD needs for approval by the Executive Committee.

94. The goal of the APS and APD programs is to recruit, train, and retain employees with the general competencies required to become good financial and performance auditors. The APS program recruits university students from accounting programs to fill the permanent needs in the Office’s financial auditing operations. The APD program recruits students who have master’s degrees from a Canadian university. The Office has determined that these requirements provide the required general competencies and capabilities.

95. During the course of these two-year programs, trainees must demonstrate that they meet additional specific Office competencies, which include delivering products according to the system of quality control.

96. In addition to assessing students’ needs, the ARPCM team was looking at group profiles at the Audit Professional (AP) level (AP1–3) and conducting an analysis of AP-level staff to help identify the competencies and capabilities the Office requires at this level.

97. The Office reviewed its Office-level governance and senior-level functions, so it could redefine these roles and responsibilities to eliminate duplicating functions and increase efficiencies. As part of this review, the ARPCM team took part in an initiative called the “realistic profile for audits.” This initiative will establish the Office’s current staff profile on a global level, the profile it requires going forward, and an appropriate recruitment and promotion strategy to ensure that the Office continues to have the competencies and capabilities it requires. The Office will review this initiative annually for any significant changes.

98. In fall 2014, the Office approved new roles and responsibilities for Senior Management. These new roles and responsibilities started in January 2015.

99. The current recruitment and retention strategy was developed for the period 2006 to 2009.

100. Area reviewed: Use of specialist skills. The Office uses specialist skills from both external and internal resources in its work. Audit teams may use audit advisory committees in the course of their audits, which provide advice on the scope and significance of issues, lines of enquiry, and audit approach. Both external and internal committee members are selected on the basis of their skills, expertise, relevant knowledge, and experience. External advisers are recognized as leaders in their fields of expertise. Typically, advisory committees include two or three external advisers, who bring different perspectives to the subject matter, and appropriate internal specialists, including those with sign-off responsibilities. Advisory committees usually meet twice: once in the planning phase and once late in the examination phase to discuss an early draft of the report.

101. Two categories of internal resource persons from the Office are also available to audit teams to consult with and provide expert advice: product leaders and internal specialists. Product Leader responsibilities include keeping up to date with standards, policies, and developments related to the audit product lines. The Office keeps a single list of internal specialists for all product lines, which is updated annually by the Assistant Auditor General of the Professional Practices Group.

102. Since the 2012 monitoring report that found a lack of clarity in the selection and appointment of internal specialists for all product lines, the Office has developed principles and a process to annually update the list of internal specialists, as well as criteria for selecting functional area. These were approved by the Executive Committee in 2013. This process does not include formalized procedures to select and appoint the internal specialists. To appoint specific individuals as internal specialists in the current process, the Assistant Auditor General of the Professional Practices Group discusses the names of potential internal specialists with individual assistant auditors general. This includes considering such things as related education and experience. However, there are no formal criteria, such as the requirement for a related designation or education, or training, development, and maintenance of specific competencies, capabilities, or designations, to ensure that the most qualified person is appointed as an internal specialist.

103. Audit teams who require additional specific expertise may hire consultants with the necessary expertise on an audit-by-audit basis.

104. Area reviewed: Assignment of professional personnel. The Office has a process in place for assigning professional personnel. According to the Chartered Professional Accountants of Canada’s Quality Assurance Manual, working with Human Resources and a staff scheduler, the engagement leader (or Principal) is responsible for assigning suitably qualified people to each engagement, who collectively have the right combination of competence and capabilities to perform the work required.

105. Office policy requires that before the planning/survey phase of an assurance engagement is completed, the Principal must assess the engagement team to be satisfied that the team, the specialists, and any auditor’s experts who are not part of the audit team, collectively have the appropriate competence and capabilities, and assign roles and responsibilities. This is documented using the Engagement Team Competency and Resource Assessment form, which captures the assessment and strategies needed to ensure that

• the Principal is assigning the appropriate staff with the necessary competencies, and that
• there are adequate resources and time available for the assurance engagement under their responsibility.

106. Whereas principals are responsible for the resource planning of audit teams, the ARPCM team supports them in the following ways:

• managing and monitoring the corporate resource planning tool, Retain, a resource database that contains information by auditor and by audit product;
• conducting resource needs analyses to support product leaders, senior Office committees, and audit teams;
• assisting staff transfer and sharing among groups; and
• identifying possible solutions for critical resource requirements.

107. The ARPCM team may be consulted

• by resource managers or principals when an audit team is looking for a resource or for an assignment or transfer for their auditors, and
• by auditors who are looking for an assignment or a transfer.

108. Area reviewed: Training and professional development. In its learning vision the Office states that it is committed to building and promoting a learning culture that adds value to its work for Parliament and the Canadian people and supports the lifelong learning of Office employees.

109. For a few years, the Office has invested heavily in renewing the audit training curriculum, methodology, and tools. The Office used a training needs analysis that assesses training and professional development needs by competency and skill level, developed a professional development business plan to address gaps and opportunities and add to its value proposition, created training and professional development budgets, and dedicated resources to training and professional development.

110. As part of this, the Office has developed the Leadership Program, which focuses on people management, to meet the professional development needs of the Office’s leaders and assist continuous learning in this area. Leadership is a key component of the system of quality control. The program follows a multi-dimensional approach that includes formal training, interactive knowledge-sharing events, practical tools and resources, and coaching, as well as support services to resolve issues.

111. The Office has developed a new vision for learning focused on continuous learning beyond the classroom. One of the key elements of this vision is emphasizing on-the-job coaching and offering on-the-job experiences that are relevant to staff. The role of the Office’s Professional Development (PD) team is to provide staff with the best formal training possible and to offer support to managers by helping them provide feedback and coaching as staff experiment with newly acquired skills. In the fall of 2013, the Executive Committee endorsed the new vision to reinforce the Office’s culture of continuous learning.

112. PD does an annual scan of the training and professional development environment by consulting with product leaders, reviewing training evaluations, and consulting with accounting firms (that is, PricewaterhouseCoopers and Deloitte) on what is happening in the industry. Based on the results, PD updates training and professional development initiatives.

113. Area reviewed: Performance management. The Office has a process for performance management in place that includes goal setting, competencies, ongoing feedback, assessment processes, corrective actions, training and development, and career planning. All active staff receive mandatory annual performance appraisals.

114. The performance appraisal process includes assessing values and competencies, and being required to manage products to a high level of quality according to standards. For any quality-related issues identified by management, Human Resources helps to remedy the situation through coaching and mentoring, more frequent follow-ups, training, and other appropriate corrective actions.

115. During 2014, performance appraisals were completed for most active staff. Human resources monitors and follows up on performance appraisals to ensure that all active staff and all staff who received performance pay have completed their performance appraisals.

116. We reviewed documentation and carried out interviews on the following:

• recruitment (assessing staffing needs);
• use of specialist skills;
• assigning of professional personnel (reviewed the mandate of the ARPCM team);
• staff training and professional development; and
• performance management.

The Office encouraged the reporting of complaints and allegations about how it conducted its work

117. The Office encouraged the reporting of complaints and allegations about the conduct of its work.

118. This finding is important because it shows that the Office deals appropriately with complaints and allegations. The Office receives complaints and allegations

• if the work it performs fails to comply with professional standards and applicable legal and regulatory requirements, and
• if allegations of non-compliance with the system of quality control are made.

119. We made no recommendation in this area.

120. The Office’s policies—OAG Audit 1012 Audit Quality and OAG Audit 1091 Complaints and Allegations—meet the requirements of CSQC 1 for addressing complaints and allegations. The Office communicates these policies to all employees via the INTRAnet.

121. The Office receives external and internal complaints via a public inbox managed by the Communications team. Complaints are then tracked in a database, where they are addressed and investigated by the Auditor General or the Chair of the Audit Committee and whomever is appointed as investigator. Targeted response time on all issues is 90 days.

122. The Executive Committee receives a quarterly status report on all closed and outstanding complaints and allegations. In 2014, the Office received no complaints or allegations either internally or externally about how it conducted its audits in regard to the system of quality control.

123. We reviewed documentation and conducted interviews with Legal Services about complaints and allegations received during 2014 on work performed by the Office about failing to comply with professional standards and/or the Office’s system of quality control.

The Office communicated the results of the monitoring process

124. The Office communicated the results of the monitoring process.

125. This finding is important because it shows that the Office communicates the results of the monitoring process internally and externally.

126. We made no recommendation in this area.

127. The Office’s policy, OAG Audit 1012 Audit Quality, meets the requirements of CSQC 1 for communicating the results of the monitoring process. The Office communicates this policy to all employees via the INTRAnet.

128. The results of the monitoring process are published on the Office’s Internet site and include the reports

• Monitoring Report on the System of Quality Control—2013,
• Report on a Review of the Annual Audit Practice—Practice Reviews Conducted in the 2013–14 Fiscal Year, and
• Report on a Review of Direct Report Practitioners 2013–14.

129. The Office communicates the approval by the Executive committee of the Monitoring Report on the System of Quality Control to all staff.

130. We reviewed the publications of the monitoring reports on the Office’s Internet and the corporate messages to all staff showing the approval by the Executive Committee of the Monitoring Report on the System of Quality Control.

The Office addressed cases in which the results of monitoring procedures showed that reports may have been inappropriate or that procedures were omitted during the performance of the engagement

131. The Office complied with the requirements to address cases in which the results of the monitoring procedures showed that a report may have been inappropriate or that procedures were omitted during the performance of the engagement.

132. This finding is important because it shows that the Office deals appropriately with cases in which the results of monitoring procedures indicate that reports may be inappropriate or that procedures were omitted during the performance of the engagement.

133. We made no recommendation in this area.

134. The Office’s Protocol for Practice Reviews and Internal Audits defines the process in cases where the results of the monitoring procedures show that a report may be inappropriate or that procedures were omitted during the performance of the engagement.

135. We reviewed the Protocol for Practice Reviews and Internal Audits and the Practice Review Programs.

The Office monitored changes to standards and updated methodology in a timely fashion

136. The Office monitored changes to standards and updated methodology in a timely fashion.

137. This finding is important because it shows that the Office has a process in place to maintain the System of Quality Control and ensure that the SoQC is up to date and reflects current standards and CSQC 1 requirements.

138. We made no recommendation in this area.

139. The Office’s Process for maintaining the System of Quality Control and the Audit Methodology and ensuring it is up to date meets the requirements of CSQC 1.

140. We reviewed the Monitoring of Emerging Standards Process and the Annual deployment of changes to standards into methodology.

Office specialists were consulted when required

141. Audit teams consulted the various Office specialists and documented the extent of their consultations as required by the SoQC.

142. This finding is important because it shows that the Office has a process in place to enable specialist consultations when conducting audits and preparing appropriate audit reports.

143. We made no recommendation in this area.

144. The Office’s policy, OAG Audit 3081 Consultations, defines the importance of consultations within the conduct of audits so that it can reduce the risk of error and improve how professional judgment is applied. The Policy also defines the process for consultations and the requirements for documenting the consultations.

145. Audit teams consulted the specialists when dealing with complex, unusual, or unfamiliar issues.

146. We reviewed consultation data and details from audit teams of various specialists.

Operational effectiveness of the system of quality control at the engagement level

147. Overall, we found that the system of quality control is operating effectively at the engagement level. In the practice reviews of all engagements, the auditor’s reports were supported and appropriate.

148. A system of quality control consists of policies designed to achieve the objectives of the Canadian Standard on Quality Control (CSQC 1), as well as the procedures necessary to carry out those policies. It is important to monitor this system at the engagement level to determine if there are any deficiencies and to recommend appropriate remedial action. Operational effectiveness at the engagement level entails examining whether the Office of the Auditor General is carrying out its responsibilities by assessing whether its assurance engagements comply with Office policies and standards.

149. Monitoring the operational effectiveness of the system of quality control at the engagement level is performed at the Office by the Practice Review and Internal Audit team.

Engagements complied with professional standards for quality control

150. The Practice Review and Internal Audit team made recommendations about engagement performance and presented one finding of non-compliance on one specific element of the SoQC in the Direct Engagement practice.

151. Overall, we found that the deficiencies noted by the Practice Review and Internal Audit team did not represent systemic, repetitive, or other significant deficiencies requiring prompt corrective action.

152. This finding is important because the Office ensures the quality of its audit work by following professional auditing standards. Assessing compliance with professional standards helps the Office determine whether it is carrying out its responsibilities. It also contributes to continuous improvement by creating an opportunity for audit teams and the Office to learn from experience.

153. You can find our recommendations from the monitoring of the operational effectiveness of the system of quality control at the engagement level in the following reports on our Internet site:

• Report on a Review of the Direct Report Audit Practice—Direct Report Audits Completed in the 2014–15 Fiscal Year, and
• Report on a Review of the Annual Audit Practice—Annual Audits Completed in the 2014–15 Fiscal Year.

154. We made no other recommendation in this area.

155. Area reviewed: Supervision and review. Ensuring that the Office’s assurance engagements are completed to the highest quality requires team members to be adequately supervised, and audit work and documentation to be reviewed. Supervision is important to ensure that engagement teams are organized and that the quality of the work produced during the engagement is monitored for quality. Review is important to ensure that

• the work has been performed according to professional standards,
• the work supports the conclusions reached,
• the evidence obtained is sufficient and appropriate, and
• the objectives of the engagement procedures have been achieved.

156. Area reviewed: Engagement quality control review. Quality reviews objectively evaluate the significant judgments made by the engagement team and the conclusions reached in formulating the assurance engagement report. Quality reviewers are assigned to each annual audit of entities that issue or have securities outstanding in public markets. They are also assigned to other assurance engagements based on the assurance engagement’s level of risk. Quality reviewers have the technical qualifications to perform the role, as well as sufficient and appropriate experience and authority.

157. The Office has a process in place to select and appoint quality reviewers. The selection of quality reviewers is based on the level of risk associated with the engagement. The Professional Practices Group receives risk assessment input from each audit team and prepares a risk assessment for all engagements using selection criteria outlined in the methodology. It is normally recommended that engagements assessed as high risk be selected for a quality review. Low- to medium-risk audits are not normally assigned a quality reviewer.

158. For engagements selected for a quality review, a quality reviewer is then appointed based on specific criteria. The Professional Practices Group consults the assistant auditors general annually to review the list of audit engagements for their group and whether their senior personnel are available for quality review assignments. Once the Professional Practices Group compiles all risk assessments, the product leaders are consulted on the recommended quality reviewer selection and appointment.

159. Area reviewed: Differences of opinion. During the course of an assurance engagement, the team, those consulted about the assurance engagement, and the engagement leader and quality reviewer may have differences of opinion. Audit team members have the right to form their own conclusions on significant matters in the areas of the assurance engagement for which they are responsible, and ensure that their views receive adequate consideration. An assurance engagement report should not be dated until team members have resolved all differences of opinion.

160. Area reviewed: Engagement documentation. This component addresses the confidentiality, safe custody, integrity, accessibility, retrievability, and retention of engagement documentation and the completion of the final assembly of engagement files on time.

161. Area reviewed: Ethics and independence. This component addresses the audit staff independence requirement to the engagement.

162. The detailed observations from monitoring the operational effectiveness of the system of quality control at the engagement level can be found in the following reports on our Internet site:

• Report on a Review of the Direct Report Audit Practice—Direct Report Audits Completed in the 2014–15 Fiscal Year, and
• Report on a Review of the Annual Audit Practice—Annual Audits Completed in the 2014–15 Fiscal Year.

163. We conducted a detailed review of 2014–15 practice review files to determine whether they complied with the system of quality control at the engagement level. We reviewed six attest audit files and eight direct reporting engagement files in the following areas:

• supervision and review,
• engagement quality control review,
• differences of opinion,
• engagement documentation, and
• ethics and independence.

Engagement team competencies were assessed and documented

164. We found that engagement team competencies were being assessed and documented.

165. This finding is important because it is critical to ensure that the engagement team has the appropriate competence and capabilities to conduct the engagement. Engagement leaders need to document their assessments and strategies to ensure that

• they are assigning appropriate staff with the necessary competencies, and
• they have adequate resources and time available for the assurance engagement they are responsible for.

166. There are no recommendations in this area from either the Practice Review and Internal Audit team or from the monitoring of the operational effectiveness of the SoQC.

167. Area reviewed: Engagement team competencies. Before the planning/survey phase was completed, the engagement leader assessed the team to be satisfied that members, specialists, and others collectively had the appropriate competence and capabilities, and this leader documented the assessment.

168. We conducted a detailed review of 2014–15 practice review files (eight direct reporting engagements and six attest audits) to determine team competencies were assessed and whether the assessment was thorough enough.

Engagement teams undertook and documented appropriate consultation

169. Overall, we found that the engagement teams performed and documented appropriate consultation.

170. This finding is important because consultation is critical during the course of an assurance engagement to help promote quality and improve professional judgment, as well as to reduce the risk of error. A key aspect of an assurance engagement is the formal and informal consultation that takes place within audit teams, between audit teams, and between audit teams and the Office’s internal specialists or others.

171. There are no recommendations in this area from either the Practice Review and Internal Audit team or from the monitoring of the operational effectiveness of the SoQC.

172. Area reviewed: Consultation. Audit teams consult with internal and external specialists and senior Office staff when dealing with difficult or contentious matters or other matters requiring specialized knowledge or experience. Before the date of the assurance report, both the individual seeking consultation and the party consulted agree to the nature and scope of consultations, and the conclusions resulting from them. The conclusions resulting from consultations are then carried out.

173. We conducted a detailed review of 2014–15 practice review files to determine the nature of the consultations undertaken and whether they were sufficient.

Conclusion

174. Based on the work performed in the 2014 monitoring exercise, we concluded that there was reasonable assurance that the policies and procedures for the Office of the Auditor General’s system of quality control were relevant and adequate. We did not identify any serious deficiencies.

175. Based on the work performed in the 2014 monitoring exercise, we concluded that there was reasonable assurance that the Office of the Auditor General’s system of quality control was operating effectively at both the Office and engagement levels, so that reports issued by the Office were appropriate in the circumstances. We did not identify any serious deficiencies.

176. However, we noted one area for improvement related to the operational effectiveness of the system of quality control. Nevertheless, this weakness did not affect the adequacy or relevance of the system or the appropriateness of the audit reports issued. We noted the following weakness:

• The Office did not have a formal process or procedures in place to select and appoint internal specialists.

Looking Ahead

177. The Office’s Recruitment and Retention strategy is several years old, but remained valid for the period under review. In the fall of 2014, new roles and responsibilities were approved for senior management working in audit operations. In April 2015, the Office approved a new Bilingualism in the Workplace Strategy. Human Resources is currently consulting on a new proposed People Management Framework. Future monitoring reports will evaluate and conclude on the operating effectiveness of the key elements and components of this framework.