External Validation of Self-Assessment

External Validation of Self-Assessment—24 March 2018

Prepared by JB Lamothe Consulting incorporatedInc.

External Validation Report

1. Purpose of the report

This document presents the results of an external validation of a self-assessment that the Office of the Auditor General of Canada (the Office) conducted of its internal audit function. We at JB Lamothe Consulting Inc. conducted the external validation on behalf of the Office’s Practice Review and Internal Audit (PRIA) team. We assessed whether the internal audit function conformed with the Institute of Internal Auditors’ (IIA) International Standards for the Professional Practice of Internal Auditing (IIA Standards).

2. Background and context

The IIA’s International Professional Practices Framework (IPPF) requires that an external assessment of an organization’s internal audit function be conducted at least every five years by a qualified independent assessor or assessment team from outside the organization. An external assessment may be accomplished through a full external assessment or through a self-assessment with external validation.

The Office’s Chief Audit Executive (CAE), the head of the PRIA team, elected to conduct a self-assessment with external validation for the Office’s first external assessment. The CAE contracted the services of JB Lamothe Consulting Inc. to conduct an independent validation of the self-assessment of the internal audit function prepared by the CAE. The PRIA team completed the self-assessment in 2016 and updated it in early 2017.

3. Objective and Scope

The objective of this engagement was to conduct an independent external validation of the PRIA team’s self-assessment of the internal audit activity, in accordance with the International Standards for the Professional Practice of Internal Auditing, established by the IIA. We were to express an opinion on the degree to which the Office’s internal audit function conformed with the IIA Standards and Code of Ethics.

We conducted the external validation during the months of January through March 2018. The scope included audit engagements conducted since 2014, up to and including the 2017–18 fiscal year, as well as associated processes in place over that period.

4. Methodology

The PRIA team conducted the self-assessment using the methodology outlined in the IIA’s Quality Assessment Manual for the Internal Audit Activity (2013). Conformance to the IIA’s International Professional Practices Framework was evaluated using the following scale:

• Generally Conforms (GC): means there is no material deficiency, although there may be some minor deficiencies.
• Partially Conforms (PC): means there is one material deficiency and there may be some minor deficiencies.
• Does Not Conform (DNC): means that there is more than one major deficiency in practice that are judged to be so significant as to seriously impair or preclude the internal audit function from performing adequately in all or in significant areas of its responsibilities.

As part of the external validation, we

• Met with the Audit Committee and the Auditor General prior to the start of the engagement;
• Reviewed the self-assessment material prepared by the CAE;
• Reviewed documentation provided by the PRIA team, such as the PRIA Charter, the Audit Committee Charter, Risk-based Audit Plans, and Audit Committee records of decisions, etceteraetc.;
• Reviewed the PRIA team’s Internal Audit Manual and related audit processes;
• Reviewed the audit file and working papers of three (3) audit engagements conducted during the scope period:
  • Audit of Human Resources Planning;
  • Audit of Managing information technologyIT Security; and
  • Audit of Professional Development (Planning Phase only).
• Carried out interviews with the CAE and directors, as well as with two clients/auditees;
• Summarized observations and findings, and prepared recommendations related to opportunities for improvement identified;
• Prepared the report; and
• Presented the findings to the CAE, the Audit Committee, and the Auditor General.

5. Summary of Findings

The following table provides a summary of the assessment against each area reviewed.

6. Conclusion

Our overall opinion is that PRIA generally conforms with the requirements of the Institute of Internal Auditors’ International Professional Practices Framework. The rating of “generally conforms” is the highest rating an internal audit function can receive from an external assessment. We identified some minor opportunities for improvement during the course of our external validation. We provide these observations and recommendations in the next section.

7. Observations and recommendations

The CAE and small audit team did a good job of building a solid foundation to ensure that quality products were delivered to the Audit Committee and the Auditor General. The team developed policies, practices, and processes that were well documented in an internal audit manual. The team implemented the TeamMate audit management software to facilitate the planning, examination, and reporting of internal audit engagements. Engagements were well supervised by the CAE, who reviewed and approved all working papers in TeamMate.

We noted a couple of observations and recommendations during our external validation, which the PRIA team may wish to consider.

Observation 1: Quality Assurance and Improvement Program (Standard 1300)

IIA Standard 1300 states that a quality assurance and improvement program must include both internal and external assessments. Internal assessments must include ongoing monitoring of the performance of the internal audit activity as well as periodic self-assessments to evaluate conformance with the Code of Ethics and the Standards.

PRIA’s Quality Assurance and Improvement Program Policy states that “Periodic assessments are conducted on the entire internal audit activity or on specific areas of the internal audit activity to ensure conformance to the Code of Ethics and Standards.” PRIA’s Quality Assurance and Improvement Program further states that periodic assessments will be conducted through the following activities:

• Periodic customer surveys;
• Periodic self-assessment of specific internal audit activities to ensure conformance with the Standards;
• Peer review of completed internal audit engagement for performance in accordance with internal audit policies and with the Standards;
• As required, a self-assessment of the internal audit activities to ensure conformance with the Standards;
• Review of the internal audit performance metric(s) and benchmarking of best practices; and
• Periodic activity and performance reporting to the Auditor General and the Audit Committee.

PRIA currently conducts peer reviews of the working papers of its’ internal audit engagements at the completion of the engagement and after the CAE’s review. We suggest that PRIA considers conducting these reviews after each phase of the engagement, before the CAE’s review and before the audit team proceeds to the next phase of the engagement. This would allow the team to address any resulting changes or issues on a timelier basis, and as a result, add more value to this quality assurance activity.

We noted that many elements of the PRIA team’s Quality Assurance and Improvement ProgramQAIP had been developed and were in place. However, some elements still needed to be developed.

Other than the self-assessment conducted for this external validation and periodic reviews of internal audit engagement standards, there was no evidence that the team had performed any other periodic self-assessments of non-engagement-related standards during the scope period. As stated in PRIA’s QAIP, periodic self-assessments of all internal audit activities are to be conducted to evaluate conformance with the Standards. These self-assessments should be conducted using the review programs included in the IIA’s 2017 Quality Assessment Manual.

The team had also not fully implemented periodic customer surveys or internal audit performance metrics had also not been fully implemented as of the date of this external validation.

Recommendation:

The PRIA team should continue its efforts to implement and report on

• periodic self-assessments of non-engagement activities, which include the assessment of whether activities conform with the PRIA Charter, the IIA’s Code of Ethics, and all the IIA Standards; and
• performance metrics and the recently developed client satisfaction survey.

Observation 2: Engagement Planning (Standard 2200)

Standard 2210.A1 states that “Internal Auditors must conduct a preliminary assessment of the risks relevant to the activity under review. Engagement objectives must reflect the results of this assessment.”

The review of the three engagement working paper files showed that risk assessments had been completed and documented to various degrees. The more recent audit engagements showed increasingly better documentation of the risk assessment conducted. For the files reviewed, auditors used various methodologies and templates for developing and documenting the risk assessments.

Good practices noted in other government departments and agencies include the creation of a standard methodology and template for conducting and documenting the risk assessment. The template may include a description of the key risks, some of the context and background learned during the planning phase, a likelihood and impact rating, expected or perceived controls, fraud considerations, residual risk levels, and the decision on whether to include the assessment of the risk in the audit program, along with justification for the decision.

Recommendation:

To help focus limited internal audit resources on the higher risk areas of the audit entity, the PRIA team should develop a standardized methodology and template for engagement-level risk assessments, and ensure that it applies and documents them consistently in all internal audit engagements.

Independent Validation Statement

The Office of the Auditor General of Canada (the Office) engaged us, JB Lamothe Consulting Inc., to conduct an independent validation of the Office’s self-assessment of the internal audit function. The Office’s Practice Review and Internal Audit (PRIA) team prepared the self-assessment. The primary objective of the engagement was to conduct an independent external validation of the self-assessment of the internal audit function and assess conformance with the Institute of Internal Auditors’ International Professional Practices Framework, which includes the IIA Standards and Code of Ethics.

In acting as validator, we are fully independent of the Office and have the necessary knowledge and skills to undertake this engagement. The validation, conducted during the months of January through March 2018, consisted primarily of a review and validation of the self-assessment material provided by the PRIA team; a review of additional documentation provided by the team; and a review of the audit file and working papers of three audit engagements conducted during the scope period. In addition, we held interviews with the Office’s Chief Audit Executive and directors, as well as with two clients (auditees).

We concur with the Chief Audit Executive’s overall assessment. On the basis of our review, the PRIA team generally conforms with the requirements of the mandatory requirements of the International Professional Practices Framework of the IIA. We suggested some minor opportunities for improvement and included them in the final report.

Independent Validator
JB Lamothe Consulting Inc.