2018 Fall Reports of the Auditor General of Canada to the Parliament of Canada Independent Auditor’s ReportReport of the Auditor General of Canada to the Board of the Canada Council for the Arts—Special Examination—2018
2018 Fall Reports of the Auditor General of Canada to the Parliament of CanadaReport of the Auditor General of Canada to the Board of the Canada Council for the Arts—Special Examination—2018
Independent Auditor’s Report
This report reproduces the special examination report that the Office of the Auditor General of Canada issued to Canada Council for the Arts on 4 May 2018. The Office has not performed follow-up audit work on the matters raised in this reproduced report.
Introduction
Background
1. The Canada Council for the Arts is a federal Crown corporation established under the Canada Council for the Arts Act. It reports to Parliament through the Minister of Canadian Heritage. The Corporation fosters and promotes the study and enjoyment of the arts, and the production of works of art. The Corporation offers a range of grants, prizes, and services to Canadian artists and arts organizations, including those involved in music, theatre, writing and publishing, visual arts, dance, and media arts.
2. The Corporation’s main source of funding is the Government of Canada, with $222.6 million in appropriations in the 2016–17 fiscal year. Other funding sources include investment income and rental of art. The Corporation achieves its objectives mainly through grant awards to artists and arts organizations, which in the 2016–17 fiscal year totalled $185.0 million.
3. In the 2017–18 fiscal year, the Corporation launched new grant programs as part of a significant transformation, which involved a smaller number of programs that had broader focus, each including many disciplines. This transformation (further detailed in paragraph 28) involved restructuring work units, hiring staff, and designing new tools, including information technology tools, such as an online portal for grant applications.
4. This transformation coincides with a doubling of the Corporation’s annual parliamentary appropriations, as announced in the 2016 Budget, from approximately $182 million in the 2015–16 fiscal year to $362 million by 2020–21.
Focus of the audit
5. Our objective for this audit was to determine whether the systems and practices we selected for examination at the Canada Council for the Arts were providing it with reasonable assurance that its assets were safeguarded and controlled, its resources were managed economically and efficiently, and its operations were carried out effectively as required by section 138 of the Financial Administration Act.
6. In addition, section 139 of the Financial Administration Act requires that we state an opinion, with respect to the criteria established, whether there was reasonable assurance there were no significant deficiencies in the systems and practices examined. A significant deficiency is reported when the systems and practices examined did not meet the criteria established, resulting in a finding that the Corporation could be prevented from having reasonable assurance that its assets are safeguarded and controlled, its resources are managed economically and efficiently, and its operations are carried out effectively.
7. Based on our assessment of risks, we selected systems and practices in the following areas:
- corporate management practices, and
- management of grant programs for the arts.
The selected systems and practices and the criteria used to assess them are found in the exhibits throughout the report.
8. More details about the audit objective, scope, approach, and sources of criteria are in About the Audit at the end of this report.
Findings, Recommendations, and Responses
Corporate management practices
The Corporation had good corporate management practices, with weaknesses in risk mitigation
Overall message
9. Overall, we found that the Corporation had good systems and practices in corporate management. However, there were weaknesses in the area of risk mitigation. For example, the Corporation had identified weaknesses in how its information technology function supported operations, such as the lack of a project management framework for developing new tools. Though the Corporation had begun taking corrective actions, it had yet to create a detailed plan to address the weaknesses. We also found that the Corporation did not have an updated and comprehensive business continuity plan.
10. This finding matters because the Corporation relies on having sound corporate management practices to operate efficiently and effectively. Good information technology practices will remain essential to reducing risks, especially as the Corporation implements tools that are to help it administer its new grant programs efficiently and support its overall business strategy associated with the increase in federal funding over the next few years. Furthermore, as no organization is immune to major interruptions, such as power outages or critical system failures, it is important for the Corporation to address identified weaknesses in its business continuity plan.
11. Our analysis supporting this finding discusses the following topics:
- Corporate governance
- Strategic planning and performance measurement, monitoring, and reporting
- Risk management
12. The Board is responsible for the overall management of the Corporation and for the achievement of its mandate. The Board has up to 11 members, who are appointed by the Governor in CouncilDefinition i. The Board is supported by an Audit and Finance Committee, a Governance and Nominating Committee, and an Investment Committee.
13. Our recommendations in this area of examination appear at paragraphs 22 and 23.
14. Corporate governance. We found that the Corporation had good systems and practices in corporate governance (Exhibit 1).
Exhibit 1—Corporate governance—key findings and assessment
Systems and practices | Criteria used | Key findings | Assessment against the criteria |
---|---|---|---|
Legend—Assessment against the criteria Met the criteria Met the criteria, with improvement needed Did not meet the criteria |
|||
Board independence |
The Board functioned independently. |
|
|
Providing strategic direction |
The Board provided strategic direction. |
|
|
Board oversight |
The Board carried out its oversight role over the Corporation. |
|
|
Board appointments and competencies |
The Board collectively had capacity and competencies to fulfill its responsibilities. |
|
15. Strategic planning and performance measurement, monitoring, and reporting. We found that the Corporation had good systems and practices for strategic planning and performance measurement, monitoring, and reporting (Exhibit 2).
Exhibit 2—Strategic planning and performance measurement, monitoring, and reporting—key findings and assessment
16. Risk management. We found that the Corporation had good systems and practices for risk identification, assessment, monitoring, and reporting. However, we found weaknesses in risk mitigation (Exhibit 3).
Exhibit 3—Risk management—key findings and assessment
17. Weaknesses—Risk mitigation. We found that, although the Corporation had identified risks related to how its information technology function supported operations, it had yet to develop and implement concrete plans for mitigating those risks. Specifically, in 2016, the Corporation carried out an assessment of its information technology function. The assessment found weaknesses that had to be addressed so that information technology could better support the Corporation in achieving its strategic objectives.
18. One of the short-term recommendations for addressing the identified weaknesses was to develop a transformation strategy for the information technology function, with a detailed and executable roadmap linked to the Corporation’s business strategy. The Corporation started developing this strategy, but progress was constrained by the need to focus on information technology projects in support of the Corporation’s new granting programs. As such, an action plan to implement the information technology that would support all of the Corporation’s operations remained undefined.
19. The 2016 assessment had also identified weaknesses in project management for information technology such as lacking a project management framework. We found that while the Corporation had begun taking steps, it had not yet fully addressed these weaknesses.
20. We also found that the Corporation lacked an updated and comprehensive business continuity plan. The purpose of such a plan is to allow an organization to continue its critical operations in the event of a serious, unexpected interruption. In the 2015–16 fiscal year, the Corporation conducted a security assessment that identified weaknesses in its business continuity plan. The Corporation’s risk mitigation plans had targeted completion of a new business continuity plan by June 2017, but we found that this had not yet been done.
21. These weaknesses matter because the Corporation relies on information technology to operate efficiently and effectively. If the Corporation does not develop and implement a comprehensive information technology plan, its investments in this area might not meet its future needs. Furthermore, a project management framework is necessary to delivering future information technology initiatives successfully. Finally, without an updated and comprehensive business continuity plan, the Corporation was not as prepared as it could be to continue its critical operations in the event of a serious, unexpected interruption.
22. Recommendation. The Corporation should accelerate the development and implementation of its information technology plan, including adopting best practices in information technology project management.
The Corporation’s response. Agreed. The Corporation is accelerating the development and implementation of its information technology and information management (IT/IM) plans and has developed a three-year roadmap for IT/IM, under the responsibility of the Chief Information Officer (CIO), a position newly created in February 2018. The CIO is also responsible for implementing rigorous project management practices and an adequate structure to advance IT projects, beginning in the first quarter of the 2018–19 fiscal year. The Corporation is also making an additional investment to advance on its plans related to IT.
23. Recommendation. The Corporation should update its business continuity plan in a manner that addresses the weaknesses it has identified.
The Corporation’s response. Agreed. The work on the business continuity plan is under way and the Corporation will complete it before the end of June 2018.
Management of grant programs for the arts
The Corporation had good management practices in its granting operations, and opportunities to learn from weaknesses in its legacy grant programs
Overall message
24. Overall, we found that the Corporation had good systems and practices in managing its grant programs, despite some weaknesses in the processes for selection of recipients in its legacy grant programs. For example, some requirements in the application forms were not as specific as those given in the guidelines for the related programs, generic checklists for Corporation employees to assess application criteria did not reflect specific programs, and there were instances where employees involved with awarding grants had failed to complete mandatory conflict of interest forms. Moving forward, the Corporation should ensure that weaknesses noted in its legacy programs are not repeated in its new programs. We also found that the Corporation had identified, through monitoring and employee feedback, that problems had emerged in the implementation of its new programs and was in the process of resolving them.
25. This finding matters because well-designed, consistent, and rigorous assessment processes for grant applications, along with compliance with corporate requirements for managing conflict of interest, help the Corporation demonstrate that it awards grants fairly, on the basis of merit, and in a manner that meets program objectives. If unresolved, the issues with the implementation of the new programs could cause delays in granting funds to applicants, and affect the morale of employees who are adapting to a new work environment and processes.
26. Our analysis supporting this finding discusses the following topic:
27. The Corporation devotes most of its resources to supporting artists and arts organizations through its grant programs. Under these programs, artists and arts organizations compete for grants designed to achieve diverse objectives. Typically, the Corporation’s employees first review applications to ensure that applicant and project eligibility criteria and other information requirements are met. Eligible applications are then assessed by internal committees or by external committees of peers, who assess submissions according to the guidelines provided by the Corporation, and rank applicants by merit. Most of the funding is provided as a result of the work by external peer committees. While the peer committees may recommend the amounts of grants to be awarded to each successful candidate, the Corporation makes the final decisions.
28. The calendar year 2017 was pivotal in the Corporation’s history. The Corporation had been planning for years to transform its grant programs, a process that led to significant changes:
- The number of programs was reduced from over 140 to 6 regular programs with 30 components, and 4 targeted programs called strategic funds. In contrast to the old programs, the new programs do not focus on specific artistic disciplines, such as dance or film. Instead, they are grouped around themes, such as the creation and dissemination of innovative, vibrant, and diverse art; support for Indigenous artists and art organizations; and the promotion of Canadian arts abroad.
- The Corporation developed new policies and operational guidelines, and introduced new tools for officials and grant applicants.
- Staff were reorganized into new teams, and additional staff were hired; the Corporation developed courses and delivered training to grant program staff.
The implementation of the transformation began in December 2016, with the launch of an online portal for clients to register their profiles. The formal launch of the new programs, and the associated funding, began in the 2017–18 fiscal year. After the launch, the Corporation encountered numerous implementation problems that affected both program delivery and employee morale. By the end of our examination, senior management had placed a priority on resolving them.
29. Our recommendations in this area of examination appear at paragraphs 35 and 36.
30. Management of grant programs. We found that the Corporation had good systems and practices for operational planning and monitoring. In its legacy grant programs, it had good systems and practices for determining grant amounts and making payments, but we found weaknesses in the processes for selection of grant recipients (Exhibit 4).
Exhibit 4—Management of grant programs—key findings and assessment
31. Weaknesses—Selecting eligible grant recipients. We reviewed a sample of grant applications and grant competitions conducted under the Corporation’s legacy grant programs, which were in place until 31 March 2017. We found weaknesses with some aspects of the design and use of tools in the selection process:
- Application forms in several of the competitions did not ask for some details that could have facilitated assessment of the applicants’ eligibility against program guidelines. For example, eligibility criteria of one program required that for arts organizations, “if they are governed by a board of directors, at least 51% of the directors must be Aboriginal people.” The application form did not specifically ask about board representation, but employees told us that this requirement could still be assessed through employees’ or committee members’ familiarity with the applicant.
- For all programs, Corporation employees used a generic checklist to document their assessment of application requirements, including eligibility criteria, even though some of the requirements could be different for each program. Because a single line in the checklist could represent several requirements, the checklists could not clearly show that all the requirements had been properly assessed. We found instances where a requirement was not met and there was no explanation in the checklists as to how it had been analyzed. Nevertheless, the Corporation’s guidelines did permit program officers to allow a peer assessment committee to consider an application if the officers were unsure whether an applicant was eligible. During the period covered by the audit, most of the applications reached the committee assessment phase.
- Both internal and external peer committees used scoresheets to rank applications, but use of the scoresheets was inconsistent across programs and competitions. For example, while one program used a scoresheet to assess each criterion articulated in the program guidelines, another program with similar criteria required an overall score without breakdown by criterion. Regardless of the specific scoring tool used, deliberation by committee members led to their agreement on the final ranking of applications.
32. These weaknesses matter because well-designed, consistent, and rigorous grant application assessment tools and processes help the Corporation demonstrate that it is awarding grants fairly, on the basis of merit, and in a manner that meets program objectives.
33. In our sample of grant competitions assessed through internal committees, we also found weaknesses in the management of conflict of interest for the Corporation’s employees involved in selecting recipients. During the 2016–17 fiscal year, the Corporation strengthened its conflict of interest practices by requiring employees involved in assessing grant applications to sign a conflict of interest form for each competition they participated in. However, for two of the eight selected internal competitions, the forms were incomplete. Furthermore, the Corporation required all employees to annually sign conflict of interest forms. For our sample of competitions, the Corporation did not have most of these employees’ annual forms on file.
34. This weakness matters because even apparent or potential conflicts of interest may bring into question the integrity and fairness of the granting process. Documenting conflicts of interest as required by the Corporation’s policies and procedures helps demonstrate and ensure that program staff are aware of and respect the expected conduct when faced with situations where conflicts of interest may exist.
35. Recommendation. The Corporation should evaluate whether the weaknesses in its legacy programs are relevant to the new programs. As applicable, the Corporation should ensure that
- information requested in its grant application forms aligns with the eligibility criteria and other requirements set out in its program guidelines,
- employees clearly document whether grant applications have met requirements, and
- scoresheets are consistently used and are consistent with the assessment criteria set out in program guidelines.
The Corporation’s response. Agreed. The weaknesses identified in the legacy programs are being addressed in the new funding model. Under the leadership of the Director General of the Arts Granting Programs Division, the Granting Program Operations section will be playing a key role in providing centralized support to the new programs in order to ensure operational integrity and coherence. This includes responsibility for the application of the operational policies, training and orientation on granting systems and procedures, alignment of program guidelines and application forms, documentation to ensure that grant application requirements are met, and the consistent use of scoresheets. The Corporation will also reinstate the practice of regular verification of grant application files, beginning in April 2018. The Corporation will also continue to develop new functionality in its online portal, including plans to clearly communicate the rationale for ineligibility to applicants via the portal and to create an integrated, digital scoring tool.
36. Recommendation. The Corporation should ensure that employees follow its conflict of interest requirements.
The Corporation’s response. Agreed. The Corporation will ensure that conflict of interest forms are completed by employees annually. Management will also ensure that conflicts of interest are consistently recorded for program officers involved in the internal assessment of grant applications.
Conclusion
37. In our opinion, based on the criteria established, there was reasonable assurance that there were no significant deficiencies in the Corporation’s systems and practices that we examined. We concluded that the Canada Council for the Arts maintained its systems and practices during the period covered by the audit in a manner that provided the reasonable assurance required under section 138 of the Financial Administration Act.
About the Audit
This independent assurance report was prepared by the Office of the Auditor General of Canada on the Canada Council for the Arts. Our responsibility was to express
- an opinion on whether there is reasonable assurance that during the period covered by the audit, there were no significant deficiencies in the Corporation’s systems and practices that we selected for examination; and
- a conclusion about whether the Corporation complied in all significant respects with the applicable criteria.
Under section 131 of the Financial Administration Act (FAA), the Canada Council for the Arts is required to maintain financial and management control and information systems and management practices that provide reasonable assurance that
- its assets are safeguarded and controlled;
- its financial, human, and physical resources are managed economically and efficiently; and
- its operations are carried out effectively.
In addition, section 138 of the FAA requires the Corporation to have a special examination of these systems and practices carried out at least once every 10 years.
All work in this audit was performed to a reasonable level of assurance in accordance with the Canadian Standard for Assurance Engagements (CSAE) 3001—Direct Engagements set out by the Chartered Professional Accountants of Canada (CPA Canada) in the CPA Canada Handbook—Assurance.
The Office applies Canadian Standard on Quality Control 1 and, accordingly, maintains a comprehensive system of quality control, including documented policies and procedures regarding compliance with ethical requirements, professional standards, and applicable legal and regulatory requirements.
In conducting the audit work, we have complied with the independence and other ethical requirements of the relevant rules of professional conduct applicable to the practice of public accounting in Canada, which are founded on fundamental principles of integrity, objectivity, professional competence and due care, confidentiality, and professional behaviour.
In accordance with our regular audit process, we obtained the following from the Corporation’s management:
- confirmation of management’s responsibility for the subject under audit;
- acknowledgement of the suitability of the criteria used in the audit;
- confirmation that all known information that has been requested, or that could affect the findings or audit conclusion, has been provided; and
- confirmation that the audit report is factually accurate.
Audit objective
The objective of this audit was to determine whether the systems and practices we selected for examination at the Canada Council for the Arts were providing it with reasonable assurance that its assets were safeguarded and controlled, its resources were managed economically and efficiently, and its operations were carried out effectively as required by section 138 of the Financial Administration Act.
Scope and approach
Our audit work examined the Canada Council for the Arts. The scope of the special examination was based on our assessment of the risks the Corporation faced that could affect its ability to meet the requirements set out by the Financial Administration Act.
In performing our work, we reviewed key documents related to the systems and practices selected for examination. We interviewed Board members, senior management, and other employees of the Corporation. We tested the systems and practices in place to obtain the required level of audit assurance.
The systems and practices selected for examination for each area of the audit are found in the exhibits throughout the report.
Our audit work on the Corporation’s operational planning and monitoring of its grant programs focused primarily on activities associated with the Corporation’s launch of its new programs in the 2017–18 fiscal year. In examining the selection of eligible grant recipients and the determination of grant amounts, our audit work included sampling approaches to examine the delivery of the Corporation’s legacy programs, which ended on 31 March 2017. Testing of grants under the new programs was not performed, due to insufficient amount of time available to perform audit work on appropriate competitions, given the competitions’ expected end dates. Our sampling covered grant competitions conducted between 31 October 2016 and 31 March 2017; together, these programs awarded $34 million in grants. We sampled primarily on a random basis from data that the Corporation extracted from its Arts Tracking System. We selected
- 18 of 62 grant competitions, to examine the linkage between eligibility criteria and program objectives;
- 24 of 3,743 applications, to examine how employees assessed them and documented their eligibility assessments;
- 20 of 93 committee assessments, to examine some aspects of committees’ roles; and
- 20 of 1,078 grant awards, to examine whether appropriate amounts had been paid to chosen recipients.
In carrying out the special examination, we did not rely on any internal audits.
Sources of criteria
The criteria used to assess the systems and practices selected for examination are found in the exhibits throughout the report.
Corporate governance
Meeting the Expectations of Canadians: Review of the Governance Framework for Canada’s Crown Corporations, Treasury Board Secretariat, 2005
Internal Control—Integrated Framework, Committee of Sponsoring Organizations of the Treadway Commission, 2013
Corporate Governance in Crown Corporations and Other Public Enterprises—Guidelines, Department of Finance and Treasury Board, 1996
20 Questions Directors Should Ask about Risk, Canadian Institute of Chartered Accountants, 2006
Performance Management Program for Chief Executive Officers of Crown Corporations—Guidelines, Privy Council Office, 2016
Practice Guide: Assessing Organizational Governance in the Public Sector, The Institute of Internal Auditors, 2014
Strategic planning and performance measurement, monitoring, and reporting
20 Questions Directors Should Ask about Risk, Canadian Institute of Chartered Accountants, 2006
20 Questions Directors Should Ask about Strategy, Canadian Institute of Chartered Accountants, 2012
Meeting the Expectations of Canadians: Review of the Governance Framework for Canada’s Crown Corporations, Treasury Board Secretariat, 2005
Guidelines for the Preparation of Corporate Plans, Treasury Board Secretariat, 1996
Corporate Governance in Crown Corporations and Other Public Enterprises—Guidelines, Department of Finance and Treasury Board, 1996
Recommended Practice Guideline 3, Reporting Service Performance Information, International Public Sector Accounting Standards Board, 2015
Risk management
20 Questions Directors Should Ask about Risk, Canadian Institute of Chartered Accountants, 2006
Internal Control—Integrated Framework, Committee of Sponsoring Organizations of the Treadway Commission, 2013
Corporate Governance in Crown Corporations and Other Public Enterprises—Guidelines, Department of Finance and Treasury Board, 1996
Management of grant programs
Guidelines for the Preparation of Corporate Plans, Treasury Board Secretariat, 1996
What Boards Should Expect from chief financial officersCFOs, Canadian Institute of Chartered Accountants, 2003
Internal Control—Integrated Framework, Committee of Sponsoring Organizations of the Treadway Commission, 2013
Plan-Do-Check-Act management model adapted from the Deming Cycle
Policy on Transfer Payments, Treasury Board, 2008
Directive on Transfer Payments, Treasury Board, 2008
Framework for Identifying Risk in Grant and Contribution Programs, Office of the Auditor General of Canada in collaboration with Industry Canada, 2000
Period covered by the audit
The special examination covered the period between 1 November 2016 and 31 October 2017. This is the period to which the audit conclusion applies. However, to gain a more complete understanding of the significant systems and practices, we also examined certain matters that preceded the starting date of this period.
Date of the report
We obtained sufficient and appropriate audit evidence on which to base our conclusion on 27 March 2018, in Ottawa, Canada.
Audit team
Principal: Dusan Duvnjak
Lead Director: Daniel Thompson
Director: Laurie Girard
Françoise Bessette
Meaghan Burnham
David Vaillancourt
List of Recommendations
The following table lists the recommendations and responses found in this report. The paragraph number preceding the recommendation indicates the location of the recommendation in the report, and the numbers in parentheses indicate the location of the related discussion.
Corporate management practices
Recommendation | Response |
---|---|
22. The Corporation should accelerate the development and implementation of its information technology plan, including adopting best practices in information technology project management. (16 to 21) |
The Corporation’s response. Agreed. The Corporation is accelerating the development and implementation of its information technology and information management (IT/IM) plans and has developed a three-year roadmap for IT/IM, under the responsibility of the Chief Information Officer (CIO), a position newly created in February 2018. The CIO is also responsible for implementing rigorous project management practices and an adequate structure to advance IT projects, beginning in the first quarter of the 2018–19 fiscal year. The Corporation is also making an additional investment to advance on its plans related to IT. |
23. The Corporation should update its business continuity plan in a manner that addresses the weaknesses it has identified. (20 and 21) |
The Corporation’s response. Agreed. The work on the business continuity plan is under way and the Corporation will complete it before the end of June 2018. |
Management of grant programs for the arts
Recommendation | Response |
---|---|
35. The Corporation should evaluate whether the weaknesses in its legacy programs are relevant to the new programs. As applicable, the Corporation should ensure that
|
The Corporation’s response. Agreed. The weaknesses identified in the legacy programs are being addressed in the new funding model. Under the leadership of the Director General of the Arts Granting Programs Division, the Granting Program Operations section will be playing a key role in providing centralized support to the new programs in order to ensure operational integrity and coherence. This includes responsibility for the application of the operational policies, training and orientation on granting systems and procedures, alignment of program guidelines and application forms, documentation to ensure that grant application requirements are met, and the consistent use of scoresheets. The Corporation will also reinstate the practice of regular verification of grant application files, beginning in April 2018. The Corporation will also continue to develop new functionality in its online portal, including plans to clearly communicate the rationale for ineligibility to applicants via the portal and to create an integrated, digital scoring tool. |
36. The Corporation should ensure that employees follow its conflict of interest requirements. (33 and 34) |
The Corporation’s response. Agreed. The Corporation will ensure that conflict of interest forms are completed by employees annually. Management will also ensure that conflicts of interest are consistently recorded for program officers involved in the internal assessment of grant applications. |