2017 Fall Reports of the Auditor General of Canada to the Parliament of Canada Independent Auditor’s ReportReport 3—Settlement Services for Syrian Refugees—Immigration, Refugees and Citizenship Canada

2017 Fall Reports of the Auditor General of Canada to the Parliament of CanadaReport 3—Settlement Services for Syrian Refugees—Immigration, Refugees and Citizenship Canada

Independent Auditor’s Report

Introduction

Background

3.1 In November 2015, the Government of Canada committed to bringing 25,000 Syrian refugees to Canada by February 2016. Subsequently, it committed to bringing about 10,000 additional Syrian refugees to Canada by the end of December 2016.

3.2 The federal government also committed to processing—by early 2017—approximately 12,000 privately sponsored Syrian refugee applications that had been submitted by 31 March 2016 to Immigration, Refugees and Citizenship Canada (the Department).

3.3 The Syrian refugees arrived in Canada under three streams:

3.4 As of 30 April 2017, the Department reported that 44,919 Syrian refugees had arrived in Canada through the three streams since November 2015. This number is unprecedented in recent history. Each year since 1995, the Government of Canada has admitted about 11,000 refugees—excluding the recent Syrian refugees.

3.5 The Immigration and Refugee Protection Act recognizes that the integration of newcomers into Canada is the responsibility of both Canadian society and the newcomers themselves. However, the Act gives the Department express responsibility for promoting the successful integration of permanent residents into Canada. The goal of integration is that newcomers contribute to the country’s economic, social, political, and cultural development.

3.6 The federal government’s responsibility for refugees who settle in Quebec differs from its responsibility for those who settle elsewhere in Canada. Under the Canada–Québec Accord relating to Immigration and Temporary Admission of Aliens, the Province of Quebec is solely responsible for providing services to those who settle in that province, and it has an agreement with the federal government to fund those services. Of the almost 45,000 Syrian refugees who came to Canada, about 10,000 settled in Quebec.

3.7 The federal government granted permanent resident status to all Syrian refugees who arrived in Canada, which allowed them to access services funded through the Department’s Settlement Program. These services included needs assessments and referrals, language training, employment-related services, and other support services, such as child-care and transportation assistance.

3.8 The Settlement Program is available to all permanent residents outside Quebec, not just refugees. Its purpose is to help newcomers integrate into Canadian society. Before the Syrian refugee initiative, the Department’s annual budget for the Settlement Program was about $600 million.

3.9 In 2015, the federal government approved the allocation of approximately $760 million of supplementary funds, over four years, for the Syrian refugee initiative. The supplementary funds were for selecting and transporting the refugees to Canada, supporting the Department’s Resettlement Assistance Program, and expanding settlement services for refugees who settled outside Quebec.

3.10 About $141 million of the supplementary funds were to be allocated over four years to expand the Settlement Program to help the Syrian refugees integrate into Canada. In November 2016, the Department increased the Settlement Program supplement to $257 million over five years.

Focus of the audit

3.11 This audit focused on whether Syrian refugees received selected settlement services funded by Immigration, Refugees and Citizenship Canada that were needed to help them integrate into Canada. It also examined whether the Department measured the outcomes of its efforts to settle Syrian refugees.

3.12 The settlement services we selected were needs assessments, language assessments, and language training.

3.13 The audit covered Settlement Program services, funded by the Department, for Syrian refugees who arrived in Canada between 4 November 2015 and 31 December 2016 under the Government of Canada’s Syrian refugee initiative.

3.14 This audit is important because the Syrian refugee initiative will succeed in the long term only if the people it brought to Canada integrate into Canadian society.

3.15 We did not examine

3.16 More details about the audit objective, scope, approach, and criteria are in About the Audit at the end of this report.

Findings, Recommendations, and Responses

Delivering settlement services

Overall message

3.17 Overall, we found that most Syrian refugees received needs assessments, language assessments, and language training during their first year in Canada. More than 80 percent had their needs assessed, and 75 percent of those who received language assessments attended language classes. Syrian refugees received settlement services at a higher rate than other refugees who arrived in Canada during the same period.

3.18 We also found that Immigration, Refugees and Citizenship Canada did not allocate all of its available funds to service providers at the beginning of the 2017–18 fiscal year. The purpose of these funds was to sustain additional settlement services established in 2016. When they did not receive funding, some service providers cut settlement services for at least three months.

3.19 These findings matter because for Syrian refugees to integrate into Canadian society, they must be able to access the settlement services they need when they need them.

3.20 The Department uses contribution agreements with about 500 service providers to fund the delivery of settlement services across the country. Some service providers deliver several settlement services and have more than one contribution agreement with the Department. Contribution agreements for the Settlement Program normally run for three years. The contribution agreements that were in place when Syrian refugees started to arrive in 2015 were due to expire 31 March 2016. Because of the Syrian refugee initiative, the Department extended them for one year. Between September 2016 and March 2017, the Department negotiated almost 750 new three-year contribution agreements for the Settlement Program. These agreements took effect on 1 April 2017.

3.21 The Department’s contribution agreements to fund service providers must comply with the Treasury Board’s Policy on Transfer Payments. The purpose of this policy is to ensure that departments and agencies of the federal government properly manage transfer payment programs and use them to address government priorities.

3.22 A main goal of the Department’s Settlement Program is to meet the unique needs of each client. However, individual clients can choose whether to access settlement services. Clients who access the services are expected, but not required, to have needs assessments and referral plans to identify the services best suited to their needs.

3.23 To enroll in language classes, each client must have a language assessment. The purpose of this assessment is to determine each person’s language proficiency and to identify the most appropriate program for language training.

3.24 The Department requires service providers to enter information into its web-based application called the Immigration Contribution Agreement Reporting Environment (iCARE) system. The application collects information on the settlement services funded by the Department.

Syrian refugees received needs assessments and language services

3.25 We found that during their first year in Canada, most Syrian refugees received the settlement services we examined. For example, more than 80 percent had their needs assessed, and 75 percent of those who received language assessments attended language classes. Syrian refugees accessed settlement services at a higher rate than other refugees who arrived in Canada during the same period.

3.26 Our analysis supporting this finding presents what we examined and discusses the following topic:

3.27 This finding matters because needs assessments direct refugees to the settlement services that can best help them integrate into Canada. Access to language assessments and training is important because many refugees have limited knowledge of Canada’s official languages. Not being able to communicate in either of Canada’s official languages can affect a refugee’s employability, ability to pursue education, and ability to interact with school representatives, doctors, and neighbours.

3.28 We made no recommendations in this area of examination.

3.29 What we examined. We examined whether 16,643 Syrian refugees who were at least 17 years old and arrived between 4 November 2015 and 31 December 2016 received needs assessments, language assessments, and language training funded by Immigration, Refugees and Citizenship Canada. We did not examine those who settled in Quebec, or children.

3.30 Between November 2015 and December 2016, 9,092 non-Syrian refugees arrived in Canada from overseas. We compared their use of settlement services with that of Syrian refugees who arrived at the same time.

3.31 We examined data reported by service providers in the iCARE system during the period from November 2015 to March 2017. To test the reliability of the iCARE system’s data, we used representative sampling to select 51 services that the system indicated had been delivered to individual clients. We then obtained source documents held by the service providers to confirm service delivery to these clients.

3.32 The results of our sample showed that the delivery of services matched the reporting in the iCARE system in all cases. We were therefore able to rely on the system’s data to evaluate whether Syrian refugees received the settlement services we selected.

3.33 Although we could rely on the data in the iCARE system for this part of our audit, we found problems with the system’s data for language training wait-lists, as described later in this report (see paragraphs 3.57 to 3.61).

3.34 Delivery of settlement services. We found that between November 2015 and March 2017, more than 80 percent of Syrian refugees received needs assessments and language assessments. In addition, more than 75 percent of those who received language assessments also received language training. Syrian refugees accessed the three settlement services we examined at a higher rate than non-Syrian refugees who arrived during the same period (Exhibit 3.1).

Exhibit 3.1—Syrian refugees received selected settlement services

Bar chart showing percentages of Syrian and non-Syrian refugees who received selected settlement services between November 2015 and March 2017

Source: Data provided by Immigration, Refugees and Citizenship Canada

Exhibit 3.1—text version

This bar chart shows the percentages of Syrian and non-Syrian refugees who received selected settlement services between November 2015 and March 2017. The three selected settlement services were

  • needs assessments,
  • language assessments, and
  • language training.

The chart lists percentages of total Syrian refugees and total non-Syrian refugees who received each of these three services.

Percentages are also provided for Syrian refugees according to their entry stream:

  • government-assisted refugees,
  • privately sponsored refugees, or
  • blended visa office–referred refugees.

Percentage of refugees who received service

Refugee group Needs assessment Language assessment Language training

Total non-Syrian refugees

71%

68%

63%

Total Syrian refugees

83%

81%

75%

  • Government-assisted Syrian refugees

91%

85%

82%

  • Privately sponsored Syrian refugees

75%

76%

63%

  • Blended visa office–referred Syrian refugees

74%

80%

76%

Note: The figures for language training include those refugees who had their language assessed and received language training.

Source: Data provided by Immigration, Refugees and Citizenship Canada

3.35 The Department noted that Syrian refugees who were in the privately sponsored or blended visa office–referred category used fewer settlement services than did Syrian refugees in the government-assisted category. The Department determined that Syrian refugees in the privately sponsored category had more knowledge of English or French than did Syrian refugees in the government-assisted category (82 percent of the privately sponsored refugees knew some English or French on arrival, compared with 16 percent of the government-assisted refugees). This difference could help explain why privately sponsored refugees attended fewer language classes than did the government-assisted refugees.

The Department provided additional funding to settle Syrian refugees, but this funding was not always timely

3.36 We found that Immigration, Refugees and Citizenship Canada identified the settlement services needed by the Syrian refugees, and that it allocated funding to organizations that offered these services. However, some funding for maintaining services that the Department had put in place for Syrian refugees in 2016 was delayed. As a result, on 1 April 2017, some service providers cut settlement services for at least three months.

3.37 Our analysis supporting this finding presents what we examined and discusses the following topics:

3.38 This finding matters because the Department must be able to act quickly to identify and fund the services that refugees need so that they receive them in the right location at the right time. Similarly, appropriate funding can avoid problems such as long wait-lists.

3.39 Our recommendation in this area of examination appears at paragraph 3.49.

3.40 What we examined. We examined whether the Department allocated settlement funds to meet the needs of Syrian refugees in a timely manner. Specifically, we looked at the funds the Department allocated to settlement service providers for the period November 2015 to June 2017.

3.41 Needs-based planning for settlement services. We found that the Department identified the settlement services the Syrian refugees needed, and that it allocated funds according to this analysis.

3.42 The Department allocated settlement program funds to its regional offices in proportion to the number of Syrian refugees settling in those regions. In turn, regional officials worked with local service providers to fund services where the Syrian refugees most needed them. Regional officials analyzed information on where Syrian refugees had settled locally, identified gaps in settlement services (such as too few child-care spaces), and assessed the capacity of service providers. The Department provided additional funds according to this analysis.

3.43 In early 2016, the Department funded services that the Syrian refugees needed immediately, such as needs assessments and language assessments. In April 2016, the Department identified 10 areas of the Settlement Program that needed more funds to meet the Syrian refugees’ needs. These areas included language training, child-care assistance, transportation, and mental health services.

3.44 The Department responded by increasing funds for settlement services for Syrian refugees by $116 million, from $141 million over four years to $257 million over five years (Exhibit 3.2). This increase was necessary because the refugees who arrived under the Syrian refugee initiative needed more settlement services than expected. The Department had originally based its estimates of the costs of the initiative on past trends in providing services to refugees. However, previous refugees had generally required fewer services than did those arriving under the Syrian refugee initiative.

Exhibit 3.2—Immigration, Refugees and Citizenship Canada increased funding for settlement services for Syrian refugees

Bar graph showing original and additional amounts of Settlement Program funding allocated for the Syrian refugee initiative for the 2015–16 to 2019–20 fiscal years

Source: Data provided by Immigration, Refugees and Citizenship Canada

Exhibit 3.2—text version

This bar graph shows the yearly amounts of Immigration, Refugees and Citizenship Canada’s original Settlement Program funding that were approved by the Treasury Board for the Syrian refugee initiative and the amounts of additional funding later allocated by the Department. Funding amounts are shown for the 2015–16 to 2019–20 fiscal years.

Fiscal year Original funding approved by Treasury Board ($ millions) Additional funding ($ millions)
2015–16 $10.0 Not applicable
2016–17 $43.7 $18.5
2017–18 $44.0 $50.8
2018–19 $43.7 $29.0
2019–20 Not applicable $17.7

Source: Data provided by Immigration, Refugees and Citizenship Canada

3.45 Funding delays. We found that some of the funding for maintaining services that were put in place for Syrian refugees in 2016 was delayed, resulting in cuts to some settlement services.

3.46 The Department’s $116 million increase to the budget for settlement services for Syrian refugees included $18.5 million to expand services in the last quarter of the 2016–17 fiscal year. These services included language training, child-care assistance, transportation, and mental health services. The Department noted in August 2016 that to maintain these additional services, it would need to give the service providers who offered them another $51 million in the 2017–18 fiscal year.

3.47 Even though the Department received approval in November 2016 to allocate the additional $51 million in the 2017–18 fiscal year, it did not include these funds in the new contribution agreements that took effect on 1 April 2017. The Department did not start to amend the contribution agreements to include the additional $51 million in services until June 2017, three months into the fiscal year.

3.48 To maintain the new services during the funding delay, the Department suggested that the 113 providers of these services continue to provide them, using existing resources, on the promise that the extra funds would arrive later. However, 16 service providers were not prepared—or were unable—to risk not receiving additional funding. The Department’s delay therefore resulted in cuts to some settlement services for at least three months. For example, some service providers who had added more language training classes to accommodate the Syrian refugees discontinued classes as of 1 April 2017 while waiting for the additional funding from the Department.

3.49 Recommendation. Immigration, Refugees and Citizenship Canada should improve its planning and approval processes to ensure the timely transfer of funding to service providers in order to meet urgent programming needs resulting from such events as the Syrian refugee initiative.

The Department’s response. Agreed. A key element of Immigration, Refugees and Citizenship Canada’s success in managing its grants and contributions is ensuring that funds are made available on a timely basis, aligned with where demand for services exist, and that the Department exercises due diligence in its disbursement. The Department planned for and completed deployment of funding for almost 750 new settlement and resettlement contribution agreements with about 500 separate recipients for the 2017–18 fiscal year, including the disbursement of funds within the unprecedented context of the Syrian refugee initiative.

The Department intends to review where it can make further improvements to its planning and approval processes, particularly for urgent and unexpected programming needs such as the Syrian refugee initiative, by December 2017. This review will include business processes established by the Department to effectively manage grants and contributions. The review will also examine the Department’s mechanisms to engage and work collaboratively with provincial and territorial governments and other stakeholders, regarding Settlement Program funding, priorities, and intake processes.

Managing information for decision making

Overall message

3.50 Overall, we found that Immigration, Refugees and Citizenship Canada did not have sufficient information to efficiently manage language training wait-lists for Syrian refugees. It also lacked information to ensure the consistent delivery of services to Syrian refugees in all regions. Although the Department expected the Syrian refugees to receive a standard and consistent level of service across the country, many of the contribution agreements we examined contained no service expectations.

3.51 These findings matter because many of the Syrian refugees who arrived in Canada needed extensive settlement services, and the federal government had allocated about $257 million to deliver these services. It was therefore important for the government to have accurate and timely information about the demand for language training, and for it to set clear expectations for the services it funded to ensure that those services would meet the needs of clients.

The Department did not have the information needed to manage language training wait-lists efficiently

3.52 We found that Immigration, Refugees and Citizenship Canada did not have accurate or timely information to manage wait-lists for language training. As a result, the Department could not closely monitor where demand exceeded supply, and it was less able to make timely decisions on where to spend money most effectively.

3.53 Our analysis supporting this finding presents what we examined and discusses the following topic:

3.54 This finding matters because many Syrian refugees spoke or understood little, if any, English or French when they arrived in Canada, so they needed timely access to language training to help them integrate into Canadian society. The Department needed to know whether it directed its resources to the right locations and whether its investments made a difference.

3.55 Our recommendation in this area of examination appears at paragraph 3.62.

3.56 What we examined. We examined how the Department determined whether language training would be available quickly to Syrian refugees.

3.57 Language wait-lists. Even before the Syrian refugees arrived, the Department had information showing that some Settlement Program clients had to wait for language training. However, we found that the Department did not know how long people waited for those services across the country. The addition of many Syrian refugees who also needed language training meant that the Department needed better information to manage wait-lists for language training services.

3.58 We found that the Department had good information about the extent of wait-lists in Ontario. In 2010, the federal government and the Province of Ontario had co-funded a system that gave the Department, the Province, and service providers real-time information on language training services. This information included how many seats were available in classes and which service providers had wait-lists. When the Syrian refugees started to arrive in 2015, the Department used its information about Ontario language training providers to allocate funds to them according to the demand in their areas. The system also allowed the service provider network in Ontario to easily identify available classes for refugees across the network.

3.59 The Department had not implemented a similar approach in other provinces before Syrian refugees started to arrive. We found that the Department did not ask language trainers outside Ontario to report on clients waiting for language classes. Therefore, the Department did not know how many people were on wait-lists across the country when Syrian refugees started to register for language training. After the Syrian refugees started to arrive, the Department’s regional offices had to collect wait-list information directly from the service providers to guide their funding decisions. One region developed its own system for tracking wait-list information.

3.60 In 2016, the Department made changes to the Immigration Contribution Agreement Reporting Environment (iCARE) system to allow a better understanding of the language training wait-lists across the country. However, we found that the information about wait-lists in iCARE was not reliable. The system contained duplicate entries, errors in the data entered by service providers, and incorrect information about how long clients were on wait-lists. Therefore, the Department did not have accurate information about how long Syrian refugees had to wait for urgently needed language training.

3.61 Because the data about wait-lists in iCARE was unreliable, we could not determine whether the extra resources spent to expand language training services were used efficiently and gave Syrian refugees quick access to language training. We also could not determine whether refugees in all parts of the country had equal opportunities to access language training.

3.62 Recommendation. To more efficiently manage and meet the demand for language training, Immigration, Refugees and Citizenship Canada should improve its management of wait-lists in consultation with service providers.

The Department’s response. Agreed. Immigration, Refugees and Citizenship Canada recently created the Settlement and Integration Sector to provide a single point of accountability for the strategic vision and oversight of the Department’s settlement and integration policy, programs, and operations. This will also allow for a more focused and integrated approach and will help support our commitment to ensuring that newcomers to Canada receive timely, high-quality Department-funded settlement services. The Department has already conducted a review of language training and wait-time issues so that it can better understand the dynamic nature of contributing factors, such as financial, structural, geographic, and personal factors. This review is already informing plans to improve wait-list management monitoring and reporting and, going forward, will encompass the sharing of best practices among provinces, territories, and key stakeholders.

Further, in collaboration with provinces and territories, the Department is currently advancing a Pan-Canadian Language Strategy to improve the coordination of English as a second language (ESL) and French as a second language (FSL) programming for adult immigrants and refugees in Canada (outside of Quebec). The strategy, which was approved by senior federal and provincial officials in May 2017, is now being implemented, with enhancements anticipated by March 2019 in the areas of employment-related language training, ESL and FSL literacy, and e-learning. Finally, the Department is exploring options for a national wait-list management system, including expanding the Ontario system to other provinces on a pilot basis.

Most contribution agreements with service providers did not establish expectations for quality and consistent program delivery

3.63 We found that Immigration, Refugees and Citizenship Canada did not establish service expectations in most of its contribution agreements with service providers. As a result, the Department was less able to ensure the quality and consistency of services provided to Syrian refugees and other clients.

3.64 Our analysis supporting this finding presents what we examined and discusses the following topic:

3.65 This finding matters because Immigration, Refugees and Citizenship Canada expects Settlement Program clients to receive a standard and consistent level of service across the country. When contribution agreements do not state service expectations, the quality of settlement services may differ. In addition, assessing the performance of service providers, and of the Settlement Program as a whole, requires a comparison with service expectations.

3.66 Our recommendation in this area of examination appears at paragraph 3.71.

3.67 What we examined. We examined whether the Department’s contribution agreements with service providers included service expectations. An example of a service expectation is that only qualified teachers can lead language-training classes. We selected four settlement services to examine: needs assessments, language assessments, language training, and employment-related services.

3.68 To conduct our analysis, we used representative sampling to examine the 812 amendments the Department made to its contribution agreements to fund the four services for Syrian refugees between November 2015 and March 2017. We also used representative sampling to examine the 502 new contribution agreements the Department funded for the four selected settlement services from April 2017 to March 2020.

3.69 Service expectations in contribution agreements. We found that in the November 2015 to March 2017 contribution agreements, the Department specified service expectations for only 47 percent of language assessments and 57 percent of language training services. In contrast, we found that the Department’s April 2017 to March 2020 contribution agreements were much better—93 percent of the agreements contained service expectations for both language assessments and language training.

3.70 However, the Department did not specify service expectations for either needs assessments or employment services in any of the contribution agreements from November 2015 to March 2017, or in those from April 2017 to March 2020.

3.71 Recommendation. To ensure that the quality of settlement services across Canada is consistent, Immigration, Refugees and Citizenship Canada should include clear service expectations in contribution agreements for services delivered by settlement service providers.

The Department’s response. Agreed. Immigration, Refugees and Citizenship Canada is committed to ensuring value for money in the delivery of all its grant and contribution programs. The Department has already set clear service expectations in almost all April 2017 to March 2020 contribution agreements for language training and assessment services. The Department will expand this by developing more robust service expectations for other settlement services in new and amended agreements. Service expectations will build on existing program descriptions, standards, and policies, as well as any new guidance or policies emanating from follow-up actions related to the Settlement Program evaluation. This will be addressed throughout the duration of the existing agreements and before the signing of the next round of agreements, which is planned for March 2020.

Measuring outcomes

The Department had not fully implemented its strategy to measure the integration of Syrian refugees into Canadian society

Overall message

3.72 Overall, we found that although Immigration, Refugees and Citizenship Canada had developed a strategy for measuring the integration of Syrian refugees into Canada, it did not collect information from the provinces for some important indicators, such as access to health care providers and school attendance.

3.73 This finding matters because measuring outcomes of the Syrian refugee initiative allows the Department to assess whether the Syrian refugees receive the settlement services they need and whether they are successfully integrating into Canadian society.

3.74 Our analysis supporting this finding presents what we examined and discusses the following topic:

3.75 One of the intended outcomes for the Syrian refugee initiative was to help refugees integrate into Canadian society, have a path to Canadian citizenship, and benefit from Canada’s social, medical, and economic systems. The Department funded settlement services with the intention of contributing to this outcome.

3.76 When the federal government provides funding for major undertakings such as the Syrian refugee initiative, the departments involved are required to measure and report on both short-term and longer-term outcomes.

3.77 Our recommendation in this area of examination appears at paragraph 3.91.

3.78 What we examined. We examined whether the Department had a performance measurement approach to assess how well Syrian refugees were starting to settle and integrate into Canadian society. We also examined whether the Department had measured the performance indicators it had identified, with a focus on the indicators it had committed to measuring within our audit period.

3.79 Measurement of integration. We found that the Department had created the Outcomes Monitoring Framework to measure and report the results of the Syrian refugee initiative. The Department’s intention was for the framework to help it measure the short-, medium-, and long-term outcomes of the initiative. The framework had three methods:

3.80 For the first method in the framework, we found that the Department had prepared a performance measurement strategy to assess all aspects of the Syrian refugee initiative—from the selection of refugees overseas to the assessment of their integration into Canada. The Department stated that its performance measurement strategy would be the foundation for ongoing tracking of the outcomes of Syrian refugees.

3.81 The strategy included 36 indicators that the Department had committed to measuring by July 2017 to monitor the delivery of the Department’s settlement services and to assess whether the Syrian refugees were starting to integrate into Canada. The Department had specific indicators for each of three periods.

3.82 For Period 1, the Department measured 2 of its 3 indicators regarding short-term services. For example, it measured how many refugees had their overall needs and language abilities assessed. It did not measure its indicator of whether Syrian refugees received services in Canada’s official languages or in their mother tongue.

3.83 For Period 2, the Department measured only 7 of its 16 indicators regarding whether the refugees were living safely and independently in Canada. Of the 9 indicators the Department did not measure, 5 were about health care and education services, for which it planned to get information from the provinces. However, the Department did not arrange to obtain this information.

3.84 The Department did not collect any information for the following 5 health care and education indicators:

3.85 We were concerned about the Department’s inability to track whether the Syrian refugees had access to basic provincial services, such as health care and education—especially considering that part of the Department’s objective was to help Syrian refugees benefit from Canada’s social, medical, and economic systems.

3.86 For the 4 other indicators not measured, 1 pertained to the effectiveness of language training, 2 to the average number of months that Syrian refugees received income assistance, and 1 to the average income assistance per capita.

3.87 For Period 3, the Department had measured 6 of the 17 indicators by June 2017. However, it was then too early to measure many of the indicators.

3.88 The Department had stated its intention to report publicly, in its annual Departmental Results Report, on some indicators from the performance measurement strategy. However, at the time of the audit, the Department had not yet reported against these indicators.

3.89 For the second method in the Outcomes Monitoring Framework, the Department published a report of its evaluation on its website in April 2017. The report, entitled Rapid Impact Evaluation of the Syrian Refugee Initiative, included early measures of settlement outcomes for the Syrian refugees who arrived in Canada between 4 November 2015 and 29 February 2016. Some of the report’s findings illustrated that the settlement of the Syrian refugees was challenging. For example, 55 percent of the government-assisted Syrian refugees faced a language barrier. However, the report concluded that Syrian refugees were integrating at approximately the same rate as other refugees.

3.90 We did not examine the third method of the framework for assessing the results of the Syrian refugee initiative—that of supporting academic research of the Syrian refugee population—because this research was ongoing during our audit period.

3.91 Recommendation. Immigration, Refugees and Citizenship Canada should update and implement its performance measurement strategy for the Syrian refugee initiative to ensure that the Department can fully and consistently monitor the integration of Syrian refugees into Canada.

The Department’s response. Agreed. Immigration, Refugees and Citizenship Canada developed a comprehensive framework for the monitoring of outcomes for Syrian refugees. This framework was anchored in a multi-year approach and based on multiple data sources. The Department also completed a rapid impact evaluation of early outcomes of the Syrian refugee initiative. As a significant amount of performance information on resettlement and settlement was collected and reported regularly to partners and the public, data collection strategies were adjusted to reflect and meet the critical information and reporting needs of both internal and external stakeholders.

The Department will update its Outcomes Monitoring Framework, including its performance measurement strategy, by January 2018, and continue to report internally and externally on the integration of Syrian refugees as more data and performance information become available.

Conclusion

3.92 We concluded that Syrian refugees were provided with selected settlement services funded by Immigration, Refugees and Citizenship Canada that were needed to facilitate their integration into Canada. The Department identified and funded the settlement services Syrian refugees needed to help them integrate, and most Syrian refugees received needs assessment and language services during their first year in Canada. However, funding delays resulted in cuts to some settlement services for at least three months, and challenges in collecting provincial data on health and education made it difficult to fully monitor the refugees’ progress in integrating into Canadian society.

About the Audit

This independent assurance report was prepared by the Office of the Auditor General of Canada on the provision of settlement services to Syrian refugees. Our responsibility was to provide objective information, advice, and assurance to assist Parliament in its scrutiny of the government’s management of resources and programs, and to conclude on whether Syrian refugee settlement complied in all significant respects with the applicable criteria.

All work in this audit was performed to a reasonable level of assurance in accordance with the Canadian Standard for Assurance Engagements (CSAE) 3001—Direct Engagements set out by the Chartered Professional Accountants of Canada (CPA Canada) in the CPA Canada Handbook—Assurance.

The Office applies Canadian Standard on Quality Control 1 and, accordingly, maintains a comprehensive system of quality control, including documented policies and procedures regarding compliance with ethical requirements, professional standards, and applicable legal and regulatory requirements.

In conducting the audit work, we have complied with the independence and other ethical requirements of the Rules of Professional Conduct of Chartered Professional Accountants of Ontario and the Code of Values, Ethics and Professional Conduct of the Office of the Auditor General of Canada. Both the Rules of Professional Conduct and the Code are founded on fundamental principles of integrity, objectivity, professional competence and due care, confidentiality, and professional behaviour.

In accordance with our regular audit process, we obtained the following from management:

Audit objective

The objective of this audit was to determine whether Syrian refugees were provided with selected settlement services funded by Immigration, Refugees and Citizenship Canada that were needed to facilitate their integration into Canada.

Scope and approach

The audit focused on Syrian refugees who arrived in Canada between 1 November 2015 and 31 December 2016.

The audit included an examination of whether the 16,643 Syrian refugees who were at least 17 years old and arrived in Canada during this period received needs assessments, language assessments, and language training services during their first year in Canada. To do so, we analyzed the information in the Department’s Immigration Contribution Agreement Reporting Environment (iCARE) data system. We used data covering the period from 4 November 2015 to 10 March 2017.

We also examined whether Immigration, Refugees and Citizenship Canada identified and allocated funding to the settlement services Syrian refugees needed to support their integration into Canada. We examined the planning process the Department conducted to identify the settlement services Syrian refugees needed and how it allocated funding to settlement service providers to meet these needs.

In addition, we examined whether the Department had developed a performance measurement framework and measured outcomes related to the settlement and integration of Syrian refugees. We focused on the indicators the Department had committed to measuring within our audit period.

The audit approach included document review and interviews with entity officials at headquarters and in the Department’s offices in the Ontario Region, the British Columbia and Yukon Region, and the Prairies and Northwest Territories Region.

To determine whether the Department established service expectations in contribution agreements with service providers, we used representative sampling of contribution agreements stratified by regions for each of four selected settlement services we selected. For the 812 amendments to contribution agreements in place from 1 November 2015 to 14 March 2017, the sample sizes were

For the 502 new contribution agreements in place starting 1 April 2017, the sample sizes were

To determine whether we could rely on the data in the iCARE system regarding the services delivered to clients, we used representative sampling of those services. The sample size was 51 from a total population of 105,852 services provided to clients from 4 November 2015 to 10 March 2017. To confirm that clients had received the services as indicated in iCARE, we obtained source documents from service providers.

Where we used probability sampling, sample sizes were sufficient to report on the sampled population with a confidence level of 90 percent and a margin of error of +10 percent.

The audit did not examine

Criteria

To determine whether Syrian refugees were provided with selected settlement services funded by Immigration, Refugees and Citizenship Canada that were needed to facilitate their integration into Canada, we used the following criteria:

Criteria Sources

Immigration, Refugees and Citizenship Canada identified and allocated funding to the settlement services that Syrian refugees needed to support their integration into Canada.

  • Immigration and Refugee Protection Act
  • Settlement Program, Terms and Conditions, Immigration, Refugees and Citizenship Canada, 2016
  • Policy on Transfer Payments, Treasury Board, 2008; updated 2012
  • Grants and Contributions Manual—Chapter 2, Planning; Immigration, Refugees and Citizenship Canada, 2016
  • SE 1 Settlement Manual, Citizenship and Immigration Canada, 2011
  • SE 2 Planning Manual, Citizenship and Immigration Canada, 2011

Syrian refugees were provided with selected services funded by Immigration, Refugees and Citizenship Canada to meet their settlement needs.

  • Immigration and Refugee Protection Act
  • Policy on Results, Treasury Board, 2016
  • Policy on Management, Resources and Results Structures, Treasury Board, 2010
  • Policy on Transfer Payments, Treasury Board, 2008, updated 2012
  • Settlement Program, Terms and Conditions, Immigration, Refugees and Citizenship Canada, 2016
  • Resettlement Assistance ProgramRAP Service Delivery Handbook, Immigration, Refugees and Citizenship Canada
  • Inland ProcessingIP 3 In Canada Processing of Convention Refugees Abroad and Members of the Humanitarian Protected Persons Abroad Classes—Part 3 (Private Sponsorship of Refugees Program), Immigration, Refugees and Citizenship Canada

Immigration, Refugees and Citizenship Canada monitored and adapted settlement services to meet the needs of Syrian refugees.

  • Immigration and Refugee Protection Act
  • Policy on Transfer Payments, Treasury Board, 2008, updated 2012
  • Settlement Program, Terms and Conditions, Immigration, Refugees and Citizenship Canada, 2016
  • Contributions in Support of Resettlement Assistance Program, Terms and Conditions, Citizenship and Immigration Canada, 2015
  • Settlement Program, Performance Measurement Strategy, Citizenship and Immigration Canada, May 2013
  • SE 1 Settlement Manual, Citizenship and Immigration Canada, 2011
  • Private Sponsorship of Refugees Program (PSR) and Blended Visa Office-Referred (BVOR) Program, Performance Measurement Strategy, Immigration, Refugees and Citizenship Canada

Immigration, Refugees and Citizenship Canada designed a performance measurement strategy that enabled it to measure the integration of Syrian refugees into Canadian society.

  • Immigration and Refugee Protection Act
  • Policy on Results, Treasury Board, 2016
  • Directive on Results, Treasury Board, 2016
  • Policy on Management, Resources and Results Structures, Treasury Board, 2010
  • Syrian Refugee Resettlement and Settlement Performance Measurement Strategy Framework, Immigration, Refugees and Citizenship Canada

Immigration, Refugees and Citizenship Canada implemented a performance measurement strategy that enabled it to measure the integration of Syrian refugees into Canadian society.

  • Immigration and Refugee Protection Act
  • Policy on Results, Treasury Board, 2016
  • Directive on Results, Treasury Board, 2016
  • Policy on Management, Resources and Results Structures, Treasury Board, 2010
  • Syrian Refugee Resettlement and Settlement Performance Measurement Strategy Framework, Immigration, Refugees and Citizenship Canada

Immigration, Refugees and Citizenship Canada reported accurate and complete results on the integration of Syrian refugees into Canadian society in accordance with its reporting commitments.

  • Immigration and Refugee Protection Act
  • Policy on Results, Treasury Board, 2016
  • Directive on Results, Treasury Board, 2016
  • Policy on Management, Resources and Results Structures, Treasury Board, 2010
  • Syrian Refugee Resettlement and Settlement Performance Measurement Strategy Framework, Immigration, Refugees and Citizenship Canada

Period covered by the audit

The audit covered the period between 1 November 2015 and 30 June 2017. This is the period to which the audit conclusion applies. However, to gain a more complete understanding of the subject matter of the audit, we also examined certain matters that preceded the starting date of the audit.

Date of the report

We obtained sufficient and appropriate audit evidence on which to base our conclusion on 18 September 2017, in Ottawa, Ontario.

Audit team

Principal: Nicholas Swales
Director: Mary Anne Strong

Anastasia Chebakova
Jan Jones
Joanna Murphy
Molik Yadnik
Adam Zenobi
Paul Zind

Acknowledgement

We would like to acknowledge the contribution of Nancy Cheng, Assistant Auditor General, to the production of this report.

List of Recommendations

The following table lists the recommendations and responses found in this report. The paragraph number preceding the recommendation indicates the location of the recommendation in the report, and the numbers in parentheses indicate the location of the related discussion.

Delivering settlement services

Recommendation Response

3.49 Immigration, Refugees and Citizenship Canada should improve its planning and approval processes to ensure the timely transfer of funding to service providers in order to meet urgent programming needs resulting from such events as the Syrian refugee initiative. (3.45–3.48)

The Department’s response. Agreed. A key element of Immigration, Refugees and Citizenship Canada’s success in managing its grants and contributions is ensuring that funds are made available on a timely basis, aligned with where demand for services exist, and that the Department exercises due diligence in its disbursement. The Department planned for and completed deployment of funding for almost 750 new settlement and resettlement contribution agreements with about 500 separate recipients for the 2017–18 fiscal year, including the disbursement of funds within the unprecedented context of the Syrian refugee initiative.

The Department intends to review where it can make further improvements to its planning and approval processes, particularly for urgent and unexpected programming needs such as the Syrian refugee initiative, by December 2017. This review will include business processes established by the Department to effectively manage grants and contributions. The review will also examine the Department’s mechanisms to engage and work collaboratively with provincial and territorial governments and other stakeholders, regarding Settlement Program funding, priorities, and intake processes.

Managing information for decision making

Recommendation Response

3.62 To more efficiently manage and meet the demand for language training, Immigration, Refugees and Citizenship Canada should improve its management of wait-lists in consultation with service providers. (3.57–3.61)

The Department’s response. Agreed. Immigration, Refugees and Citizenship Canada recently created the Settlement and Integration Sector to provide a single point of accountability for the strategic vision and oversight of the Department’s settlement and integration policy, programs, and operations. This will also allow for a more focused and integrated approach and will help support our commitment to ensuring that newcomers to Canada receive timely, high-quality Department-funded settlement services. The Department has already conducted a review of language training and wait-time issues so that it can better understand the dynamic nature of contributing factors, such as financial, structural, geographic, and personal factors. This review is already informing plans to improve wait-list management monitoring and reporting and, going forward, will encompass the sharing of best practices among provinces, territories, and key stakeholders.

Further, in collaboration with provinces and territories, the Department is currently advancing a Pan-Canadian Language Strategy to improve the coordination of English as a second language (ESL) and French as a second language (FSL) programming for adult immigrants and refugees in Canada (outside of Quebec). The strategy, which was approved by senior federal and provincial officials in May 2017, is now being implemented, with enhancements anticipated by March 2019 in the areas of employment-related language training, ESL and FSL literacy, and e-learning. Finally, the Department is exploring options for a national wait-list management system, including expanding the Ontario system to other provinces on a pilot basis.

3.71 To ensure that the quality of settlement services across Canada is consistent, Immigration, Refugees and Citizenship Canada should include clear service expectations in contribution agreements for services delivered by settlement service providers. (3.69–3.70)

The Department’s response. Agreed. Immigration, Refugees and Citizenship Canada is committed to ensuring value for money in the delivery of all its grant and contribution programs. The Department has already set clear service expectations in almost all April 2017 to March 2020 contribution agreements for language training and assessment services. The Department will expand this by developing more robust service expectations for other settlement services in new and amended agreements. Service expectations will build on existing program descriptions, standards, and policies, as well as any new guidance or policies emanating from follow-up actions related to the Settlement Program evaluation. This will be addressed throughout the duration of the existing agreements and before the signing of the next round of agreements, which is planned for March 2020.

Measuring outcomes

Recommendation Response

3.91 Immigration, Refugees and Citizenship Canada should update and implement its performance measurement strategy for the Syrian refugee initiative to ensure that the Department can fully and consistently monitor the integration of Syrian refugees into Canada. (3.79–3.90)

The Department’s response. Agreed. Immigration, Refugees and Citizenship Canada developed a comprehensive framework for the monitoring of outcomes for Syrian refugees. This framework was anchored in a multi-year approach and based on multiple data sources. The Department also completed a rapid impact evaluation of early outcomes of the Syrian refugee initiative. As a significant amount of performance information on resettlement and settlement was collected and reported regularly to partners and the public, data collection strategies were adjusted to reflect and meet the critical information and reporting needs of both internal and external stakeholders.

The Department will update its Outcomes Monitoring Framework, including its performance measurement strategy, by January 2018, and continue to report internally and externally on the integration of Syrian refugees as more data and performance information become available.