Introduction

Background

2.1 The Canada Revenue Agency processes more than 30 million tax returns annually. It also administers and collects business payroll payments and goods and services taxes.

2.2 The Agency’s call centres are meant to give individuals and businesses timely, accurate information about their taxes, credits, and benefits. At the time of the audit, the Agency operated nine call centres across Canada.

2.3 The Agency’s online services and telephone call centres are the primary ways for the public to obtain tax information. The Agency’s call centres help callers with specific questions. They are also the main source of information for those who do not have Internet access, those who are uncomfortable using computers, and those who cannot find answers on the Agency’s website.

Focus of the audit

2.4 This audit focused on whether the Canada Revenue Agency’s call centres provided Canadian taxpayers with timely access to accurate information.

2.5 We examined calls received on the call centres’ lines for addressing questions from individuals, from businesses, and about benefits. To evaluate whether taxpayers had timely access to information through the call centres, we defined an answered call as either one that was answered by an agent or one that was answered by the Agency’s automated self-service system and lasted at least a minute. We also looked at whether callers reached an agent on the first attempt.

2.6 To evaluate whether agents gave callers accurate information, we examined whether the responses that agents gave reflected relevant tax legislation correctly. We did not examine the accuracy of information given by the automated self-service system. We also did not examine how easy it was to find information on the Agency’s website or the quality of that information.

2.7 This audit is important because call centres are a key source of information. If taxpayers cannot get timely access to accurate information, they may file incorrect returns, miss filing deadlines, pay too little or too much tax (and later be subject to reassessment), or miss out on benefits they are eligible to receive.

2.8 We did not examine the Agency’s information technology other than how the Agency uses it to control the number of callers who get access to agents. We examined the human resource capacity at call centres, but we did not look at the overall human resource strategy for call centres.

2.9 More details about the audit objective, scope, approach, and criteria are in About the Audit at the end of this report.

Findings, Recommendations, and Responses

Access and timeliness

Call centre agents answered only about one third of calls to the Canada Revenue Agency’s call centres

Overall message

2.10 Overall, we found that the Canada Revenue Agency gave taxpayers very limited access to its call centre services, including both the automated self-service system and call centre agents.

2.11 We found that the Agency blocked more than half of the calls it received (about 29 million out of 53.5 million) because it could not handle the volume. Blocked calls were those that did not reach either an agent or the automated self-service system. Instead, they were given either a busy signal or a message to go to the website or call back later. This means that each caller made an average of three or four call attempts per week. Even after several attempts, some callers did not always reach an agent or the automated self-service system.

2.12 This finding matters because taxpayers need timely access to accurate information to help them prepare their tax returns and to ensure that their benefits are correct. For callers who do not have Internet access, the call centres may be the only way for them to ask for information.

2.13 Our analysis supporting this finding presents what we examined and discusses the following topic:

• Unanswered calls

2.14 Between March 2016 and March 2017, individuals and businesses made more than 53.5 million telephone calls to the Canada Revenue Agency about taxes or benefits.

2.15 During the period covered by the audit, the Agency’s Call Centres Services Directorate managed the nine call centres. The Directorate had a number of telephone lines, including those for addressing questions from individuals, from businesses, and about benefits. Depending on their questions, callers could respond to recorded voice prompts and select from a variety of options to hear general information, and in some cases, to get account-specific information.

2.16 The Agency had a traffic team in each of its nine call centres. The teams were tasked with ensuring that wait times to speak with an agent did not exceed two minutes. One of the ways the traffic team accomplished this was by blocking calls or directing them to the automated self-service system when the wait time approached two minutes. Blocked calls were any calls that did not reach either an agent or the automated self-service system. Callers who did not succeed in reaching an agent needed to redial the Agency if they still wished to speak with one.

2.17 The Agency’s Taxpayer Bill of Rights describes 16 taxpayer rights and the treatment to which taxpayers are entitled when they deal with the Agency. These include the right to complete, accurate, clear, and timely information.

2.18 Our recommendation in this area of examination appears at paragraph 2.32.

2.19 What we examined. We analyzed the demand for telephone services for addressing individual, business, and benefit questions. We examined callers’ access to the Agency’s call centres for both the automated self-service system and call centre agents. We defined “answered calls” as calls answered by an agent or calls answered by the automated self-service system that lasted at least a minute.

2.20 We considered how many calls were blocked by the Agency. We also reviewed the Agency’s practice compared with other countries.

2.21 Unanswered calls. We found that between March 2016 and March 2017, the Agency answered only 36 percent of the calls it received—32 percent were answered by an agent, and 4 percent went to the automated self-service system and lasted at least a minute (Exhibit 2.1). On average, callers made three or four call attempts per week. However, even after several attempts, callers did not always reach either an agent or the automated self-service system.

2.22 The remaining 64 percent of calls were unanswered calls: 54 percent were blocked (that is, received a busy signal or a message to go to the website or call back later), and 10 percent were directed to the automated self-service system, but the caller hung up in less than a minute.

2.23 Compared with similar tax administrations in other countries, the Agency blocked more calls (Exhibit 2.2). Blocked calls were any calls that did not reach either an agent or the automated self-service system.

2.24 The Agency’s call centre traffic teams monitored the time callers waited to speak with agents. When the average wait time approached two minutes, the traffic team either blocked calls (by giving them a busy signal or a message to go to the website or call back later) or directed them to the automated self-service system. The Agency blocked calls so that it could meet its service standard for agent wait times. Depending on the telephone line, the service standard stipulates that 75 to 80 percent of the time, callers should wait no more than two minutes to speak with an agent.

2.25 Because the Agency blocked calls, we found that on average, in the 2016–17 fiscal year, callers made three or four call attempts per week. Even so, their calls were not always answered.

2.26 We found that calls to the Agency’s call centres increased by 27 percent between the 2012–13 and 2016–17 fiscal years. The Agency experienced the highest call volume in the 2015–16 fiscal year, likely due to changes in tax legislation, such as the Canada child benefit.

2.27 Over the past five years, the Agency increased its average number of agents by 23 percent, from 2,022 in the 2012–13 fiscal year to 2,482 in the 2016–17 fiscal year. We found that despite these added resources, 68 percent of calls did not reach an agent in the 2016–17 fiscal year (Exhibit 2.3).

2.28 The Agency told us that callers would prefer a busy signal or an automated message to waiting a long time to speak with an agent. However, the Agency had not surveyed callers to verify this assumption.

2.29 When responding to calls, both the agents and the automated self-service system offered information to callers on various self-service options available on the website, such as the My Account portal on the Agency’s website.

2.30 The Agency has studied caller profiles to better understand the reasons for calls and determine whether callers’ questions had changed over time. The Agency asked its agents to gather this information by inviting callers to participate in a survey. It also gathered information about the use of its website.

2.31 As part of the survey, agents asked callers whether they had tried to get information from the website before calling. Based on 2013–14 information, the Agency found that 13 to 40 percent of callers—depending on the telephone line (individual, business, or benefit)—had first tried to get information from the Agency’s website. However, most of their questions could not be answered by visiting the website. The Agency told us that it is using this and other information to make continuous improvements to the website so that the information there will be easier to find and understand.

2.32 Recommendation. The Canada Revenue Agency should

• review how it manages its incoming calls to improve access by considering practices such as increasing their maximum wait times to speak with an agent; and
• consider giving callers information on call wait times to access an agent so they can decide if they prefer to wait, use self-service options such as the website, or call back later.

The Agency’s response. Agreed. The Canada Revenue Agency’s aging call centre technology does not allow it to automatically route incoming calls to the next available agent across its national network, nor is it capable of providing callers with the estimated wait time to be served by an agent. As a result, the Agency manages the number of callers who can access its phone queues at each site. By limiting the number of callers who join the queue, the Agency can maintain reasonable wait times for those in the queue. (The current service standard is two minutes or less, 80 percent of the time.) As a result, when call volumes are high, it often takes callers several tries to get into the queue to speak with an agent.

In summer 2017, the Canada Revenue Agency signed a commitment to transition its call centres to a new telephony platform as part of the Government of Canada’s Contact Centre Transformation Initiative. The migration is expected to begin in early 2018, with the bulk of the transition being completed by the end of 2018. This new technology will allow the Agency to inform callers of estimated wait times to speak with an agent. Being able to predict wait times will allow the Agency to adjust its current approach of managing incoming calls so as to allow more callers to access the agent queues.

In the interim, throughout the 2017–18 fiscal year, the Canada Revenue Agency will examine how it manages wait times in an effort to reduce the number of attempts that callers need to make to reach a call centre agent. In doing so, the Agency will consult with Canadians on acceptable wait times and update its service standards by the end of the 2017–18 fiscal year.

Additionally, the Agency will continue to enhance the self-service options available in its Interactive Voice Response system so as to offer callers more self-service options where possible instead of waiting to speak with an agent.

Accuracy

Overall message

2.33 When agents responded to our tax questions, they gave us wrong information almost 30 percent of the time. This meant that the actual rate of agent errors was significantly higher than the Canada Revenue Agency’s own test results. One reason for the high rate of incorrect responses could be gaps in training. Another could be that agents must use too many different applications to look for answers. We also found that the Agency’s National Quality and Accuracy Learning Program did not test the accuracy of agents’ responses effectively or independently.

2.34 This finding matters because the Agency made a commitment under the Taxpayer Bill of Rights to give taxpayers accurate information. To do this, the Agency has to know what percentage of call centre agent responses are wrong, and use that information to improve both the training and information available to agents.

2.35 A key component of call centre management is ensuring that telephone agents provide quality services. The Agency’s Call Centres Services Directorate defines a quality call as one that responds to the caller’s needs and that fulfills the Agency’s commitments under the Taxpayer Bill of Rights.

2.36 Each year, the federal budget contains changes to tax rules that Canadians must comply with. To provide accurate information to taxpayers, call centre agents must be familiar with the rules and keep up to date with changes.

2.37 The Agency has training programs for its call centre agents and gives them access to information they can use to answer callers’ questions.

2.38 The Agency’s call centre system is not able to record calls to assess the quality of agents’ responses. During the period covered by the audit, evaluators from the Agency’s National Quality and Accuracy Learning Program listened to at least six calls for each agent every quarter to assess how calls were handled. The evaluators looked at whether the responses were accurate. They also looked at some 20 other items, such as whether the agent followed policies and procedures, showed good communication and technical skills, and made reference to the Agency’s website.

Call centre agents gave us inaccurate information almost 30 percent of the time

2.39 Our test results found that agents gave wrong information to callers almost 30 percent of the time. We also found that other assessors had encountered similar error rates over the past five years.

2.40 Our analysis supporting this finding presents what we examined and discusses the following topic:

• Agent error rates

2.41 This finding matters because callers assume the information they get from call centre agents is accurate. This could lead to callers paying too much or too little tax and later being subject to reassessments or objections. It might also result in taxpayers not receiving benefits they are entitled to.

2.42 Our recommendation in this area of examination appears at paragraph 2.66.

2.43 What we examined. Between February and April 2017, we made 255 calls to Agency call centres to assess the accuracy of the information provided by agents. We used 17 questions, some of which were the same as those used by the Agency or other external studies to assess the accuracy of information provided by call centre agents. The questions we asked were general, so agents did not need to access a specific taxpayer account. We also reviewed accuracy studies conducted by third parties and by the Agency.

2.44 As well, we examined the Agency’s quality assurance system, which measures, monitors, and reports internally on the accuracy of information provided by call centre agents.

2.45 Agent error rates. Exhibit 2.4 shows the results of internal and external tests of agent accuracy. The Agency’s internal assessment found inaccurate responses between 6 and 20 percent of the time, while external tests showed inaccuracy rates of from 24 to 31 percent. Our tests found that responses were inaccurate almost 30 percent of the time overall.

2.46 Exhibits 2.5, 2.6, and 2.7 show some of the questions we asked for which we received a high rate of incorrect responses. We considered a response to be correct if the agent provided an accurate response, or if the agent did not know the answer but referred us to a more senior agent.

2.47 We found that agents were always courteous, professional, and attentive to our questions.

2.48 Regarding Exhibit 2.5, the Agency’s policy was to stop collection action when a taxpayer formally disputed the amount assessed. The agents’ incorrect responses may have affected callers’ decisions to object to the Agency’s tax assessment or may have affected when the caller paid a disputed amount. Late payments would be subject to interest, and early payments could restrict the taxpayer’s cash flow.

2.49 Regarding Exhibit 2.6, many agents advised us to keep the purchase documents of a rental property for less time than required. If a taxpayer did not keep proof of the purchase of a rental property, the taxpayer might not easily be able to prove the property’s original cost at the time of a future sale to calculate a capital gain or loss. The taxpayer might incur costs in the future to get proof of the original cost of the property.

2.50 Regarding Exhibit 2.7, accurate information is important to ensure that both parents receive their fair share of the Canada child benefit. If changes in marital status are not updated on a timely basis, one parent may receive too much and the other too little.

The Agency’s testing for accuracy was unreliable and underestimated how often agents provided wrong answers

2.51 We found that the Agency’s National Quality and Accuracy Learning Program did not test the accuracy of agents’ responses effectively or independently. Therefore, the results of its tests were unreliable. For example, in most cases, agents knew that their calls were being monitored, which may have encouraged them to change their behaviours to improve their performances. Our testing confirmed that the actual rate of agent errors was significantly higher than what the Agency found.

2.52 The Agency has a training program for new agents. However, we found that agent training records were not centralized to identify agents’ training needs. We also found that agents had to use too many tools to locate accurate information, which slowed them down.

2.53 Our analysis supporting this finding presents what we examined and discusses the following topics:

• Accuracy assurance
• Training
• Information tools

2.54 This finding matters because without independent ways to assess the accuracy of agents’ responses, the Agency cannot know if it is providing correct information to callers. Also, if agents do not receive the training they need to answer questions, or if they have difficulty finding answers to the questions they are asked, it is more likely that they will give inaccurate responses.

2.55 Our recommendation in this area of examination appears at paragraph 2.66.

2.56 Accuracy assurance. We found that the Agency’s National Quality and Accuracy Learning Program did not test the accuracy of agents’ responses effectively or independently. One method it used was to have a “certified listener” sitting beside an agent to listen to both sides of the call. The certified listener assessed the overall quality of the call, including the agent’s technical skills, ability to follow policies and procedures, and accuracy. With this method, agents would know that their calls were being monitored, which may have encouraged them to change their behaviour to improve their evaluation.

2.57 Another method the Agency used to test agents’ accuracy was to have agents make anonymous calls to other agents and ask non-account-specific questions. Agents told us that in these cases, they often recognized the caller’s voice, since it was the voice of one of their colleagues. In many cases, their telephone system also identified that the call was coming from a testing line.

2.58 The Agency also used remote listening assessments where agents were unaware that their calls were being monitored. Agents had to volunteer to be evaluated using this type of testing and could withdraw at any time. However, due to technology constraints, the ability to listen remotely to calls existed in only three of the nine call centres. We found that, as a result, an average of just 11 percent of agents were tested this way.

2.59 The Agency’s methods of testing underestimated the error rate. Exhibit 2.4 shows that the error rates estimated by the Agency’s National Quality and Accuracy Learning Program were consistently lower than those estimated by other testers.

2.60 Training. The Agency provided training to new call centre agents working on the telephone lines for individuals and businesses. This training was based on the 10 topics that were the subject of most calls. During the training, agents answered questions and learned about the research tools available to them. Coaching was provided as part of the National Quality and Accuracy Learning Program. Agents were told that if they could not respond to a call, they were to transfer the call to an appropriate line.

2.61 We found that the Agency knew the individual training needs of its call centre agents. However, agent training records were not centralized, which made it difficult for headquarters to follow up on whether the training had been provided.

2.62 Furthermore, according to a 2016–17 study conducted by the Agency, agents referred almost 8 percent of the calls they received to another agent at the same level or higher. In 89 percent of these cases, agents did this because they had not been trained to answer the call.

2.63 The Agency’s National Quality and Accuracy Learning Program found an inaccuracy rate of between 6 and 7 percent (Exhibit 2.4). However, our testing resulted in an inaccuracy rate of almost 30 percent. The Agency recommended only 3 percent of all agents for further training. Agents may need additional training or coaching on providing accurate information.

2.64 We also found that the Agency was inconsistent in assessing agents’ training results. Agents were tested after the initial training, but not always after subsequent training.

2.65 Information tools. Agents told us it was a challenge to find information in the Agency’s systems when responding to questions. Agents used about 29 different applications for the lines for individuals and about 25 for the lines for businesses, making it difficult to find the right information quickly. Depending on the nature of the question, the agent may have only been required to access a few applications or tools.

2.66 Recommendation. The Canada Revenue Agency should ensure that its quality assurance practices generate more effective results in order to improve accuracy, identify opportunities for continuous improvement, and identify and monitor training needs.

The Agency’s response. Agreed. The Canada Revenue Agency is committed to ensuring that its quality assurance practices are effective and result in improved accuracy, and has developed a three-pronged improvement plan.

First, in the fourth quarter of the 2017–18 fiscal year, the Agency will launch a new approach to training and evaluating agents to better assess agent readiness across the national network.

Second, technology will be improved. The current call centre technology is outdated and does not offer industry-leading quality assurance features. The new telephony platform will offer the Agency modern call monitoring tools. These features will be implemented in call centres in the 2018–19 fiscal year, and will strengthen quality and accuracy assessments. Until the new technology is in place, the Agency will continue to examine quality issues that are raised to correct them through the training programs.

Finally, as part of this migration, the Agency will establish a new national quality monitoring team in spring 2018 to supplement existing local quality practices. This will ensure consistency across the national network, leverage quality improvement features available with the new technology, facilitate continuous improvement, and identify training needs, all of which will benefit Canadians when they interact with the Agency.

Reporting

The Agency overstated its call centres’ results in its public reporting

Overall message

2.67 Overall, we found that the Canada Revenue Agency’s public reporting overstated its call centres’ results. The reporting did not account for the Agency’s blocking of about 29 million calls, more than half its total call volume. These are the calls that received a busy signal or a message to go to the website or call back later.

2.68 According to the Agency, about 90 percent of callers are connected to either the automated self-service system or a call centre agent. By blocking and redirecting calls, the Agency was able to report that it had met its targets for all telephone lines. However, when blocked calls are factored in, the Agency’s overall success rate was 36 percent.

2.69 This finding matters because the Agency needs performance measures that reflect callers’ experiences in order to improve its services. Its current reporting methods did not lead to improvement, nor did they accurately inform the public of the Agency’s actual level of service.

2.70 Our analysis supporting this finding presents what we examined and discusses the following topic:

• Reporting on call centre services

2.71 Under its Taxpayer Bill of Rights, the Agency has committed to providing complete, accurate, clear, and timely information to taxpayers.

2.72 Under the Canada Revenue Agency Act, the Agency must report on how its performance compared with its objectives. One measure of the Agency’s performance is how successful its call centres were in providing callers with accurate and timely information.

2.73 Our recommendation in this area of examination appears at paragraph 2.83.

2.74 What we examined. We examined the Agency’s methods to assess its performance and to report on caller access to its call centre services. We also analyzed how long callers stayed connected to the automated self-service system, and how many callers hung up on the automated system and called again to reach an agent.

2.75 Reporting on call centre services. We found that when it measured caller access, the Agency disregarded how many calls a person needed to make before they got through to the Agency’s call centre services. For example, the Agency considered that a caller had 100 percent access to call centre services even if the caller had to try four times, experiencing three blocked calls before getting through.

2.76 Other countries, such as the United Kingdom, Australia, and New Zealand, reported performance measures that either reported or accounted for all calls, including blocked calls. By contrast, the Agency did not report on or account for blocked calls.

2.77 While the Agency tracked the total number of calls it received, answered, or blocked, we found that it did not report publicly on the blocked calls. It only reported the percentage of callers who were able to access either an agent or the automated self-service system, regardless of the number of attempts the caller had to make before getting through.

2.78 The Agency’s weekly target is to connect 80 to 85 percent of callers to its services (that is, to agents or the automated self-service system). We found that this measure disregarded how many calls the Agency blocked. In its 2016–17 Departmental Results Report, the Agency reported a success rate of between 87 and 90 percent. If the Agency had reported on access to its call centre services from a taxpayer’s perspective, it would have defined an answered call as one that was not blocked and was answered by an agent or connected to automated self-service options for at least one minute. Using this approach, the Agency would have reported a 36 percent success rate (Exhibit 2.1).

2.79 Furthermore, in its reporting, the Agency considered all calls to the automated self-service system to be “connected,” regardless of how long the calls lasted. Since December 2016, for calls to the lines for individuals, the Agency has tracked the length of calls that were redirected to the automated self-service system, where there was no option to speak with an agent. The Agency’s internal analysis found that, on average, in 75 percent of calls that were directed to the automated self-service system, the caller hung up before even listening to the main menu.

2.80 We found that for 20 percent of calls that were directed to the automated self-service system, the caller hung up within 20 seconds. Furthermore, our analysis showed that 69 percent of callers who reached the system called back repeatedly until they reached an agent. This shows that even though a call may have connected with the system and been reported as a connected call, it often did not result in the caller receiving answers.

2.81 From the 2012–13 to 2016–17 fiscal years, the Agency’s service standard was to have its agents answer calls within two minutes between 75 and 80 percent of the time. We found that this applied only to calls that were routed to agents, not to blocked calls or calls redirected to the automated self-service system (with no option to speak with an agent). The Agency achieved its service standard by blocking or redirecting calls to the automated self-service system with no option to speak with an agent.

2.82 The service standard may have given callers the impression that they had a choice to speak with an agent, when in fact, more than 60 percent of calls were blocked or automatically forwarded to the automated self-service system, which did not give the caller an option to speak with an agent.

2.83 Recommendation. The Canada Revenue Agency should assess and improve its performance indicators and its reporting on the accessibility, accuracy, and timeliness of its call centre services.

The Agency’s response. Agreed. The Canada Revenue Agency has expanded the information provided in its annual Departmental Results Report, beginning with the report for the 2016–17 fiscal year, to provide Canadians with more information about the performance of its call centres. This document is public and available to all Canadians.

The current call centre technology does not allow for sophisticated, comprehensive, or standardized reporting across the Agency’s national call centre network. The migration to a new telephony platform will offer the Agency enhanced measurement and reporting tools, which will contribute to improved and consistent reporting and performance measures.

The migration will also provide an opportunity for the Agency to review the performance indicators it uses to measure its call centre services. The Agency will complete its review and update its performance indicators by the end of the 2017–18 fiscal year.

Conclusion

2.84 We concluded that the Canada Revenue Agency’s call centres did not provide taxpayers with timely access to call centre agents. Based on our test results, agents gave taxpayers information that was not accurate almost 30 percent of the time. While the Agency reported that it met its targets for both access and timeliness, its performance measures were incomplete and its call centres’ results were overstated.