Evaluation of Industry Canada’s Involvement in the International Telecommunication Union (ITU)

Final Report

May 2015

Presented to the Departmental Evaluation Committee on May 19, 2015

Approved by the Deputy Minister on June 11, 2015

Table of Contents

• Executive Summary
• 1.0 Introduction
  • 1.1 Background and ITU Description
  • 1.2 Program Description: Canada's Involvement in the ITU
  • 1.3 Objectives of ITU Participation
  • 1.4 Stakeholders
  • 1.5 Logic Model
• 2.0 Methodology
  • 2.1 Evaluation Approach
  • 2.2 Objective and Scope
  • 2.3 Evaluation Issues and Questions
  • 2.4 Data Collection Methods
  • 2.5 Limitations to the Data Collection Methods
• 3.0 Findings
  • 3.1 Relevance
  • 3.2 Performance: Achievement of Outcomes
  • 3.3 Performance: Economy and Efficiency
• 4.0 Additional Priorities
  • 4.1 Governance and Accountability
  • 4.2 Issues Surrounding the Mandate of the ITU, Internet Governance and IC's Involvement in the ITU-D Sector
• 5.0 Conclusions and Recommendations
  • 5.1 Conclusions
  • 5.2 Recommendations
• Annex A - Management Response and Action Plan (MRAP)

Abbreviations and Definitions used in this Report

List of Tables

List of Figures

Executive Summary

Program Overview

The International Telecommunication Union (ITU) is the United Nations Specialized Agency for telecommunications and information and communication technology (ICT) issues, acting as the global focal point for governments and the private sector in developing networks, services and related standards. Industry Canada (IC) has the leading role for Canada's ITU representation and provides an annual financial contribution to the ITU. In 2012, Canada's authorization level for the contribution was reduced from $6,808,000 to $4,808,000 Canadian per year.

Based in Geneva, Switzerland, with 193 member states (i.e. governments) party to its treaty instruments, the ITU also engages more than 700 private sector entities and academic institutions.^{Footnote 1} Founded on the principle of international cooperation between the public and the private sectors, the ITU represents a global forum for government and industry to work towards consensus on a wide range of issues. Nevertheless, the ITU is an intergovernmental agency where negotiations take place between member states.

The ITU's role in helping the world communicate spans three core sectors: Radiocommunication (ITU-R), Telecommunication Standardization (ITU-T), and Telecommunication Development (ITU-D). Canada's key objectives in contributing to the ITU are as follows:^{Footnote 2}

• To secure Canada's interests in the international regulation of the radio frequency spectrum and international telecommunication regulation to protect Canadian interests in access to the spectrum and satellite orbit resource as a means, for example, of facilitating communications across Canada and of protecting Canadian sovereignty in remote areas through modern digital technologies;
• To protect Canadian growth and infrastructure by ensuring international recognition of Canadian radio stations and ensuring that Canadian stakeholders have equitable access to radio frequency spectrum resources;
• To provide leadership and focus in promoting the competitiveness of the Canadian telecommunications industry's interests, e.g., by working closely with Canadian stakeholders to obtain spectrum allocations at the global conferences, and development of global standards for telecommunications through the work of the ITU;
• To work with other countries to harmonize policy and regulatory frameworks, promote interconnection and interoperability of global telecommunication networks and services, and deal strategically with them to facilitate access to key markets;
• To build effective consultation and information dissemination processes with Canada's key trading partners; and
• To promote Canadian expertise, products and services with the ITU Member countries and sector members and associates.

Evaluation Purpose and Methodology

In accordance with the Policy on Evaluation and the Directive on the Evaluation Function, this evaluation assessed the relevance and performance of Industry Canada's involvement in the ITU. Findings and conclusions are based on the analysis of multiple lines of evidence including a document and literature review, interviews, review of the 2013 survey of stakeholders and follow-up stakeholder workshop, rapid impact questionnaire and financial data analysis.

Findings

Relevance

There is a continued need for IC to be involved in the ITU, in working groups and preparatory meetings, as well as the major conferences themselves. The ITU is the only intergovernmental agency where nations negotiate for the international allocation of radio spectrum, register satellite orbital positions, and develop global telecommunications standards. The work of the radiocommunication (ITU-R) and telecommunication standardization (ITU-T) sectors makes significant contributions to the Canadian economy and standard of living. As such, the Government of Canada, led by IC, must be active in the ITU-R and ITU-T so that Canadian interests are reflected in the development of international telecommunication regulations and standards. There continues to be a role for the Government of Canada, led by IC, to play in the development (ITU-D) sector, to build capacity in developing countries to both assist the developing world and provide market access for Canadian industry.

IC's involvement in the ITU is consistent with federal government priorities related to Canada's digital future, securing economic growth, and asserting Canada's sovereignty in the North. The objectives and activities of Canada's involvement in the ITU are also in line with IC's mandate and strategic outcomes, and consistent with federal roles and responsibilities relating to telecommunications and international regulation.

Performance

Overall, IC is achieving the majority of its immediate outcomes. IC is effective in influencing the ITU to the benefit of Canada with regards to ITU-R and ITU-T outcomes. IC has been less effective with regards to ITU-D outcomes because Canada has been playing a smaller role in the ITU-D. There is a risk that IC may not be able to continue achieving the same level of benefits for Canada, because its capacity to fully participate in the ITU is declining.

IC has been influential at the ITU in developing global technical standards and providing access to and protection of spectrum, satellite orbits, and frequency assignments, resulting in benefits for Canadian companies. Canadian companies have benefited by exploiting or developing new services and technologies and taking advantage of economies of scale. Stakeholders agree that involvement in the ITU has contributed to connectivity and interoperability of global communications networks. Providing access to spectrum, satellite orbits and frequency assignments and protection from interference also contributes to benefits for Canadian companies and Canadians. Continued success in these areas relies primarily on the use of and access to spectrum, which is determined through negotiations by IC representatives with other member states at ITU conferences.

Operational costs represented 35% of the total program costs on average over the last five years. IC's involvement in the ITU has made efficient use of operational expenditures by continuing to achieve results with fewer resources. While a reduction in Canada's monetary contribution may not significantly harm Canada's performance, reductions in staff and the size of delegations sent to ITU meetings risk reducing the effectiveness of IC's involvement.

Recommendations from the 2009 evaluation to implement a succession plan and improve measuring and reporting IC's outcomes at the ITU have not been implemented or fully realized. The lack of succession planning has left IC in a vulnerable position over the next decade when many experienced staff will retire with fewer qualified staff ready to take over the role of ITU preparation and negotiation.

Finally, during the course of the evaluation, program staff identified additional priorities related to governance, accountability, and the mandate of the ITU (notably internet governance). These priorities should be included in future performance reporting exercises.

Recommendations

The findings and conclusions of the evaluation lead to the following recommendations.

1. The program should adopt a strategic approach to determining the size and composition of the IC part of the Canadian delegation for ITU-related meetings. This approach should include:
   1. Implementing a succession strategy for IC staff working on ITU matters and taking into account this strategy when composing IC delegations so that the next wave of IC representatives can obtain experience and develop relationships with senior staff and key delegations at ITU;
   2. Continuing the prioritization exercise to determine which meetings and events are of most importance to Canadian and IC interests and also taking into account the nature of the meeting/event and IC participation required (e.g. negotiation, networking, other duties); and
   3. While recognizing that IC should continue being the lead, the program should approach other government departments, where appropriate, to seek additional support and contributions to Canada's participation at the ITU.
2. The program should continue to improve its measurement and reporting on outcomes. For the purpose of communicating outcomes to senior management, stakeholders and for performance measurement and evaluation purposes, the program should consider preparing high level summaries of all the delegate/post conference reports. The summary reports should align Canada/IC's achievements at conferences, objectives for the conferences and the expected outcomes of the program overall. In addition, to reduce the reporting burden on the heads of delegations and to effectively capture the essential performance data, the pre and post conference performance measurement survey should be streamlined and improved and ideally, replaced with a more robust database.
3. The program should examine its foundational documents (e.g. program profile, logic model, Performance Measurement Strategy) that guide the planning, operations and performance monitoring of IC's involvement in the ITU so that program priorities are identified and made explicit.

1.0 Introduction

This report presents the results of an evaluation of the relevance and performance of Industry Canada's (IC's) involvement in the International Telecommunication Union (ITU). The report is organized into four sections:

• Section 1 provides the program context and profile of the ITU;
• Section 2 presents the evaluation methodology and data limitations;
• Section 3 presents the findings pertaining to relevance and performance; and
• Section 4 summarizes the study's conclusions and provides recommendations.

1.1 Background and ITU Description

Federal government policy directs regulators to use international standards as the basis of national telecommunication and radio communication regulations and standards to the maximum extent possible. As a result, many Canadian standards and regulations are adopted directly from those agreed to internationally, such as within the ITU.

The ITU is a United Nations specialized agency based in Geneva, Switzerland, with 193 member states (i.e. governments) party to its treaty instruments. The ITU also engages more than 700 sector members and associates (i.e. private sector entities) and academic institutions.^{Footnote 3} Founded on the principle of international cooperation between the public and the private sectors, the ITU represents a global forum for government and industry to work towards consensus on a wide range of issues. Nevertheless, the ITU is an intergovernmental agency where member states negotiate with each other, and only member states may vote on issues when required.

The ITU's work spans three core sectors: Radiocommunication (ITU-R), Telecommunication Standardization (ITU-T), and Telecommunication Development (ITU-D). The specific functions of the ITU are as follows:

1. To effect through treaty level regulatory instruments, the allocation of bands of the radio-frequency spectrum, the allotment of radio frequencies and the registration of radio-frequency assignments. For space services, the ITU assigns any associated orbital position in the geostationary-satellite orbit or any associated characteristics of satellites in other orbits, in order to avoid harmful interference between radio stations of different countries;
2. To coordinate efforts to eliminate harmful interference between radio stations of different countries and to improve the use of the radio-frequency spectrum for radio communication services and of the geostationary-satellite and other satellite orbits;
3. To effect through binding global treaty-level provisions the facilitation of international interconnection and interoperability of information and communication services, as well as ensuring their efficiency and widespread public usefulness and availability;
4. To provide a unique world-wide venue for government and industry to work together to develop interoperable international standards for telecommunications equipment and services which, in turn, may form the basis of many national standards and proposed regulations and statutes.^{Footnote 4}

1.2 Program Description: Canada's Involvement in the ITU

Canada is bound by its Treaty obligations to participate in the ITU and has been a member state of the ITU since 1932 following an act of Parliament. Canada's membership and IC's role conforms to Section 6 (e) of the Department of Industry Act whereby the Minister shall "take any action that may be necessary to secure, by international regulation or otherwise, the rights of Canada in communication matters." IC's financial contribution to the ITU enables the Government of Canada, working with other stakeholders, including Canadian telecommunication carriers, service providers and manufacturers, to participate in the ITU to enhance Canada's competitiveness in telecommunications worldwide. Under the ITU system member states are required to commit themselves to a level of contribution which is based on a unit system, and not to a specific annual contribution amount. Canada's contributory class has been constant since 1965, but the actual amount fluctuated depending on the value of the Swiss Franc. In 2012, Canada's authorization level was reduced from $6,808,000 to $4,808,000 Canadian per year.

The International Telecommunications Policy and Coordination Directorate (DIT) in the Telecommunications Policy Branch (SPS-DGTP) is the lead for the IC-ITU program overall. The Engineering Planning and Standards Branch (SITT-DGEPS) plays leading roles in other specific areas of IC-ITU work. Each of the involved directorates has broader departmental mandates, and the ITU activities form a portion of their work. Each directorate is responsible for organizing and coordinating their participation at ITU preparatory meetings and international treaty-binding conferences. An organizational chart representing how the work of the IC-ITU program was organized at the time of the evaluation is included in Appendix A.

The IC-ITU work is organized by ITU sector. Table 1 describes each work area, and identifies the branch and directorate that plays the lead role for each area.

Study Groups, Preparatory Meetings and Conferences

Much of the work within the ITU sectors is done in study groups and preparatory meetings outside of or in preparation for international conferences/assemblies. Participation in study groups and Canadian delegations to international meetings and conferences involves representatives from IC, other government departments and Canadian sector members and associates. Currently, IC participates in study groups for the ITU-R and the ITU-T, but does not attend the two study groups for the ITU-D. Table 2 summarizes the consultations, meetings, study groups and conferences associated with each ITU sector.

Study Groups

Study Groups may be national or international (i.e. ITU Study Groups). National Study Groups study the issues and develop country positions and contributions to take to the ITU Study Groups or other meetings. Each study group has a specific focus and participants from both government and private sector work together to define the frameworks that will ensure optimum functioning of all services, both existing and future. The main outputs of the various study groups are the establishment of technical standards or guidelines (referred to as Recommendations by the ITU) and the determination of country positions and contributions, depending on the nature of the study group.

The ITU-R Study Groups (SGs) develop the technical bases for supporting decisions taken at World Radiocommunication Conferences (WRC) and develop global standards, reports and handbooks on radiocommunication matters. More than 4,000 specialists, from member states, the telecommunications industry and academic organizations throughout the world, participate in the work of the study groups.^{Footnote 5} Refer to Appendix B for a list of ITU-R Study Groups.

Standardization work is carried out by the technical Study Groups in which representatives of the ITU-T membership develop Recommendations (i.e. standards) for the various fields of international telecommunications. The Study Groups drive their work primarily in the form of study questions. Each of these addresses technical studies in a particular area of telecommunication standardization. Refer to Appendix C for a list of ITU-T Study Groups.

There are two Study Groups associated with ITU-D, Study Group 1 and Study Group 2. These Study Groups examine specific task-oriented telecommunication/ICT questions of priority to developing countries, to support them in achieving their development goals.

Regional Meetings

In addition to the study groups, Canadian delegations also attend regional meetings such as the Inter-American Telecommunication Commission (CITEL) which is the telecommunications and ICT entity of the Organization of American States (OAS). IC, as head of the Canadian delegation, attends CITEL advisory committee meetings to represent and advance Canada's positions. Attendance at these meetings is critical to coordinate regional preparations for specific ITU events in all three ITU sectors, including the preparation of Inter-American Proposals (IAPs) and common positions, and to conduct inter-regional consultations in preparation for international events and conferences.

Major Conferences

Conferences of the ITU are working meetings as opposed to symposiums or presentations for information purposes. The work of the ITU-R is to ensure rational, equitable, efficient and economical use of the radio-frequency spectrum by all radiocommunication services, including those using geostationary-satellite or other satellite orbits through establishing ITU Radio Regulations. The Radio Regulations are drawn up and approved by the WRC, held every three to four years. The technical basis for the work of the WRCs and thus the ITU Radio Regulations, are the results of the ITU-R studies and recommendations. This sector carries out its work through the WRC, the Radio Regulations Board, the Radiocommunication Assemblies (defines the next period of study for the ITU-R), the Radiocommunication Advisory Group, the Radiocommunication Bureau (BR, the executive arm of the ITU-R), the study groups, various working parties or task groups and regional meetings (e.g. CITEL).

The work of the ITU-T is carried out by its sector members, associates, academia, and member states and supported by the Telecommunication Standardization Bureau (TSB). The technical work is the development of Recommendations (i.e. international standards) through 10 study groups, additional special issue focus groups and regional meetings (e.g. CITEL), composed of experts in telecommunications from all over the world. Every four years, the ITU-T sector holds the World Telecommunication Standardization Assembly (WTSA) which requires regional preparatory meetings and defines the next period of study for the ITU-T.

The work of the ITU-D is carried out by member states, the Telecommunication Development Advisory Group, two Study Groups and Regional Preparatory meetings between conferences, and is supported by the Telecommunication Development Bureau (BDT). As part of the ITU-D, World Telecommunication Development Conferences (WTDC) are convened every four years to review the progress made in telecommunications in developing countries and establish ICT development priorities, strategies and action plans for the future. Priority is accorded to the expansion and modernization of networks, the mobilization of resources and regulatory reform needed to boost telecommunication penetration and access in the world's poorer countries.^{Footnote 6} The WTDC feeds into the ITU Council^{Footnote 7} and the Plenipotentiary Conference of the ITU.

Every four years, the ITU Plenipotentiary Conference is convened. It is the key event at which ITU member states decide on the future role and governance of the organization as well as revise and update, as necessary, the legal instruments of the ITU. As with the other international conferences, much work is done in preparation, including meetings of the CPC for the Plenipotentiary, Canadian National Organization (CNO)/ITU, Council and regional meetings such as CITEL, among others. IC leads the Canadian delegations which also include other government departments and Canadian sector members and associates.

Due to ITU's culture of consensus, voting is relatively rare at ITU conferences, with the exception of the election of ITU's top management and the members of the Radio Regulation Board and ITU Council. Nonetheless, a vote may be called and only member states can cast their one vote—sector members, associates, academia and observers are not entitled to vote.

1.3 Objectives of ITU Participation

Active Canadian participation in ITU activities supports both government and industry objectives. The key objectives in contributing to the ITU as per the program's foundational documentation are as follows:^{Footnote 8}

• To secure Canada's interests in the international regulation of the radio frequency spectrum and international telecommunication regulation to protect Canadian interests in access to the spectrum and satellite orbit resource as a means, for example, of facilitating communications across Canada and of protecting Canadian sovereignty in remote areas through modern digital technologies;
• To protect Canadian growth and infrastructure by ensuring international recognition of Canadian radio stations and ensuring that Canadian stakeholders have equitable access to radio frequency spectrum resources;
• To provide leadership and focus in promoting the competitiveness of the Canadian telecommunications industry's interests, e.g., by working closely with Canadian stakeholders to obtain spectrum allocations at the global conferences, and development of global standards for telecommunications through the work of the ITU;
• To work with other countries to harmonize policy and regulatory frameworks, promote interconnection and interoperability of global telecommunication networks and services, and deal strategically with them to facilitate access to key markets;
• To build effective consultation and information dissemination processes with Canada's key trading partners; and
• To promote Canadian expertise, products and services with the ITU Member countries and sector members and associates.

1.4 Stakeholders

IC's contributions to the ITU allows the Government of Canada, working with stakeholders, to reflect Canadian interests and requirements in the formulation of treaty-binding regulations, the development of global standards and enabling of Canadian industry to use ITU processes and services to enhance Canada's competitiveness in telecommunications worldwide. IC ascertains the requirements and interests of Canadian companies and other government departments through a national consultative process and through the Canadian National Organizations (CNOs) that parallel ITU activities.

Other government departments require adequate access to and protection from interference for spectrum, radio frequency and satellite orbits for their national defence, public safety, law enforcement, weather prediction, civil aviation, space program, and maritime safety purposes. In addition to participating in Canadian consultations, their representatives may attend ITU meetings and conferences as part of the Canadian delegations. Other government department stakeholders include the following:

• Department of Foreign Affairs, Trade and Development (DFATD)
• The International Development Research Centre (IDRC)
• The Canadian Space Agency (CSA)
• Department of National Defence (DND)
• Public Safety Canada
• Canadian Radio-television and Telecommunications Commission (CRTC)
• Transport Canada (TC)
• Department of Fisheries & Oceans (DFO) / Canadian Coast Guard (CCG)
• Environment Canada (EC)
• National Research Council (NRC)
• Health Canada (HC)

Many Canadian companies and other organizations participate directly in the ITU.^{Footnote 9} At the time of the evaluation, there were 17 organizations that were official ITU sector members or associates.^{Footnote 10}

1.5 Logic Model

The current logic model for IC's involvement in the ITU (Figure 1) was drafted during the 2009 evaluation and updated in 2010 as part of the performance measurement strategy.

Figure 1: IC-ITU Program Logic Model

2.0 Methodology

This section provides information on the evaluation approach, objective and scope, specific evaluation issues and questions that were addressed, data collection methods, and data limitations for the evaluation.

2.1 Evaluation Approach

IC's Audit and Evaluation Branch (AEB) conducted the evaluation study. It is based on the goals and expected outcomes of the program as stated in the program's foundational documents and logic model. The evaluators measured the outcomes with a particular emphasis on current concerns and issues of the program, extending the lines of evidence from the 2009 evaluation and assessed follow-up actions in response to the recommendations of the 2009 evaluation.

2.2 Objective and Scope

The objectives of this evaluation were to address the core issues of relevance and performance in accordance with the Directive on the Evaluation Function. The evaluation covered the five-year period of 2009 to 2014. IC's involvement in the ITU was previously evaluated in 2009 with generally positive findings. As a result, this evaluation took a calibrated approach by focusing on immediate outcomes and relying as much as possible on secondary data and by conducting a rapid impact survey of stakeholders.

2.3 Evaluation Issues and Questions

Based on the program performance measurement strategy, and subsequent consultations with the program, the evaluation addressed the following questions:

Relevance

1. Is there a continued need for IC to be involved in the ITU?
2. Does Industry Canada's involvement in the ITU align with the priorities of government and the strategic outcomes of IC?
3. Do the activities associated with IC's involvement in the ITU align with the roles and responsibilities of the federal government?

Performance

4. To what extent has IC's involvement in the ITU achieved its immediate outcomes?
5. To what extent does IC's involvement in the ITU demonstrate efficiency and economy?
   • Were resources allocated and utilized efficiently?
   • Could the same results be achieved with fewer resources or could the results be improved with the same resources?
6. Are there improvements IC could make with respect to its involvement in the ITU?
   • Were follow-up actions in response to the recommendations of the 2009 evaluation effective?

2.4 Data Collection Methods

Multiple lines of evidence were used to address all evaluation questions. The data collection methods included a document review, literature review, interviews, rapid impact survey, review of the 2013 stakeholder survey and consultation, and an analysis of financial data.

Document Review

The document review was conducted to gain an understanding of the program and to gain insight into both the relevance and the performance of the program. Key documents included Federal Budgets and Speeches from the Throne, Final Reports of IC's 2013 ITU Stakeholder Consultation, the 2013 Canadian National Organization / ITU-T Manual, the 2010 Performance Measurement Strategy, and the 2009 program evaluation. Evaluators also reviewed the documentation related to the major ITU related conferences, including preparatory documents (e.g. briefing books, objectives reports and high level summaries), post conference delegate reports, and post conference survey reporting completed by heads of delegations. To make the review of documentation manageable, the evaluation focused on IC's participation in the eight major conferences that took place during the timeframe of the evaluation: WTPF 2009; WTDC 2010; PP 2010; WRC 2012; RA 2012; WCIT 2012; WTSA 2012; and WTPF 2013.

Literature Review

A literature review was conducted to better understand the ITU and Canada's role in this organization and to examine the core issue of relevance of both the ITU and Canada's involvement. Evaluators also looked for evaluation approaches of government programs involved in international organizations and best practices in participation in such organizations.

Rapid Impact Survey

Evaluators conducted a short survey with Canadian stakeholders (IC employees, employees of other government departments and ITU sector members, associates or private industry representatives) using a rapid impact evaluation approach which involves surveying a small sample of subject matter experts (both internal and external to a program), on the importance of a program's objectives and the degree to which they are achieved. The survey was conducted from September 11 to 26, 2014 with a final sample of 17 respondents. Interviews were then conducted with the survey respondents to obtain more in-depth responses.

Interviews

The objective of the interviews was to gather in-depth information, including views, explanations and factual information that address the evaluation questions. The interviews were designed to obtain qualitative feedback from a range of respondents. The interviews were conducted in-person, or by telephone if an in-person interview was not possible. A total of 25 interviews were conducted with the following types of respondents:

• IC management and staff involved with the ITU (5)
• Canadian companies/organizations participating in the ITU as sector members or associates (7)
• Other government departments /agencies implicated by IC's involvement in the ITU (4)
• ITU officials (4)
• Representatives of other ITU member states (5)

For the first three types of respondents, a two-part process was used. Respondents were asked to participate first in the online rapid impact survey and then were interviewed to have them expand on their responses in the survey.

Review of ITU Stakeholder Consultation (2013)

In 2013, IC conducted a consultation of Canadian ITU stakeholders in two phases. In March 2013, 66 industry and 13 government stakeholders were invited to complete a detailed online questionnaire on the following themes:

• the relevance of the ITU to stakeholder priorities;
• the effectiveness of ITU activities related to these priorities; and
• the effectiveness of IC in advancing stakeholder interests at the ITU.

The results of the survey were presented at a workshop attended by 26 of the stakeholders to validate the quantitative findings and add richer qualitative information. The findings and conclusions of this consultation were reviewed for the purposes of the evaluation.

Financial Data Analysis

Financial information on IC's involvement in the ITU was collected and interviews with stakeholders were conducted to examine the evaluation issues of efficiency and economy. The efficiency of travel dollars and the allocation and utilization or resources were reviewed.

2.5 Limitations to the Data Collection Methods

One of the challenges of this evaluation was obtaining concise, simple and consistent performance information on each of the many immediate outcomes from existing secondary data. The program produces very detailed, technical and complex pre-and-post ITU meeting and conference reports, but the reports differ in content and format from one to another and are difficult to analyze. The reports also do not consistently link outcomes to stated objectives for each conference or for the ITU program. This was noted in the previous evaluation in 2009 and the program had addressed this in two ways. First, summary reports were prepared by the heads of delegations to link the conference outcomes to IC's outcomes, but only in a few instances.

Second, the program designed a performance measurement survey for conference delegates internal to IC to complete after attending international meetings and conferences. While a good first step on the continuum of performance reporting, the survey's design was problematic (e.g. long and burdensome on staff which led to data quality issues, no data analysis plan was created for how the data collected would be used, the data had not been prepared for analysis, and the survey was unsuitable for conducting queries and generating performance reports quickly and easily). Evaluators had to clean, recode and aggregate the data collected and were only able to report on a few variables (e.g. adoption of Canadian contributions by meeting type. See section 3.2.1). In addition, only 16 months' worth of data (November 2012 to March 2014) had been collected and was available to be analyzed, and again the data on the outcomes of the conference and how they relate to the program's objectives was limited.

The evaluators sought to mitigate these challenges for the evaluation by collecting primary data using the rapid impact survey with subject matter experts both internal and external to the program and in-depth follow-up interviews. The interviews, while qualitative, were detailed structured interviews covering each of the immediate outcomes allowing the evaluators to gain richer information. Interviews were conducted with as wide a range of interest groups as possible to provide all perspectives and reduce bias.

3.0 Findings

3.1 Relevance

3.1.1 Is there a continued need for Industry Canada to be involved in the ITU?

Rationale for IC's involvement with the ITU

The ITU is the United Nations specialized agency responsible for coordinating the global development of telecommunications. It is the multilateral forum for the negotiation among 193 nations of binding international agreements on the use of the radio frequency spectrum, the development of global standards, and the promotion of all facets of the development of the global communications network. Canada is a signatory to the ITU Constitution and Convention, the treaty agreement which is revised, as necessary, through negotiation every four years at its plenipotentiary conference^{Footnote 12}. Furthermore, Canada is one of 48 elected members of the ITU Council, and as a long-time member of the Council, Canada directly influences the work and the governance of the ITU.^{Footnote 13}

According to departmental documents, the goal of IC's involvement in the ITU is to ensure the orderly and secure evolution of Canada's radio spectrum and telecommunications infrastructure through the development of a coherent regulatory framework, promotion of competition, establishment of sufficient regulation, enforcement of domestic and international requirements, and negotiation and promotion of international standards and treaties. This provides Canadian industries with the favourable conditions they need to develop, introduce and bring to market leading technologies and services. Another goal of IC's involvement in the ITU is to make telecommunications services accessible to Canadians and to ensure that the public derives maximum benefit from their use.^{Footnote 14}

According to the IC-ITU program's foundational documents, the main rationale for Canada's ITU membership is to help ensure Canadian citizens' access to, use of, and participation in the market for, effective telecommunications services.^{Footnote 15} Canada's participation in the ITU (which includes participation in the national and regional preparatory meetings leading up to each major ITU conference) ensures interconnectivity and interoperability with global networks and services; prevents harmful interference to the radio spectrum; helps promote harmonization of telecommunications policies and regulatory frameworks with the other member states; helps the Canadian telecommunication industry gain product and market intelligence, maintain competitiveness and market products abroad; and is one of the prime vehicles for Canadian companies to advance their product lines and research and development activities internationally.^{Footnote 16}

Is the ITU still relevant?

In recent years, the ITU has been criticized because it cannot keep up with the pace of technological change^{Footnote 17 Footnote 18 Footnote 19}. Program staff and stakeholders also criticized the ITU for being slow to adapt. Some say the relevance of the ITU is declining because telecommunications policy at a global level has become more heavily influenced by the market place than by government policy. Two big changes, a transition from state controlled monopolies of communications to a liberalized environment of global competition and the rise of the Internet as the dominant infrastructure for global communications, with separate governance arrangements, pose challenges for the ITU.^{Footnote 20} The ITU has to evolve from governing the traditional telecommunications network (Public switched telephone network or PSTN) that was designed to facilitate two-way voice communication to dealing with telecommunications through Internet Protocol (IP-based) networks and the Internet. Legacy PSTN regulations cannot be transferred or applied directly to the Internet, as there is little commonality between the two.^{Footnote 21}

There were also criticisms in the literature of the ITU for overstepping its role and importance in attempting to centralize decision-making power on the Internet.^{Footnote 22 Footnote 23 Footnote 24} This was proposed at the WCIT in Dubai in 2012 by several nations, in favor of a new plan that would give a role in the technical administration of the Internet to the ITU. Canada was among the nations that did not sign the proposed treaty. Similarly, other countries promote the establishment of new international instruments to legitimize greater state control over information that is transmitted in cyberspace.

Some program staff and stakeholders criticized the ITU for attempting to broaden its scope particularly with respect to Internet governance. However, other stakeholders thought that the ITU has an important role to play in Internet governance, that there is a need to have a regulatory or policy body at the global level, and that the ITU is an appropriate place for these international debates to occur.

The program consulted with its stakeholders in 2013 on the relevance of the ITU to Canadian priorities. The consultation found that Canadian industry stakeholders generally feel that the activities of the ITU are important to their organizations and relevant to their priorities. The ITU-R received the highest ratings for importance and effectiveness among the three sectors, closely followed by the ITU-T. Ratings of the development sector (ITU-D) were lower but still positive.

The literature, interviews with stakeholders and the program's stakeholder consultation provided further insight into the ongoing relevance of Canada's participation in the ITU. The findings are organized by area of concern, since the ITU's work is divided into three distinct sectors, the radiocommunication, telecommunication standardization, and development sectors.

The Radiocommunication Sector (ITU-R)

The WRC is the forum where the Radio Regulations, the international treaty, governing the allocation and use of radio-frequency spectrum and satellite orbits are reviewed and revised. The literature acknowledges the importance of the ITU to the global management of spectrum and telecommunications. The OECD reports that policy makers and regulators have a vital role to play in making sure there is adequate available spectrum among other responsibilities.^{Footnote 25} The ITU itself points out that regulators around the world struggle with developing appropriate regulatory models and defends the role of the ITU as a common venue at the international level where all member states can participate in allocating spectrum for new uses and develop standards and plans that maximize and harmonize use of spectrum.^{Footnote 26}

Interviews conducted for this evaluation indicated that it is still important for Canada to be involved in the ITU-R sector, especially at the WRC process. The ITU-R is the only forum in the world where international spectrum allocations and satellite orbital positions are negotiated. Interviewees stressed that due to the global nature of satellites and the importance of avoiding frequency interference, negotiating and harmonizing spectrum is extremely important for Canadian government, industry and military. Developing spectrum regulations, which are treaty binding, and harmonize the use of spectrum, are intergovernmental endeavors. As such, representatives of member states have the most important voice and only member states may vote on issues if consensus cannot be achieved. Interviewees stressed that Canada must participate in the ITU-R negotiations in order to influence the development of spectrum regulations in line with Canadian interests. Furthermore, Canada must be adequately represented by government officials at both preparatory meetings and conferences because of the weight and authority this carries in international and intergovernmental meetings. Interviewees reported that other member state delegates prefer to deal with IC representatives and are reticent to deal with Canadian sector members or associates.

Multiple lines of evidence showed that Canada should also be adequately represented at each of the domestic meetings, the regional meetings (e.g. CITEL), and the ITU-R study groups, that lead up to the ITU-R major conferences (e.g. RA, WRC), to negotiate the allocation of spectrum Canada requires, and to have input into the international Radio Regulations. Otherwise, Canada would be absent from the decision making on regulations and spectrum allocations. The consequences of this can be quite significant: the decisions made at a WRC to revise the Radio Regulations have long-term financial implications and the allocation of spectrum is very valuable. For example, the federal government raised $5.27 billion in February 2014 in an auction of 700 MHz spectrum licences to eight Canadian companies.^{Footnote 27}

The Telecommunication Standardization Sector (ITU-T)

The literature presents mixed views on the relevance of the ITU-T. For example, some authors point out that the ITU is only one actor among a number of regional or international institutions that have emerged to deal with issues such as standards development or infrastructure development for telecommunications services across the globe. Others say the ITU's traditional standard setting for the monopoly telecommunications era does not suit it for the era of the Internet and rapid innovation in telecommunications. ^{Footnote 28 Footnote 29 Footnote 30} Nevertheless, other voices in the literature point out that the emergence of prominent national and regional standard setting organizations does not make the ITU-T a less relevant forum for standards making. The standards of the ITU-T carry more formal international weight than those of most other standards organizations that publish technical specifications of a similar form. No other organization has the global reach of the ITU and the setting of global standards is of great importance to the Canadian telecommunications industry.

For example, the ICT Standards Advisory Council of Canada (ISACC) conducted a study in 2013 of the challenges facing ICT standards in Canada.^{Footnote 31} It states that the ITU-T, along with the International Organization for Standardization (ISO), International Electrotechnical Commission (IEC), and the joint technical committee (JTC) of the ISO and IEC, have maintained their prominence in the standards development environment. It also acknowledges the shift in focus from accredited standards development organizations (SDOs) to non-accredited forums as specific issues arise and trends develop in ICT. It stresses that standards are extremely important for enterprises to meet regulatory requirements and interoperability, meet market expectations, establish common terminology, access global markets, and provide a competitive edge. The report noted, "With its notable overall contribution to the Canadian economy, it is clear that ensuring Canadian needs are met in the development of ICT standards is of critical importance to current usage and future trends."^{Footnote 32} ISACC recommends greater funding and participation in the development of Canadian input into international standards through coordinating the efforts of standards experts in national, regional and international organizations/events (including both specific issue consortia and global forums such as the ITU).^{Footnote 33}

Other lines of evidence support the literature on the importance of ITU-T to standards making. In the 2013 Canadian ITU stakeholder survey, respondents rated the relevance of ITU-T in the current telecommunications environment and the rise of other SDOs. The overwhelming majority rated it as somewhat or extremely relevant, with almost no dissenting voices. The survey findings and workshop discussions showed that stakeholders think the ITU is highly relevant to their business priorities, including the area of standardization, and that it is effective in advancing these priorities.

Even though industry representatives may take the lead in the ITU-T sector, and the standards developed there are not treaty binding, it is nonetheless important to have Canadian government representation in order to engage with other countries in the ITU-T process, particularly on global standards. Interviewees confirmed that there are many SDOs, but that the ITU-T occupies "a unique place as a global standards organization with a global perspective" because it develops standards that govern the manufacturing and use of telecommunications worldwide, which is particularly important when many countries do not develop national standards. By adopting worldwide standards, one interviewee stated, "one can take a mobile device from Canada and it will work anywhere in the world the same way it works at home without any need to change components." While a minimum of one member state representative is required for a country's delegation to the WTSA, interviewees explained that an adequate number of government delegates must be involved at the ITU-T meetings, first because the members of other member state delegations typically prefer dealing with government representatives, and second to ensure that Canadian interests are reflected in the standards. In the words of one respondent, IC "needs to be there to push it to evolve, to align with Canadian interests."

The ITU Telecommunication Development Sector (ITU-D)

The ITU-D is intended to foster international cooperation in the delivery of technical assistance and the creation or improvement of telecommunication/ICT equipment, networks and regulations in developing countries. While information communications technology has become very important to the global economy, there is a significant digital divide between the developed world and the developing world. The ITU's role in the development sector is in line with ITU's policy mission, even according to its critics,^{Footnote 34} and there is a need for developed nations to assist in improving access, coverage, and services and decreasing the cost of telecommunications in the developing world. An enabling telecommunication regulatory environment and increased competition can significantly influence the affordability of ICT services.^{Footnote 35 Footnote 36}

Interviewees indicated that the ITU development sector is still relevant to Canada, primarily for capacity building purposes, but also for humanitarian reasons. Interviewees further explained that through capacity building in developing countries, Canadian industry has the opportunity to engage with developing nations thus giving them access to growing export markets.

Program documentation on the ITU-D confirms the relevance of its work to assist developing countries to bridge the digital divide, increase broadband and internet penetration and use, and confirms that Canadian participation is important to influence the priorities of the ITU-D. Canada sent 12 delegates to the WTDC in 2010 and has been very engaged in ITU-D matters advancing its views on the need for the ITU-D (and the ITU in general) to remain bound by its mandate and core competencies and to avoid duplication of efforts among the three sectors. Further, the program documentation states that it is important for Canada to continue to participate in the ITU-D sector to advance and gain support from developing countries for its positions on the role of the ITU.

3.1.2 Does Industry Canada's involvement in the ITU align with the priorities of government and the strategic outcomes of Industry Canada?

Government strategies and policies, the 2013 Speech from the Throne and subsequent Federal Budgets indirectly support IC's involvement with the ITU. Access to and protection of spectrum, satellite orbits and frequency assignments, are the foundation for achievement of the federal government's priorities and these can only be achieved through involvement in the ITU.

Digital Canada 150 outlines the Government's intention to ensure that Canadians not only benefit from the digital revolution but also are among its leaders.^{Footnote 37} In order to achieve that goal, Digital Canada 150 presents a comprehensive approach to ensuring Canada can take full advantage of the opportunities of the digital age. This approach includes securing access to high-speed Internet at 5 megabits per second (Mbps) for over 98% of all Canadians for faster, more reliable online services; ensuring the Government of Canada is a leader in using digital technologies to interact with Canadians; and assisting industries to seize digital opportunities, promote Canadian content and play a more prominent role in the global marketplace.

The Speech from the Throne 2013^{Footnote 38} reiterates the Government's commitment to creating jobs and securing economic growth. The speech also states that our Government will renew the Canada First Defence Strategy and look at acquiring radars and satellites to improve surveillance capabilities, especially in the Arctic.^{Footnote 39} Finally, the speech states that, "our Government is securing our Northern sovereignty". RADARSAT, is a key initiative as part of Canada's Northern Strategy^{Footnote 40}. It uses earth observation data applications to protect Canada's sovereignty in the North, conduct marine surveillance, support disaster management; and monitor Ecosystems.^{Footnote 41} Subsequent Budgets (2013 and 2014) have renewed the Government's commitment to creating jobs and protecting Canada's North and investing in broadband internet service for Canadians in rural and Northern communities.

Interviewees unanimously agreed that IC's involvement in ITU aligns with the government's priorities and IC's strategic outcomes. The priorities are accomplished through access to spectrum and by decreasing radio frequency interference, both of which rely on IC's engagement with the only international body that regulates the use and protection of spectrum. As one interviewee pointed out, if Canada is not present at the ITU to protect investments like RADARSAT, and other nations wish to introduce new uses of bands that RADARSAT operates in, the government's investment will jeopardized.

Support for IC's involvement in the ITU aligns with departmental priorities under Spectrum and Telecommunications Policy and Legislation Program Activity of IC's Program Alignment Architecture (PAA). This Program Activity contributes to the Strategic Outcome: The Canadian Marketplace is Efficient and Competitive. The program's ultimate outcomes also align with IC's strategic outcomes: Advancements in Science and Technology, Knowledge, and Innovation Strengthen the Canadian Economy and Canadian Businesses and Communities are Competitive.

3.1.3 Do the activities associated with Industry Canada's involvement in the ITU align with the roles and responsibilities of the federal government?

IC's mandate supporting involvement in the ITU stems from the Department of Industry Act, 1995. Subsection 4(1) (k), defines the powers, duties and functions of the Minister to extend to and include all matters over which Parliament has jurisdiction, relating to telecommunications. In exercising these powers, the Minister is responsible for initiating, recommending, coordinating, directing, promoting and implementing national policies, programs, projects and practices with respect to the objectives set out under section 5(g) "[to] promote the establishment, development and efficiency of Canadian communications systems and facilities and assist in the adjustment to changing domestic and international conditions" The Radiocommunication Act s.5(1)(k) further details the Minister of Industry's powers to take such action as may be necessary to secure, by international regulation or otherwise, the rights of Canada in telecommunication matters.

Canada's involvement with the ITU is consistent with involvement by governments in other countries. The ITU website indicates there are 193 countries that are ITU member states. Interviewees from all groups explained that the ITU is an inter-governmental UN organization where only member state administrations can vote, despite the fact that industry representatives and academics can attend meetings and conferences. Industry representatives emphasized the importance of participation of Government of Canada representatives to influence the decisions of other member states and ultimately the ITU.

This was corroborated by the findings of the 2013 stakeholder consultation, which reported that participation of government representatives in conferences and meetings is necessary, even in areas of activity that do not involve international treaty-making and regulation (e.g. telecommunications standardization and development). Stakeholders expressed that there would be a significant impact on their organizations if IC no longer developed Canadian positions for the ITU-T and ITU-D sectors. Respondents to the evaluation's rapid impact survey and interviewees also emphasized the importance of IC's involvement in the ITU and the importance of sending enough knowledgeable, experienced government representatives to ITU conferences to defend the interests of Canadian companies and citizens at meetings and conferences. Furthermore, respondents indicated that IC's participation in the ITU is critical to the continued operation and growth of Canadian telecommunication industry.

Interviewees from all groups also indicated that Canada's longstanding involvement in the ITU and continued membership in the ITU Council, which acts as the Union's governing body in the interval between Plenipotentiary Conferences, has earned Canada great respect from other countries as a trustworthy and moderate voice at the ITU. Interviewees frequently mentioned that Canada has been seen as a leader on technical and administrative aspects because of the quality and competency of its representatives and that IC must maintain a strong presence at the ITU or Canada will risk losing its influence.

3.2 Performance: Achievement of Outcomes

3.2.1 To what extent has IC's involvement in the ITU achieved its immediate outcomes?^{Footnote 42}

IC's involvement in the ITU: Benefits to Canada

The Performance Measurement (PM) Analysis report indicates that Canadian contributions to the ITU have been highly successful. Of the 135 contributions submitted by Canada to an ITU meeting (November 2012 to March 2014)^{Footnote 43}, 43 contributions were adopted and 83 were accepted (98%). Of the remaining, three were categorized as "for information" and six were "rejected." These rejected contributions have not been among Canada's high-priority items.^{Footnote 44} Table 4 provides a breakdown of Canadian contributions by sector or meeting type.

According to the PM Analysis report, of all the ITU meetings reported within the timeframe mentioned above, Canada successfully achieved 99% of its objectives with the support of Canadian contributions, Inter-American Telecommunication Commissions Inter-American Proposals (IAPs), Canadian positions or statements. Refer to Appendix D, Achievement of Objectives for more information.

In addition, the 2013 stakeholder survey found that stakeholders think IC's role of representing Canada in the ITU and coordinating Canadian participation in its conferences, assemblies and meetings is important for their organizations and effective in advancing their priorities.

Further details on the achievement of expected results for each of the radiocommunication (ITU-R), telecommunication standardization (ITU-T) and development (ITU-D) sectors of the ITU are provided below.

ITU-R Outcomes

IC's immediate outcomes with respect to the ITU-R include:

• The ITU adopts binding regulations consistent with Canadian positions;
• The ITU allocates radio spectrum in line with Canadian proposals; and
• The ITU registers satellite orbits and frequency assignments consistent with Canadian positions

Adoption of binding regulations

The WRC is the forum where radio regulations and the international treaty governing the use of radio-frequency spectrum and satellite orbits are reviewed and revised. The Canadian Delegation Report for the WRC 2012, reports that Canadian positions were advanced and a significant number of important, treaty-binding decisions dealing with a range of complex technical and regulatory issues were taken (p.6). These were consistent, in large measure, with Canadian positions and objectives agreed between government and industry stakeholders prior to the Conference. Similarly, the 2009 evaluation reported that 74% of Canada's agenda item positions were completely satisfied as reported by the WRC-07 delegation report. Table 4 also indicates that 92% of the 86 contributions submitted by Canada at an ITU-R meeting were either adopted or accepted as the basis of further work.

The evaluation's rapid impact survey found that the majority (76%) of respondents saw the adoption of binding regulations as a high to very high priority for their organization, but they ranked it lower on performance; 53% said that this was achieved often to always; 24% said sometimes. (Refer to Appendix E for the results of the rapid impact survey.) Interviewees indicated that Canada has been more or less always successful, but they believed this success was at risk without sufficient manpower to network to obtain support for Canadian positions. For example, one interviewee said that, "we are not succeeding at this lately. We used to send two people to each working group. Now we are sending fewer…[which] makes it very difficult to get support for our positions at these meetings." Interviewees further explained that in the past Canada has been able to influence regulations that are adopted because Canada is viewed as presenting moderate views compared to other countries. As such, Canada has been seen as the "honest broker", and has been very effective in influencing regulations.

Allocation of radio spectrum

At the WRC, member states negotiate the international allocation of spectrum and the ITU, which holds a master registry of frequency allocations and orbital slots along the radio spectrum, acts on the bases of decisions taken by member states. The majority (76%) of respondents to the rapid impact survey saw the allocation of radio spectrum as a high to very high priority for their organization, yet somewhat fewer (59%) said that this was achieved often to always. Interviewees stated that IC has been good at clearly stating its objectives and needs to attend the meetings and conferences in order to negotiate with other member states. One interviewee commented that, "at the last conference, we did not have enough manpower at the conference to win additional spectrum for mobile…it makes our work very difficult after the conference." Others indicated that Canada seems to be relying more and more on following the lead of the US and in the words of one interviewee, "we follow the US…[instead] we should be doing what's best for Canadian industry and Canadians."

Registration of satellite orbits and frequency assignments

Registration of satellite orbits and frequency assignments was a lower priority area to the rapid impact survey respondents (53% saw this as a high to very high priority for their organization) and less than half (47%) said that this was achieved often to always. Nevertheless, satellite orbits and frequency assignments are very important for Canada. There are currently two geostationary satellite companies in Canada.^{Footnote 46} These companies represent approximately $960 million annually in revenues and employ over 500 people.^{Footnote 47} Interviewees stated that if IC had not laid the groundwork by negotiating satellite orbits at the ITU decades in advance, these satellite companies could not exist in Canada. Furthermore, as an interviewee clarified, "if satellite positions are not registered at the ITU, there is no protection and therefore those who invest in satellite companies would be leery to invest without ITU approval."

The 2009 evaluation revealed that IC had 99% of its frequency assignments approved for which the process was completed. For this evaluation, IC staff provided data from ITU publications demonstrating that in most cases, Canadian satellite networks planned frequency bands filed with the ITU were found favourable to Canadian interests (see Appendix F).

ITU-T Outcomes

IC's immediate outcome with respect to the ITU-T is that the ITU develops and adopts technical standards consistent with Canadian positions.

Stakeholders see Canada as frequently getting its positions adopted when it comes to developing global technical standards. The evaluation's rapid impact survey found 71% of respondents said that this was achieved often to always. Furthermore, Table 4 indicates that 100% of the 35 contributions submitted by Canada at an ITU-T meeting were either adopted or accepted as the basis of further work. Interviewees also see this as an area of strength for Canada. They indicated that IC has been influential in developing global technical standards at the ITU. Canada has also been successful in achieving its objectives by working with other member state allies, such as the US and UK, when necessary.

According to program records, Canada's representation to the ITU-T has greatly decreased over the last five years or longer. This is partly due to the fact that there are fewer large Canadian companies that participate in large numbers at ITU-T meetings (e.g. Nortel, Research In Motion/BlackBerry Ltd). In addition, IC has fewer staff members dedicated to ITU-T issues which may pose a risk to IC's continued achievement of results in the area. IC also sends a small number of representatives to the WTSA, which may pose a risk when multiple, concurrent meetings take place.

Nevertheless, many Canadian companies (e.g. BlackBerry Ltd, Ericsson Canada, Microsoft Canada, MBNA Canada) still rely on IC to maintain its presence at the ITU-T meetings. One interviewee said that he "cannot think of any standards that have been adopted that have been harmful to Canada", but as he and several other interviewees stated, "there is no way of ensuring this if we do not continue to participate at the ITU conferences."

ITU-D Outcomes

IC's immediate outcome with respect to the ITU-D is that ITU adopts development action plans consistent with Canadian positions. However, there is little evidence that IC is achieving this outcome because it has been a lower priority for Canada in recent years and Canada's engagement in the ITU-D has waned. Interviewees reported that Canada has been sending delegates to the WTDC and the Telecommunication Development Advisory Group (TDAG), whereas related study group meetings have not been attended by IC for over a decade.

The 2013 stakeholder consultation found that ratings for the ITU-D sector were lower but still positive; 20% of respondents ranked its activities as extremely important for their organizations and 44% as somewhat important. In terms of its effectiveness in addressing their priorities, 15% ranked ITU-D as excellent and 30% as good. Similarly, this was the lowest priority among respondents to the evaluation's rapid impact survey with only 35% of the respondents rating this as a high to very high priority for their organization. Forty-seven percent of respondents also indicated that the ITU adopts development action plans often to always, 12% said sometimes and 41% did not know.

Previously, Canada's development activities included involvement with the Telecommunications Executive Management Institute of Canada (TEMIC). The TEMIC was funded by private sector members, such as Nortel, SR Telecom and Teleglobe, as well as by the Canadian International Development Agency (CIDA). From 1987-2010 the TEMIC assisted developing countries to expand their telecommunications sectors by improving the managerial abilities of senior telecommunications managers.^{Footnote 48} Interviewees explained that this program brought approximately 250 mid-managers to Canada for two weeks, a few times a year, to learn about Canadian telecommunications. These same managers "became senior managers and remembered their opportunities in Canada…[which] helped get them to vote with us [at the ITU]." Interviewees pointed out that capacity building for developing nations helped to build networks, gain access to decision-makers, and identify possible markets for Canadian companies. Interviewees who mentioned TEMIC expressed disappointment that it no longer exists; however, the program studied whether reinstating TEMIC was appropriate and found that the model was no longer relevant and that there was no support from the private sector for its revival.

IC's Involvement in the ITU: Benefits to Canadian Companies

Canada's involvement in the ITU has the following expected outcomes related to benefits for Canadian companies^{Footnote 49}:

• Canadian companies exploit/develop new services and technologies;
• Economies of scale for Canadian companies;
• Connectivity and interoperability of global communications networks; and
• Protection and access of spectrum, satellite orbits and frequency assignments that are consistent with Canadian interests.

Canadian companies exploit or develop new services and technologies

Interviewees explained that Canadian companies need access to spectrum to exploit or develop new services and technologies for Canadian and international markets. With access to more spectrum, Canadian companies are also able to develop products and services in an area of rapid growth. OECD research shows "the unprecedented increase in the take-up of smartphones and other wireless devices is leading many to reassess the amount of radio frequency spectrum that will be required to support future growth in the Internet economy. This is critical, as it will underpin the ability of OECD countries to ensure competitive communication markets, which will in turn drive the innovation necessary to meet broader economic and social development."^{Footnote 50}

The 2009 evaluation found that the vast majority of stakeholders had moderate (42%) or great success (55%) in facilitating Canadian companies in exploiting their services and technologies or developing new ones. Correspondingly, the 2013 stakeholder consultation found that 85% of stakeholder organizations were able to add new products and services in accordance with their business plans and goals. For the 2014 rapid impact survey, 94% of respondents said that these benefits occurred for Canadian companies to a moderate or great extent. Respondents felt this outcome was being achieved to the greatest extent relative to the other outcomes. See Appendix G for the results of the rapid impact survey.

Interviewees also indicated that Canadian companies require access to spectrum to expand into broadband applications. The only way to gain access to more spectrum is through negotiation and participation by the Government of Canada at the WRC and in the preparatory process leading up to it. Many Canadian companies are participating in the upcoming 2015 WRC process to express their desire for additional spectrum for mobile communications. The Government of Canada's participation the WRC is important as only member state representatives can negotiate for spectrum usage.

Economies of scale for Canadian companies

The OECD^{Footnote 51} reports that spectrum harmonization, the uniform allocation of radio frequency bands across entire regions, makes economies of scale possible. When regulators align their mobile-sector spectrum with international band plans, mobile devices can be built less expensively because a single model can be manufactured for many countries. This brings down the cost of deploying networks and manufacturing wireless devices, as well as facilitating roaming services and helping to mitigate harmful cross-border interference. Additionally, "by creating economies of scale, ICTs allow Canadian companies to increase their competitive advantage and therefore their contribution to the Canadian economy." ^{Footnote 52} Although the ICT sector in Canada consists mainly of small companies, from 2007 to 2011 ICT sector revenues increased by 16%, from $134 to $155 billion.^{Footnote 53}

The 2009 evaluation's survey showed that 91% of stakeholders believed that IC's activities to influence ITU global technical standards had moderate or great success in leading to increased economies of scale for Canadian telecommunications companies. Correspondingly, the 2013 stakeholder consultation found that 100% of respondents reported that IC's ITU involvement has been either a great (50%) or moderate (50%) success in facilitating cost containment through economies of scale. Furthermore, when asked about international sales of Canadian telecommunications products and services, 55% percent reported that their international sales were increasing over time, while 40% reported that their sales were remaining stable. The 2014 rapid impact survey found that 77% of respondents said that increased economies of scale were achieved to a moderate or great extent.

Interviewees explained that at ITU conferences, decisions are taken to develop sharing conditions that enable industry to operate and develop systems within certain parameters and thus companies are able to make products that can work worldwide, rather than making a version for each country. The negotiation of worldwide standards allows manufacturers to save on testing and developing their products and as one interviewee expressed, "the ITU has contributed to forcing a narrowing of the technologies and convergence is necessary for economies of scale."

Connectivity and interoperability of global communications networks and services

Connectivity refers to the ability of telecommunication devices and networks to connect with those in other countries. For example, Canadians can use their landlines, cellular or Internet phones, or computers to contact parties and access websites in other countries. Interoperability refers to the ability to use telecommunications devices in other countries and Canadian companies can sell their devices in countries around the world. Interoperability also ensures that the networks (e.g. military and public safety) in different countries can interconnect and work with each other. Binding regulations, allocation of radio spectrum, and global technical standards all facilitate connectivity and interoperability.

The 2009 evaluation's survey showed that 91% of stakeholders believed that IC's involvement in the ITU has had moderate to great success in facilitating connectivity and interoperability of global communications networks, products and services. Interviewees unanimously agreed that connectivity and interoperability of global communications networks exists as evidenced by the pervasiveness of telephony and mobile networks worldwide.

Canada has been instrumental at the ITU in improving connectivity and interoperability of global communications. For example, Canada was successful in adding the Global Flight Tracking system to the WRC-15 agenda. The goal will be to allocate enough spectrum for a new Global Flight Tracking system that will improve surveillance but also increase air traffic control capabilities, allow more planes to fly at one time, and save money for airline companies. The adoption of a new agenda item "Global flight tracking for civil aviation" by the Plenipotentiary is a positive outcome for IC's CNO/ITU-R process and of importance to Canadian private sector stakeholders. In addition, Canada established a database for emergency public protection and disaster relief following the 2004 Indian Ocean earthquake and tsunami that hit Thailand. The database records all nations' radio frequencies and is managed by the ITU. In the event of an emergency, the ITU will provide frequency data to planes and ships providing relief.

Protection and access of spectrum, satellite orbits, and frequency assignments are consistent with Canadian interests

Canada's involvement in the ITU secures Canada's interests in the international regulation of the radio frequency spectrum. It also protects Canada's interests in the access to the spectrum and satellite orbits as a means of facilitating communications across Canada and of protecting Canadian sovereignty in remote areas through modern digital technologies. In addition, by registering satellite networks with ITU (through notifications and coordinations to the ITU), IC gains international recognition for Canada's space-based frequency assignments and the ability to protect them from harmful interference.^{Footnote 54}

In the 2009 evaluation's survey of stakeholders, 100% of stakeholders believed that IC has had moderate or great success in the protection and access of spectrum consistent with Canadian interests. Further, 96% believed that IC has had moderate or great success in the protection and access of satellite orbits and frequency assignments consistent with Canadian interests. In comparison, the 2014 rapid impact survey found that 77% of respondents said that access to spectrum was achieved to a moderate or great extent and 76% of respondents said that protection of spectrum from harmful interference occurred to a moderate or great extent. In addition, 53% of respondents said that both access to, and protection of, spectrum was achieved often to always. Although positive, these figures suggest that stakeholders perceive a decrease, compared to the 2009 evaluation, in Canada's achievement of results for this expected outcome.

Interviewees reported that IC has been successful in protecting spectrum used for the Canadian satellite RADARSAT from interference from Wi-Fi companies that want access to use the same spectrum. Canada's continued involvement is necessary to protect Canadian interests as well as to negotiate for more spectrum.

IC staff explained that protection of Canada's radio frequencies involves frequency coordination to mitigate harmful interference. Appendix H indicates that during the timeframe of this evaluation (2009-2014) there were seven incidents of interference and 1065 ITU, 68,812 US, and 36,206 Canadian notifications and coordinations (overall total 106,083). With so few incidences of harmful interference, and resolutions in all cases, evidence shows that IC is able to protect Canadian spectrum through multi-lateral relations with the US, other member states and by providing notifications and coordinations to the ITU.

With respect to the protection of satellite orbits, under the ITU regulatory framework, nations must coordinate their use of the spectrum and orbits. Interviewees indicated that there must be agreements between nations to avoid harmful interference and IC plays an important role in facilitating such agreements. For example, IC avoided potential harmful interference for Canadian companies when an agreement was signed in 2006 between Canada and the United Kingdom (UK) that set out how spectrum would be shared between the two nations.

Interviewees remarked that there are very few incidences of harmful interference and that Canada has been successful in protecting and providing access to satellite orbits and frequency assignments. One of the challenges in demonstrating the utility of the work done to protect access and reduce interference is the absence of a counterfactual. As one interviewee expressed, "If you do your job well, there won't be problems…if I don't do my job, there will be problems." This means that potential areas of interference of satellite orbits and frequency assignments are dealt with expeditiously, before critical issues arise.

3.3 Performance: Economy and Efficiency

3.3.1 To what extent does Industry Canada's involvement in the ITU demonstrate efficiency and economy?

The efficiency and economy of IC's involvement in the ITU considered the allocation and utilization of resources and the achievement of results in relation to resources.

Allocation and Utilization of Resources

IC's participation in the ITU, including all cost elements, is estimated at $9 million per year. The full breakdown of costs is shown in Table 5. There are three primary cost elements:

• Canada's financial contribution to the ITU: IC provides an annual financial contribution to the ITU. In 2012, the authorization level was reduced from $6,808,000 per year to $4,808,000 per year.
• IC's staff costs: At the time of the evaluation, there were two sectors and three branches directly involved in the ITU activities, with on average 27 FTEs (full time equivalents) devoted to ITU activities across the directorates. This represents an average of $2.5 million dollars in salary per year. Annual salary costs have declined by 24% from 2009-10 to 2013-14.
• IC's travel costs: This includes travel to consultations, meetings, conferences and assemblies. The average annual travel costs are $635,000. Annual travel expenses have declined by 46% from 2009-10 to 2013-14.

As a percentage of the total cost (Figure 2), Canada's financial contribution to the ITU makes up the largest share at 63% of the average cost per year from 2009-10 to 2013-14. Staff salaries represent 28% of the total cost of IC's participation in the ITU over this same time period. At 7%, travel costs make up the smallest share of IC's continuing participation in the ITU. On average over the last five years, the program's operational costs make up 35% of the total cost.

Figure 2: Average ITU Costs from 2009-2010 to 2013-2014 Shown as a Percentage of Total

IC interviewees explained that resources have declined in recent years. To manage reductions, IC and industry have implemented strategies for prioritizing issues at meetings and conferences. Nevertheless, IC interviewees say they do not have enough resources to fully participate in the ITU (e.g. attend all the important sessions, attend full conferences, and be present for informal negotiations). IC interviewees believe the staff members are able to address the priority issues, and to some extent the secondary issues, but that they cannot be as thorough as in previous years. One interviewee expressed, "It's risk management. Any disasters? Not yet…[but] if you don't participate at the meetings, you don't know what's going on and if you're not well prepared, you might let things slip."

Interviewees were able to offer few suggestions for improvements that would lead to greater efficiencies. This view was corroborated by other stakeholders external to IC who perceive that Canada outperforms other countries. "Canada tops the list…[I] can't see how they would be more efficient…always a skeleton staff that work very hard to address all the issues." Another interviewee from the private sector commended IC staff for managing so many issues at once at a conference.

Achievement of Results in Relation to Resources

The Contribution to the ITU

Under the ITU system, member states commit to a level of contribution based on a unit system, and not to a specific annual contribution amount. Every four years, members may select one of 18 classes of contribution, the highest class is 40 units and the lowest, 1/16 unit. The lowest units are reserved for the least developed countries as listed by the United Nations and for other members as determined by the ITU Council. Larger and more populous countries dominate in terms of actual amounts, with the top 10 member state contributors providing approximately 56 per cent of total funding in 2012-2015. Some of the ITU's largest contributors are identified in Table 6.

Since 1965, Canada had been contributing 18 units. In 2012-13, Canada's annual grant to the ITU was reduced by $2 million "to bring it in line with contributions from other G7 countries."^{Footnote 59}

Interviewees thought that the amount of Canada's assessed contribution to the ITU had been consistent with Canada's commitment to the UN in general and more or less consistent with the contributions of other countries with a similar gross domestic product (GDP) to Canada. On the other hand, the 2013 stakeholder consultation found that Canada has consistently contributed more to the ITU budget than its share of global gross national income (GNI), a benchmark that is generally used in the UN to assess countries' financial contributions. The majority of 2013 survey respondents felt that Canada's influence has been greater than expected given its size.

Interviewees explained that reducing the contribution, within treaty limits, is acceptable and there has been a trend for a number of countries to decrease (for example, France, Germany and Spain have reduced their contributions). However, in the past decade, some member states have made significant funding increases, including China, India, the Republic of Korea, the Russian Federation and Saudi Arabia.^{Footnote 60}

Other interviewees commented that the reduction in the number of delegates is more damaging than the reduction in monetary contribution because fewer delegates makes it challenging to address issues that are important to Canada. Interviewees explained that it is impossible for small delegations to attend concurrent sessions at the larger conferences, and having fewer experienced IC delegates makes it difficult to negotiate with other administrations in both formal and casual settings.

An interviewee from another country remarked that the effect of Canada withdrawing or reducing its participation would be "enormous" and that the ITU "needs more voices like the voice of Canada, which is respected and accepted by all parties. Canada has an interesting status among member states. Its position is close to that of the US, but not the same. Canada is taken very seriously by other countries and has a bigger voice compared to other countries with larger economies." Interviewees from all areas expressed that Canada consistently outperforms compared to the size of its monetary contribution due to the technical competence and experience of the IC delegates and overall preparedness of the Canadian delegations.

Interviewees explained that the reduction in delegation sizes and travel expenses is recent and that the impact may not be felt immediately. Interviewees also warned that the consequences of decisions taken at ITU conferences can take five years or longer to materialize and can be significant. Therefore, the work preparing Canadian positions is important and decisions to not take part in certain discussions or negotiations cannot be taken lightly. As one interviewee stated and others conceded, "both at an international level and at a regional level [IC's absence] is quite harmful." According to another, "while a member state's reputation takes decades to develop, it can be harmed in a single conference."

Comparison of Canadian Delegation Size vs Other ITU Member States

The PM Analysis report indicates that Canadian participation is similar to that of Australia. For example, at the 2013 ITU-R WP 5A meeting, Canada's delegation was comprised of four representatives whereas Australia had five delegates. China, on the other hand, consistently sends more delegates to most meetings.^{Footnote 61} The majority of interviewees felt that Canada does not send enough delegates to ITU meetings in comparison with other influential member states. When asked about the optimum number, interviewees said that it depends on the type of meeting and what is on the agenda. At larger conferences where the critical discussions take place at concurrent sessions, a larger number of experienced IC delegates is necessary to cover these sessions. Program staff reported that in the past, delegations to key meetings included an IC representative to provide secretariat services to assist the whole delegation. This practice was reportedly used at WRC in 2003 and 2007 and at the PP conference in 2002 and 2006.

Although long-term data on the delegation sizes of all member states at ITU conferences are not readily available, evaluators obtained some data on delegation sizes to the last three high priority ITU conferences, the World Telecommunication Standardization Assembly, the World Radio Conference, and the Plenipotentiary (see Table 7). Compared with the most active nations, Canada's total delegation sizes, including both IC staff and ITU Sector Members and Associates, have been declining, the numbers of IC staff in particular. The exceptions to this trend includes those years where Canada had a candidate running for the one of the top five management positions at the ITU and/or the travel costs were lower due to the proximity of the location.

Figures 3 to 5 compare Canada's total delegation sizes at these three major conferences with the average attendance of the 14 other influential nations listed in Table 7. The graphs show that the size of the Canadian delegations (which includes Sector Members and Associates) is lower than the average, and that the number of IC representatives to these major conferences is declining.

Figure 3: Canadian/IC Delegates Attendance vs. Average Attendance at WTSA

Figure 4: Canadian/IC Delegates Attendance vs. Average Attendance at WRC

Figure 5: Canadian/IC Delegates Attendance vs. Average Attendance at PP

When we compare the number of Canadian government representatives at these same conferences (e.g. the WTSA, the WRC and the PP) with the numbers of government representatives from a small selection of other countries^{Footnote 62}, we find that Canada sends fewer government representatives than the average and that Canada primarily sends IC representatives (whereas other countries send representatives of several government departments or agencies). At the WTSA, IC is the only Canadian government department represented. This was confirmed by interviewees who indicated that other Canadian government departments rarely attend the ITU conferences leaving the negotiations to IC. Figure 6 below shows attendance at the last three Plenipotentiary conferences of government representatives of Canada and a selection of other countries. Canada has been sending fewer delegates and even fewer IC delegates to the Plenipotentiary conference in recent years.

Figure 6: Government Representatives as Part of Canadian Delegations to the Plenipotentiary Conferences Compared to the Average for a Selection of Countries

Based on several lines of evidence, the preparatory work done before these conferences and the delegations that attend them require highly competent and knowledgeable representatives in order to participate fully in the discussions or negotiations. While the number of IC representatives attending the study groups and preparatory meetings for the ITU-R sector fluctuates on a regular basis, (see Appendix I for more details), program records show that in the last four years IC is sending fewer representatives to the ITU-T preparatory meetings (leading to the WTSA), including the highest priority preparatory meetings. See Figure 7 below.

Similarly, the number of representatives IC sends to the high priority preparatory meetings for the ITU Plenipotentiary Conference has declined in the last four years. See Appendix J. Since 2011, only one person has participated in CITEL's Working Group on preparations for the PP. This individual is the Chair of the WG, and consequently is not able to fully advance Canada's position at these regional meetings.

Figure 7: Attendance by IC Delegates at High Priority Meetings and Study Groups for ITU-T

Interviews also cautioned that if Canada continues to decrease its presence at the ITU, it will not be able to influence the negotiations as much as other countries. According to one interviewee, "[other] countries…have made their presence at the ITU [meetings and conferences] a priority and they send large delegations to increase their level of influence." Since much is based on negotiation and compromise at ITU meetings and conferences, it will be difficult for Canada to influence issues if fewer experienced IC representatives are present. In the 2013 stakeholder consultation, stakeholders drew attention to the importance of maintaining the leadership role that Canada has played for many years at the ITU (resulting from Canada's history of innovation in telecommunications and from the skills of Canada delegations to resolve complex and sensitive issues), as well as the importance of building on the competitive advantages Canada has established with other countries.

3.3.2 Are there improvements IC could make with respect to its involvement in the ITU?

Size of the delegations

As previously discussed, program records and ITU delegation lists show that the size of the Canadian delegations to the ITU has been declining. The 2013 stakeholder consultation, the interviews, the rapid impact survey the PM Analysis report confirmed this finding. One interviewee stated: "one person is juggling 10 issues whereas in the past they would normally have 4-5 people for that many issues." The PM Analysis report noted that IC is currently unable to cover multiple issues at concurrent ad hoc and plenary sessions and that human resources for conferences are greatly stretched. This is despite increased efficiency measures such as the strategy for prioritizing issues at ITU meetings. Furthermore, one of the two Canadian representatives may be obligated to fulfil duties as chair or vice-chair of a meeting. Open-ended comments in the rapid impact survey included the observation that the reduction in IC's participation at the ITU is limiting the ability of IC to lobby on behalf of Canada and Canadian industry.

Stakeholders recommended sending more IC delegates to ITU conferences and preparatory meetings. In addition, interviewees indicated that, for ITU-T, there should be at least one IC representative at each study group to speak on behalf of the Canadian government. An interviewee stated, and others concurred, "If there is no one from IC at ITU-T meetings, even if there are still sector members there…it weakens our position. It's just the nature of the organization [the ITU] that having a member state representative with you makes a lot of difference."

Succession planning

Succession planning and management involves an integrated, systematic approach aimed at identifying, developing, and retaining talent for key positions and areas in line with current and projected business objectives. The 2009 evaluation recommended the development and implementation of a succession plan for this program. To date no succession plan has been put in place for this program. Limitations on travel and delegation sizes have hampered the implementation of succession planning.

The 2013 stakeholder consultation also identified an urgent need for industry and government stakeholders to develop succession plans in relation to their ITU activities. Such planning is necessary to maintain Canada's leadership position and ensure that Canadian participation continues to be effective in the future. Another related recommendation and important point emerged from the stakeholder consultations which encouraged,

 "using all aspects of the domestic Canadian National Organization for the ITU (CNO) process, including the roles played by the CNO Secretariat and the National Study Group chairs and members, to help build the capacity of new participants from both industry and government to engage effectively in ITU activities."

All IC interviewees agreed that succession planning is critical to build capacity and expertise and acknowledged that the 2009 evaluation recommendation has not been implemented due to financial restraints. Furthermore, interviewees expressed that IC is "more vulnerable now compared to five years ago" because more senior staff are approaching retirement age and few experienced people are ready to take their places.

IC interviewees observed that staff members who attended ITU conferences over time have developed knowledge and competency on the technical issues, but have also built strong professional networks and learned how to negotiate with other administrations. Therefore, junior staff or staff in training should be accompanying more experienced IC staff to ITU meetings and conferences to learn these skills and to build their networks. One interviewee pointed out the need for inexperienced staff to shadow their colleagues: "when many of us retire, [new] people will not have enough experience…if you go there the first time alone, you will be lost…it's not the delegates from the other countries who are going to show you the ropes."

Performance reporting

The 2009 evaluation recommended developing an improved methodology for measuring and reporting the success of IC's activities at the ITU. This recommendation referred specifically to the pre and post conference reports that were found to provide anecdotal information but no matching of Canadian positions to outcomes. Commendable efforts were made to improve pre and post conference reports through an online survey that conference attendees have been completing before and after conferences for approximately 1.5 years. However, the survey is long and burdensome leading to end user data entry issues and hence data quality issues. Plans for how to prepare the data for analysis, what analyses to perform, and how to report on the data were not prepared. The tool does not allow for quick and periodic performance reports to be generated. In addition, further work is needed to align achieved outcomes for conferences with the program's expected outcomes as per the program's logic model. The program should determine what performance data is essential for periodic and evaluative reporting and then design a robust database (as opposed to a survey) with the ability to conduct queries and generate quick, automated reports on selected variables or criteria.

Evaluators reviewed documentation related to the eight conferences that took place during the timeframe of the evaluation (e.g. conference preparatory documents; post conference delegate reports; and, post conference survey reporting completed by heads of delegation). Inconsistent reporting made it problematic to align objectives to the expected results of the program. As well, due to the differences in report characteristics, it was challenging to prioritize objectives and determine achievement of pre-conference objectives for the various types of conferences.

Interviewees acknowledged the need to improve reporting processes. Extensive reporting is taking place, but it is burdensome for IC staff. Currently, reporting includes delegation reports from conferences and meetings, summary technical reports, briefing notes, informal emails to senior management and post conference survey reporting, which some interviewees found onerous and of limited value. Some interviewees provided suggestions to simplify reporting and to make it more consistent. One interviewee proposed providing a list of questions, some pre and some post, for each conference. Others suggested providing summary reports of the achievements of conference objectives in light of IC's priorities.

4.0 Additional Priorities

Subsequent to the research conducted for the evaluation, program staff identified additional priorities for IC's involvement in the ITU. These priorities were not previously included in the program's logic model or foundational documents and therefore did not lead to the development of lines of inquiry for the evaluation. However, these priorities are important to discuss because they are already part of the program's day-to-day activities, consume program resources, and have implications for program outcomes.

4.1 Governance and Accountability

Program managers indicated that a key priority for the program is to encourage the ITU to adopt measures to strengthen transparency and financial accountability so that IC may ensure it receives value for its contribution to the ITU. The program's foundational documents report that the ITU has stringent internal controls and internal mechanisms which include audit, evaluation, inspection and investigation services that ensure its visibility and transparency, (such as the ITU Financial Operating Report, approved by the ITU Council). Further, given that the financial system in the ITU is characterized by high standards of technical accountability and transparency, Canada's financial support for the organization is considered to be low risk.

On the other hand, according to one interviewee, the overall responsibility for governance and accountability of UN bodies lies with the Department of Foreign Affairs, Trade and Development (DFATD), even though IC is responsible for the governance and accountability the ITU, and Canada's contribution to the organization. This becomes problematic for both departments, when the contribution to the ITU is modified posing a risk to Canada's treaty obligations for which DFATD has responsibility.

This raises the point that much of the program's activities are devoted to governance activities with respect to how the ITU is run and the agendas set for work on the radio-communications, telecommunications and development issues. Much of the program's resources are devoted to attending Council meetings, Plenipotentiary meetings, developing contributions for these meetings, running for election to these bodies and other forms of international engagement which are important to business development, standardization and regulations. Yet, neither the IC-ITU program logic model nor its program descriptions explicitly identify any outputs or outcomes specific to this line of activity. Given the importance of this priority, the program should consider clarifying the responsibility for governance and accountability and redesigning its guiding and strategic documents to ensure that these objectives are agreed upon, measured and achieved.

4.2 Issues Surrounding the Mandate of the ITU, Internet Governance and IC's Involvement in the ITU-D Sector

Program managers at IC identified that another priority for the program is to ensure that the ITU does not expand its mandate to include responsibility for Internet governance and cybersecurity matters. They report that this one of the key elements of Canada's internet governance strategy articulated in 2010, and that "Canada strongly supports the multi-stakeholder-led model of internet governance, a model that is key to ensuring future economic opportunities for Canada and Canadians."

As this evaluation took a calibrated approach and focused on reporting on the achievement of the immediate outcomes identified in the program's logic model and because no additional research questions were proposed by the program during the evaluation planning stage, the evaluation did not investigate these issues. Some preliminary research into these issues, following the conduct of the evaluation, suggests that the topic could merit a study on its own. At the very least, the program should determine if the priority should be added to the program's foundational documentation and logic model so as to guide program operations, future performance measurement and evaluations. This priority might be specific to Internet governance (an important worldwide debate) and cybersecurity matters or a more generic objective to influence the ITU such that it remains focused on its core mandate and competencies as determined by its member organizations.

While the evaluation found that the ITU-D sector has not been a high priority for IC, program managers indicated to evaluators that IC should stay involved in the ITU-D meetings and conferences as well as in development and capacity-building activities in order to leverage or earn the support of developing countries for a multi-stakeholder led model of Internet governance. Evaluators suggest that this priority be made more explicit so that it guides the program's strategy and operations and leads to more effective involvement in the ITU-D sector and/or more resources being devoted to this line of activity.

5.0 Conclusions and Recommendations

5.1 Conclusions

Relevance

Involvement in the ITU continues to be relevant for Canadians. The ITU is the only intergovernmental agency where the nations of the world can negotiate the international allocation of radio spectrum and satellite orbital positions, and is still one of the most important telecommunication standards development organizations in the world. The work of the Radiocommunication sector (ITU-R) and the Telecommunication Standardization sector (ITU-T) are relevant to IC's priorities. Emphasis should continue to be focused on the work of these sectors. While IC has made relatively few contributions to the Telecommunication Development (ITU-D) sector, there is still an important role for Canada to play in this area.

IC's involvement in the ITU is in line with the federal government's commitments to Canada's digital future and economic growth as well as its commitment to protection of the North (e.g. through the use of sophisticated satellite earth monitoring and surveillance systems). The work of IC with the ITU contributes to the achievement of IC's strategic outcome of an efficient and competitive marketplace and lays the foundation for the telecommunications industry and communications in our society. Involvement in the ITU clearly falls within the mandate of IC and the responsibility of the Canadian government to defend Canada's interests in telecommunication and radiocommunication issues.

Performance

IC has been successful to date in achieving the majority of its objectives at the ITU-R and ITU-T meetings. IC's success is due to the competence of the staff involved, the well-developed preparatory processes, and the reputation that Canada's delegations have had at international conferences. Nevertheless, the ability of IC to continue to play an influential role at the ITU is becoming more difficult due to the limited delegations being sent to meetings and conferences.

IC and other governmental stakeholders along with Canadian industry stakeholders think that the activities of the ITU are very important and beneficial to their organizations. Industry stakeholders report that the intended benefits to industry from the program including the development of new services and technologies, the achievement of economies of scale, interoperability and protection from harmful spectrum interference, are being realized.

The evaluation found that the program is currently operating efficiently and was unable to identify further efficiencies. Canada's contribution to the ITU has been reduced by $2 million to bring it in line with the contributions of comparable nations. To address reductions in resources, the program has put in place efficient issue prioritization processes and meeting preparatory processes. However, the evaluation found that the ability of IC to continue to effectively advance Canadian interests at the ITU is becoming more challenging with the limited delegation sizes and as IC staff experienced in ITU matters approach retirement. The recommendation from the previous evaluation to implement a succession strategy has not been completed, making this requirement more emphatic now.

Improvements to the program could be made with respect to reporting on outcomes. Performance reporting should be simplified and streamlined such that the progress made by IC at advancing Canadian interests at the ITU can be more effectively and easily communicated to diverse audiences including IC senior management, program management, and industry stakeholders.

Finally, during the evaluation program staff identified additional priorities that were not part of the initial evaluation design. These priorities related to governance and accountability, and the mandate of the ITU, notably relating to internet governance. These priorities should be included in future performance reporting exercises.

5.2 Recommendations

The findings and conclusions of the evaluation lead to the following recommendations.

1. The program should adopt a strategic approach to determining the size and composition of the IC part of the Canadian delegation for ITU-related meetings. This approach should include:
   1. Implementing a succession strategy for IC staff working on ITU matters and taking into account this strategy when composing IC delegations so that the next wave of IC representatives can obtain experience and develop relationships with senior staff and key delegations at ITU;
   2. Continuing the prioritization exercise to determine which meetings and events are of most importance to Canadian and IC interests and also taking into account the nature of the meeting/event and IC participation required (e.g. negotiation, networking, other duties); and
   3. While recognizing that IC should continue being the lead, the program should approach other government departments, where appropriate, to seek additional support and contributions to Canada's participation at the ITU.
2. The program should build on the improvements made to its measurement and reporting on outcomes. For the purpose of communicating outcomes to senior management, stakeholders and for performance measurement and evaluation purposes, the program should consider preparing high level summaries of all the delegate/post conference reports. The summary reports should align Canada/IC's achievements at conferences, objectives for the conferences and the expected outcomes of the program overall. In addition, to reduce the reporting burden on the heads of delegations and to effectively capture the essential performance data, the pre and post conference performance measurement survey should be streamlined and improved and ideally, replaced with a more robust database.
3. The program should examine its foundational documents (e.g. program profile, logic model, Performance Measurement Strategy) that guide the planning, operations and performance monitoring of IC's involvement in the ITU so that program priorities are identified and made explicit.

Annex A - Management Response and Action Plan (MRAP)