Evaluation of ISED's contribution to Mitacs

This publication is available online at www.ic.gc.ca/eic/site/ae-ve.nsf/eng/h_03851.html.

To obtain a copy of this publication or an alternate format (Braille, large print, etc.), please fill out the Publication Request Form at www.ic.gc.ca/Publication-Request or contact:

Web Services Centre
Innovation, Science and Economic Development Canada
C.D. Howe Building
235 Queen Street
Ottawa, ON K1A 0H5
Canada

Telephone (toll-free in Canada): 1-800-328-6189
Telephone (Ottawa): 613-954-5031
TTY (for hearing-impaired): 1-866-694-8389
Business hours: 8:30 a.m. to 5:00 p.m. (Eastern Time)
Email: info@ic.gc.ca

Permission to Reproduce

Except as otherwise specifically noted, the information in this publication may be reproduced, in part or in whole and by any means, without charge or further permission from Innovation, Science and Economic Development Canada (ISED), provided that due diligence is exercised in ensuring the accuracy of the information reproduced; that ISED is identified as the source institution; and that the reproduction is not represented as an official version of the information reproduced, nor as having been made in affiliation with, or with the endorsement of, ISED.

For permission to reproduce the information in this publication for commercial purposes, please fill out the Application for Crown Copyright Clearance at www.ic.gc.ca/copyright-request or contact the Web Services Centre mentioned above.

© Her Majesty the Queen in Right of Canada, as represented by the Minister of ISED, (insert year of publication).

Aussi offert en français sous le titre Évaluation de la contribution d’ISDEC à MITACS

PDF version

848 KB, 27 pages

Table of contents


List of acronyms

IP
intellectual property
ISED
Innovation, Science and Economic Development Canada
Mitacs
Mathematics of Information Technology and Complex Systems
NSERC
National Sciences and Engineering Research Council
OECD
Organization for Economic Cooperation and Development
R&D
research and development
SIS
Science and Innovation Sector

Table

Table 1: ISED's contributions to Mitacs, by program


List of figures


Executive summary

Program overview

Mitacs is a national, not-for-profit, research organization dedicated to promoting high-quality research and innovation by building linkages between academia and industry through internships. Innovation, Science and Economic Development (ISED) Canada supports three Mitacs internship programs designed to increase industry-academia research collaboration, boost investment in industrial research, improve the employability of its interns and encourage highly-qualified graduates to remain in Canada.

Evaluation purpose and methodology

The purpose of the evaluation was to assess the relevance and performance of ISED-funded Mitacs programs. The evaluation relied on five data collection methods, including document and literature reviews, interviews, exit and outcome surveys of program participants conducted by Mitacs, and case studies of universities and industry partners.

Findings

Relevance

The evaluation found that Mitacs responds to the ongoing need to increase Canadian innovation by using internships to facilitate linkages between academia and industry and increase international collaboration. Strong demand for Mitacs programs is reflected in the growing number of internships.

Mitacs clearly supports federal government priorities to invest in research and innovation to strengthen the Canadian economy. The program also seeks to foster the next generation of researchers through collaboration between the private sector and academia. Further, ISED's support for Mitacs is consistent with the federal government's responsibility to foster science and technology.

Performance

Evidence shows that Mitacs has achieved its expected results. In the short-term, Mitacs internships have contributed to collaboration and resource sharing between academia and industry. They have allowed for knowledge transfer to the private sector and enabled some interns to use their research to further their academic work. The work of Mitacs interns has supported the development of new products, processes and services for industry partners. Mitacs' use of university-based business development officers is seen by stakeholders as an important contributor to program success.

The evaluation also found that over the medium-term Mitacs programs are helping to build research linkages between industry and academia in Canada, and between researchers in Canada and abroad that continue beyond the end of Mitacs projects. The program has contributed to investments in industrial research and development (R&D) by industry partners during and after the conclusion of their Mitacs projects. There is evidence that Mitacs has improved the employment prospects of its interns by providing on-the-job experience and networking opportunities. A number of companies and organizations have hired the graduate students and postdoctoral fellows involved in their Mitacs R&D projects. Finally, Mitacs is helping to keep domestic and international graduate students and fellows in Canada after they complete their studies.

Mitacs administration costs are within an acceptable range and it has made efforts to improve efficiency through upgrades to its information management systems. Mitacs has robust performance measurement systems in place. However, it did not include a way of measuring the program's overall contribution to academia-industry collaboration in the broader Canadian context.

As an organization, Mitacs is leveraging considerable resources, averaging $3.8 dollars for every dollar contributed by ISED. As ISED's total contributions to Mitacs will be increasing by nearly four times in the next five years as compared to the evaluation period, it may be prudent for SIS to conduct an external review of the organization to ensure it has the people, systems and processes in place to successfully deliver the increased programming.

Recommendations

The findings of the evaluation led to the following recommendations.

  1. SIS should consider conducting an external review to assess whether Mitacs has the resources, systems and processes in place to successfully manage future growth.
  2. SIS should consider working with Mitacs to develop the measures against which to assess Mitacs' contributions to overall industry-academia collaboration in Canada.

1.0 Introduction

This report presents the results of an evaluation of ISED's contribution to Mitacs.

The purpose of the evaluation was to assess the relevance and performance of Mitacs. The report is organized in four sections:

  • Section 1 provides the program context and profile of Mitacs;
  • Section 2 presents the evaluation methodology along with a discussion of data limitations;
  • Section 3 presents the findings pertaining to the evaluation issues of relevance and performance; and
  • Section 4 summarizes the study's conclusions and provides recommendations.

1.1 Program overview

Mitacs is a national, not-for-profit, research organization dedicated to promoting high-quality research and innovation by building linkages between academia and industry through internships. Mitacs was founded in 1999 and initially focused on supporting applied industrial research in mathematical sciences and associated disciplines.Footnote 1 Mitacs launched its first internship program in 2003 and opened its internships to all disciplines with the start of the Accelerate program in 2007. Mitacs currently delivers three internship programs with ISED funding, each of which are described below. Overall, ISED funding represented 23% of Mitacs total income during the evaluation period.Footnote 2

Mitacs Accelerate is a national internship program that provides graduate students and postdoctoral fellows with four to six month internships at a company, under the guidance of a professor. The intern spends approximately half of his/her time on site with the partner company, gaining a better understanding of and working on the research project. The balance of the intern's time is spent at the university, further advancing the research under the guidance of a faculty supervisor. Students receive an award of at least $10,000 during each internship placement.Footnote 3 Industry contributions start at $7,500 and are matched dollar for dollar.Footnote 4 ISED began funding Accelerate in 2012–13. The program is expected to achieve the following results:

  • Increase collaboration and knowledge transfer between academia and industry, in various sectors of the Canadian economy including the non-profit sector;
  • Create job opportunities for graduate students and post-doctoral fellows in various disciplines;
  • Improve employability of graduate students and post-doctoral fellows in their field;
  • Increase retention of domestic and international graduate students and post-doctoral fellows in Canada after completion of studies; and
  • Increase investment in research and development and innovation of participating companies.

Mitacs Elevate is a postdoctoral fellowship program which provides a foundation of research, business, entrepreneurship and scientific management skills to recent PhD graduates.Footnote 5 Elevate fellows work on a joint industry-academia research project for two years. Fellows lead the collaboration and spend about half of their time at the university and half at the company. In addition, Mitacs provides fellows with business skills training so they are well positioned to take on research and development (R&D) leadership roles at the end of the program. During a fellowship, fellows receive a stipend of at least $50,000 per year, of which Mitacs covers half. ISED began funding Elevate in 2014–15. Its expected results are to:

  • Improve employability of post-doctoral fellows in their field;
  • Increase retention of PhD holders in Canada and create a highly effective talent pool ready to lead innovation;
  • Increase opportunities for businesses to identify and engage with fellows, and benefit from the wealth of ideas and solutions these people bring; and
  • Connect researchers from academia and industry to develop innovative solutions to Canada's industrial and societal challenges.

Mitacs Globalink program is composed of four sub-programs, which create opportunities for international students to conduct research in Canada and for Canadian students to conduct research in other countries. Globalink helps to build international networks and contributes to the reputation of Canadian education and research by linking foreign students with research expertise, training and networking opportunities. ISED began funding Globalink in 2013-14. It is expected to achieve the following results:

  • Increase the number of foreign students undertaking research projects in Canada and applying to pursue graduate or postdoctoral studies in Canada; and
  • Increase the number of Canadian students participating in research and educational opportunities abroad.

1.2 Delivery and governance

Mitacs is governed by a Board of Directors and led by the Chief Executive Officer and Scientific Director. The majority of Board members are drawn from industry with additional members from Mitacs partner universities. The key functions of the Board are to approve Mitacs business plans and budgets, and to provide guidance on the management and direction on the strategic plans and objectives of Mitacs.

A key aspect of Mitacs program delivery is its business development model. Specifically, business development officers network with university professors, students and industry partners to identify potential research projects, match university professors, students and industry partners to proposed projects, and provide assistance with the application process. Business development officers are situated on-site at Mitacs partner universities.

Mitacs applications are submitted to the Mitacs Research Council, an independent scientific community of the Board of Directors, which is responsible for ensuring that proposed projects have the potential to produce meaningful findings, and that student-interns are not being used for routine business operations. Project applications are assigned to arms-length experts from relevant academic disciplines for review.

The Program Coordination Branch within the Science and Innovation Sector (SIS) is responsible for administering ISED's contributions to Mitacs, as well as the ongoing management and oversight of the funding.

1.3 Program resources

The table below presents total ISED contributions to Mitacs by program.  ISED's contribution to Mitacs totalled approximately $56 million during the evaluation period. Of note, Budget 2017 committed $221 million in ISED funding to Mitacs over five years starting in 2017–18.

Table 1: ISED's contributions to Mitacs, by program
Fiscal Year Accelerate Globalink Elevate Total Contribution
2012–13 $5,000,000 $5,000,000
2013–14 $7,000,000 $5,975,000 $12,975,000
2014–15 $8,975,000 $7,000,000 $3,000,000 $18,975,000
2015–16 $7,000,000 $7,000,000 $5,000,000 $19,000,000
Total $27,975,000 $19,975,000 $8,000,000 $55,950,000

1.4 Logic model

The logic model in Figure 1 depicts the program theory for Mitacs Accelerate, Globalink and Elevate programs. One logic model was developed in 2014 as part of the Performance Measurement Strategy for all three programs, recognizing that while there are some differences across the programs, there are also significant commonalities in terms of activities, outputs and intended impacts. Note in the logic model and throughout this report "industry" and "industry partners" refers to both private sector companies and non-profit organizations that participate in Mitacs programs.

Figure 1: Logic Model for ISED's contribution to MitacsFootnote 6

Mitacs Logic Model (the long description is located below the image)
Description of Figure1

description

2.0 Methodology

2.1 Evaluation scope and objectives

The purpose of this evaluation is to assess the relevance and performance of ISED's contribution to Mitacs as required by section 42.1(1) of the Financial Administration Act. The evaluation covers ISED contributions to Mitacs during the period of April 1, 2012 to March 31, 2016.

2.2 Evaluation question

Based on the expected results of ISED's funding agreements and the program's 2014 Performance Measurement Strategy, this evaluation addresses the following questions:

Relevance

  1. Is there a continued need for ISED's contribution to Mitacs?
  2. To what extent are the programs aligned with the priorities of the federal government?
  3. Is there a necessary role and responsibility for the federal government in providing funding for Mitacs?

Performance

  1. In what manner and to what extent have the programs led to increased collaboration and knowledge transfer between academia and industry?
  2. In what manner and to what extent have the projects provided innovative solutions to industry needs and issues?
  3. In what manner and to what extent have the programs increased research linkages with both domestic and international partners?
  4. In what manner and to what extent have the programs led to increased investment in industrial research, development and innovation?
  5. To what extent has the program improved employability of graduate students and postdoctoral fellows?
  6. In what manner and to what extent have the internships led to increased retention of domestic and international graduate students in Canada after completing their studies?

Efficiency and Economy

  1. How efficiently and economically is the program being delivered?

2.3 Data collection methods

The multiple lines of evidence used to address the evaluation questions are described below.

Interviews
A total of 32 formal interviews were conducted with program stakeholders, including industry partners (14), universities (4), co-funding partners (5), Mitacs staff (5), ISED senior management (2), a representative of another government department (1) and a subject matter expert from the Conference Board of Canada (1). Goss Gilroy Inc. was contracted to conduct additional interviews with unsuccessful applicants (13) to assess Mitacs efficiency and economy as it relates to the program's application and review process.

Document Review
The document review provided an understanding of Mitacs, its relevance and achievement of expected outcomes. These documents included program foundational documents; government priority setting documents (e.g. Budgets, Reports on Plans and Priorities); and Mitacs documentation including Annual Performance Reports.

An important set of information within Mitacs Annual Reports were the program's exit surveys. These exit surveys are to be completed at the end of each internship by project participants (i.e. interns, academic supervisors and industry partners) and capture the short-term results of research projects. Exit survey response rates averaged 55%.Footnote 7

Literature Review
The literature review addressed the core evaluation issues of continued need and federal roles and responsibilities. The literature reviewed included scholarly (peer-reviewed) literature as well as relevant 'grey literature', such as working papers, conference proceedings, government documents, and commissioned research reports. The review was also informed by both Mitacs and research conducted in collaboration with the ISED library.

Outcome Surveys
Mitacs regularly conducts surveys of former program participants (i.e. interns and industry partners). These surveys were a critical line of evidence for the evaluation as they capture the medium-term results of the research projects supported by the program. The surveys and their methodologies are briefly summarized below:

  • 2015 Accelerate Industry Partner Survey: This survey targeted all for-profit companies that participated in the Accelerate program from inception to 2013. A total of 253 individuals responded, with a response rate of 19%.
  • 2014 Accelerate Intern Survey: This survey targeted former interns in the program. The survey was available online from October 31, 2013 to November 29, 2013. A total of 686 former interns responded from 9 provinces and 44 Canadian universities. The response rate was 27%.  
  • 2016 Elevate Fellows Survey: This survey targeted all Elevate fellows who participated in the program between 2010 and 2015. A total of 92 Elevate fellows responded, with a response rate of 38%.
  • 2015 Globalink Intern Survey: This survey targeted former interns as well as former applicants to the Globalink Research Internship program; the latter provided a control group for comparative purposes. A total of 566 former interns responded with a response rate of 41%, and a total of 341 former applicants responded with a response rate of 9%.

Case studies
Goss Gilroy Inc. was contracted to conduct case studies of three universities and three industry partners. These case studies were to provide a detailed perspective of the impacts of Mitacs and potential improvements to enhance program effectiveness. Each case study included a document review and six interviews with business development officers, university representatives, academic supervisors, and interns.

2.4 Limitations

Lack of baseline data
This evaluation focused on whether ISED-funded Mitacs programs were working as intended and achieving their expected immediate and intermediate outcomes. However, it did not examine or try to quantify the program's contribution to the overall level of academia-industry collaboration in Canada. This would require the development of a baseline against which to measure progress, something that was beyond the scope of this evaluation. The evaluation makes a recommendation to strengthen this area.

Limited performance information on Mitacs Elevate program
Outcome surveys for Elevate industry partners were not available at the time of the evaluation. Elevate accounts for the smallest portion of ISED funding, nevertheless the evaluators sought to interview industry partners that used Mitacs Elevate program.

3.0 Findings

3.1 Relevance

3.1.1 Is there a continued need for ISED's contribution to Mitacs?

Key finding:
Mitacs responds to the ongoing need to increase Canadian innovation by using internships to facilitate linkages between academia and industry and increase international collaboration. Strong demand for Mitacs programs is reflected in the growing number of internships.

The evaluation examined this question from two perspectives: Mitacs potential to contribute to economic growth and the demand for ISED-funded Mitacs programs.

In the literature, productivity is recognized as a key driver of income growth and improved living standards for Canadians.Footnote 8 Footnote 9 However, over the course of the last decade, Canada's productivity growth has fallen below the average of other developed countries.Footnote 10 A large body of research attributes Canada's below-average productivity growth to limited Canadian business innovation.

The Innovation Canada: A Call to Action report notes that Canada invests significantly in university R&D and is effective in generating new ideas and knowledge, but faces challenges in transforming resulting ideas into commercial applications. The Council of Canadian Academies explains that Canadian businesses have lacked the incentives and awareness required to adopt innovation-based business strategies, and therefore have not applied Canada's research excellence in commercial settings. Similarly, the Conference Board of Canada explains that most university research is not conducted to support business innovation, and that universities also lack incentives to collaborate with businesses.Footnote 11 The Conference Board therefore argues that "Canada needs to find a way to bridge the valley between university research and business innovation".Footnote 12 Mitacs addresses the need to increase collaboration between universities and businesses by supporting joint R&D projects through its internship programs.

The literature also points to the need for Canada to attract, develop and retain highly-qualified people to work in R&D. For example, the Innovation Canada: A Call to Action report notes that "Canada's future as an innovation-based economy depends on ensuring there are sufficient numbers of talented, educated and entrepreneurial people". It emphasizes the need for a collaborative approach that brings together post-secondary institutions, government and industry to ensure appropriate recruitment, training and deployment for industrial innovation needs. By providing on-the-job experience to graduate students and post-doctoral fellows through its Accelerate and Elevate programs, Mitacs provides a pathway for businesses to access and recruit the highly qualified staff needed to spur R&D.

In addition, Canadian business expenditures on research and development (BERD) - an important indicator of business commitment to innovation, as well as a crucial input to many high-value added types of innovationFootnote 13 - have been declining since 2006 relative to other countries in the Organisation for Economic Co-operation and Development (OECD).Footnote 14 Canada was recently ranked last among sixteen peer countries in terms of business spending on R&D.Footnote 15 A couple of interviewees explained that Mitacs creates an incentive for businesses to invest in R&D by covering 50% of project costs. Further, nearly two-thirds of industry partners interviewed said that research projects undertaken with Mitacs funding would not have gone ahead without the program's support.

The literature also suggests that to compete effectively in the global economy, Canadian education and research must have a strong international focus. According to the 2012 report of the Advisory Panel on Canada's International Education Strategy, "international research collaboration makes a valuable contribution to Canada's innovation agenda and economic future".Footnote 16 Mitacs Globalink program supports many of the activities that underpin the internationalization of research. In addition to attracting international students to Canada and encouraging Canadian students to pursue international learning experiences, the program builds international research networks by developing partnership agreements with priority countries determined in consultation with ISED and Global Affairs Canada.Footnote 17

There is strong demand for ISED-funded internships. As ISED has expanded its support, Mitacs programs have largely met and at times exceeded the internship targets in contribution agreements. Overall, by 2015–16, ISED funding supported about 6,800 internships. This included 4,069 Accelerate internships, 224 industry placements of post-doctoral fellows through Elevate and about 2,500 Globalink international research placements for foreign students in Canada and Canadian students abroad.

Thus, both program demand and the need to encourage innovation by Canadian industry provide strong evidence of an ongoing need for Mitacs programs. However, given the significant increase in ISED funding for Mitacs programs, it will be important for ISED to monitor whether the high demand for these programs is sustained as they continue to expand.

3.1.2 To what extent is Mitacs aligned with the priorities of the federal government?

Key finding:
Mitacs clearly supports federal government priorities to invest in research and innovation to strengthen the Canadian economy. The program also seeks to foster the next generation of researchers through collaboration between the private sector and academia.

There is strong evidence ISED support for Mitacs programs align with federal priorities to invest in research and innovation, as well as to develop a pool of highly-qualified workers.

Ministerial Mandate letters identify overarching goals and priorities to support research and innovation. The Minister of ISED is to help Canadian businesses grow, innovate and export in order to create good quality jobs and wealth for Canadians and is specifically tasked with supporting business innovation. The Minister of Science is to support scientific research to create sustainable economic growth and support and grow the middle class. Mitacs delivers on both of these elements by encouraging businesses to leverage research knowledge (including scientific research) produced by academia to spur business innovation.

Federal budgets have also consistently identified research and innovation as drivers of economic growth, and allocated funding to Mitacs for this purpose. In Budget 2014, the government provided $8 million to expand Mitacs Elevate program to ensure that Canadian businesses had a highly-qualified pool of managers to lead R&D activities and increase productivity. Budget 2015 further committed $56.4 million over four years to expand R&D internships through Mitacs Accelerate program. Again in Budget 2016, the government pledged $14 million over two years to expand Mitacs Globalink program to promote Canada as a destination of choice for study and the conduct of world-class research. Finally, Budget 2017 announced $221 million in funding over five years starting in 2017-18 to renew and expand Mitacs programs. This funding will support up to 10,000 work-integrated learning placements.

Mitacs aligns with the federal government's Inclusive Innovation Agenda announced in June 2016. Specifically the program brokers internships that foster collaboration between the private sector and academia. This in turn develops a future highly-qualified workforce to foster innovation in Canada. Also, Mitacs internships are a key component of the government's Skills and Innovation Plan outlined in Budget 2017 which better aligns education and training with industry needs by increasing work-integrated learning opportunities.

Finally, Mitacs Globalink program supports Canada's 2014 International Education Strategy by contributing to the number of international students in Canada, as well as the number of international students who remain in Canada after graduation.

3.1.3 Is there a responsibility and necessary role for the federal government in providing funding for Mitacs?

Key finding:
ISED's support for Mitacs is consistent with the federal government's responsibility to foster science and technology.

ISED's support for innovation through Mitacs aligns with the federal government's responsibility to foster the development and use of science and technology in Canada and stimulate investment as laid out in the Department of Industry Act, 1995, which provides the Minister with the powers, duties and functions related to industry, technology and science in Canada. More specifically, it gives the Minister a mandate to foster and promote science and technology in Canada, to encourage the most efficient and effective use of science and technology, and stimulate investment.

Provincial partners interviewed were highly supportive of federal government funding for Mitacs. Some interviewees explained that federal support allows the program to have national reach with representation even in smaller provinces and territories. Interviewees also said that federal funding has been instrumental in leveraging funding from provincial governments and other stakeholders. Further, some interviewees noted that there were external stakeholders that would not have participated in the program in the absence of federal support.

Budget 2015 gave Mitacs primary responsibility for the delivery of federally-supported graduate level industrial R&D internships. Mitacs subsequently developed several memoranda of understanding with the National Sciences and Engineering Research Council (NSERC) to facilitate this lead role.

3.2 Performance

3.2.1 In what manner and to what extent have the programs led to increased collaboration and knowledge transfer between academia and industry?

Key finding:
Mitacs internships have contributed to collaboration and resource sharing between academia and industry. They have also allowed for knowledge transfer to the private sector and enabled some interns to use their research to further their academic work.

Mitacs outcome surveys, interviews and case studies all provide strong evidence that the program has contributed to collaboration and knowledge transfer between industry and academia. According to interviewees, Mitacs promotes a culture of collaboration between academia and industry by building awareness of research opportunities of mutual interest and exploring new research questions relevant to industry. Most of the industry partners interviewed said that participation in the program had led to increased collaboration and knowledge sharing with academia. More specifically, Mitacs outcome survey results revealed that 90% of the Accelerate industry partners who responded to the survey had developed an increased understanding of the value of R&D and innovation. Similarly, over 85% of Accelerate university supervisors who responded to the survey reported an increased understanding of the industry environment, R&D activities and challenges.

Interviews and case studies provide examples where Mitacs partnerships between industry and academia have enabled the pooling of resources. Several industry partners said that access to university equipment and laboratories through Mitacs played an important role in advancing their R&D. For instance, access to a university testing laboratory helped a Canadian firm to move a new treatment for lung cancer forward to the clinical trial stage. Another Mitacs project led to the creation of a scientific instrument to measure the mechanical properties of microcapsules. This innovation was built in a university lab, jointly funded by the industry partner and Mitacs.

Mitacs also serves to showcase research capacities and talent which may have otherwise gone undetected by industry. For example, according to one university case study, industry partners are often not aware of the skills that interns can bring to a project until the internship has occurred. In one case, the academic experience of an intern allowed him to introduce new science and techniques to improve assembly line production processes. The firm not only gained the expertise of the intern but was also able to submit questions directly to his supervising professor.

Mitacs also creates opportunities for interns and, at times, their supervisors to make academic use of their industry research. In case studies, certain universities reported that many Mitacs interns had been able to use the work carried out during their placements for their theses or dissertations and in some cases even used it to develop publications. In one case study, a water systems project supported by Mitacs led to the creation of a research network linking the university's different campuses.

The case studies and interviews also highlighted the inherent tension that exists between the interests of academia (students/fellows/professors) and private sector partners. Variation exists in the contractual agreements between industry partners and universities regarding intellectual property (IP) rights and the right to publish research in academic journals. In one case, an industry partner said that the "contract with the university says IP belongs to the company and publication rights belong to the academic. This facilitates collaboration." This policy contrasts with the experience of some students who have not been able to publish their work due to confidentiality issues. Mitacs takes no position on IP and indicates that it is to be shared according to the IP rules of the host university, unless a separate agreement is negotiated. On its website Mitacs offers to facilitate IP discussions with Canadian universities. However there may be room for Mitacs to encourage more clarity, as some students, especially post-doctoral fellows would likely seek to use their research to advance their careers in academia.

3.2.2 In what manner and to what extent have the projects provided innovative solutions to industry needs?

Key finding:
The work of Mitacs interns has supported the development of new products, processes and services for industry partners. Mitacs use of university-based business development officers is seen as an important contributor to program success.

Multiple lines of evidence indicate that Mitacs, specifically its Accelerate and Elevate programs, has been successful in providing innovative solutions to the needs of its industry partners. The industry partners interviewed for the evaluation said that having access to research talent enables them to jump-start their ideas, to build a roadmap for further investigation or implementation, and to bring their products closer to prototyping and commercialization. This access to talent is particularly important for start-up companies or small and medium-sized enterprises (SMEs) that do not have significant research capacity.

Mitacs' Accelerate outcome survey provides strong evidence of results. Of the industry partners who responded to the survey, 95% said that the program was successful in meeting their corporate needs.Footnote 18 Eighty-four percent said they had used or were still using the results, techniques and tools developed as a result of their projects. About a third of respondents said their projects had led to new or enhanced products or processes, the identification of a new market segment or an increased market share.

In the Elevate outcome survey, about a quarter of the interns who responded to the survey said their projects resulted in the development of new or enhanced processes; 20% said their projects resulted in a new or enhanced product; and 13% resulted in patent applications.

Further evidence of the impact of Mitacs projects comes from interviews with industry partner representatives, a third of whom said that Mitacs funded projects led to patent development or commercialization. According to one company representative, "We've developed 10 patents from interns and there [are] more coming" including two patents related to JavaScript security software. Other examples from the interviews and case studies include:

  • An intern working at a large Canadian manufacturer helped to develop a new functionality for aircraft antenna that has been approved for use on next generation aircraft;
  • A postdoctoral intern with a chemistry background, working at a large Canadian technology firm, supported the development of a new use for the company's core ultra-violet technology disinfecting food products, such as milk and potentially orange juice;
  • An intern working at an environmental organization was credited with collaborating with other organizations to find ways to map the movement of fish with satellite imagery, information that could help track the effects of climate change; and
  • An intern at a large Canadian manufacturing firm positioned the business to introduce new light-weight carbon fiber products.

In interviews, Mitacs staff and board attributed much of the program's success to the business development officer approach, whereby the officers actively reach out to businesses to identify problems and potential solutions. "The key to [Mitacs] success is [their] capacity to proactively identify what a business solution could be and collaboratively work towards that."

3.2.3 In what manner and to what extent have the programs led to an increase in Canadian research linkages, both with domestic industry and with international partners?

Key finding:
Mitacs programs are helping to build research linkages between industry and academia in Canada, and between researchers in Canada and abroad that continue beyond the end of their projects.

Mitacs surveys and interviews provided evidence that the Accelerate, Elevate and Globalink programs are helping to build research linkages.

In Accelerate exit surveys, industry partners and academic supervisors who responded to the survey were interested in further university-industry collaborations. Industry partner survey results showed that 82% of respondents continued to collaborate with the project's academic supervisor, after the end of the project. Further, nearly half (47%) of industry partners responding to the survey said they had engaged in new collaborations with the academic sector as a result of their participation in the program.

Elevate exit survey results also found that the vast majority of industry partners (90%) and academic supervisors (85%) responding to the survey planned to continue their collaborations or engage in new collaborations as a result of the fellowship. According to the Elevate outcome survey, more than half of the fellows responding to the survey had made new connections with companies and industrial researchers through the program and some were continuing to collaborate or conduct formal research after the end of their Mitacs project.

Of the industry partners interviewed, half indicated that their involvement in Mitacs led to further collaboration or knowledge transfer after the end of the project or anticipated this in the future. Some said their involvement also led to new or expanded projects. Evidence from case studies was somewhat weaker. Of the six case studies conducted, in two cases, project participants indicated that their industry-academia relationships were already well-established before participating in Mitacs. Nevertheless, the case studies did identify examples of ongoing collaboration. For instance, one industry partner worked closely with a Mitacs business development officer to identify university researchers in eastern Canada where they had recently opened new offices. In another case, a Mitacs project led to new research agreements between the company and the university.

Mitacs is also helping to build linkages between researchers in Canada and other countries. With Globalink, Mitacs exit surveys showed that 95% of student participants who responded to the surveys said that the internships increased their interest and involvement in international research collaborations. Further, the Globalink outcome survey showed that after the end of their projects the majority of international interns (65%) responding to the survey either had developed or intended to develop further collaborations with Canadian researchers.

3.2.4 In what manner and to what extent have the programs led to increased investment in industrial research, development and innovation?

Key finding:
Mitacs has contributed to investments in industrial R&D by industry partners during and after the conclusion of their Mitacs projects.

Mitacs' contribution to investment in industrial R&D can be assessed through both the industry partner contributions to Mitacs projects and the new downstream investment that results from those projects. Industry partner cash and in-kind contributionsFootnote 19 to Accelerate and Elevate programs have generally increased during the evaluation period. Of note, industry's cash contributions doubled from approximately $10 million in 2012–13 to approximately $23 million in 2015–16.

Results from Mitacs Accelerate outcome survey showed that 81% of industry partners who responded to the survey said they made R&D investments of $10,000 or more as result of their participation in the program (see Figure 2). Of note, 40% of companies responding to the survey reported $100,000 or more in new R&D investments a result of their Accelerate experience.

Figure 2: Value of new R&D investments by Mitacs Industry Partners as a result of Accelerate program participation

Bar chart illustrating the value of new R&D investments by Mitacs industry partners as a result of Accelerate program participation (the long description is located below the image)
Description of Figure2

description

Source: Mitacs 2015 Accelerate Industry Partner Survey

Similar results were reflected in interviews with industry partners and case studies, which identified examples of industry investments in equipment, infrastructure, personnel, training, software or materials as a result of their Mitacs projects. While specific amounts were not identified in the case studies, the value of R&D investments reported by industry partners interviewed ranged from $90,000 to $240,000.

Some interviewees also suggested that Mitacs projects have been instrumental in generating additional investments. For example, one Mitacs project led to a $20 million investment in software developed as result of the project. A mineral exploration firm said that its project led to a further $90,000 investment and a manufacturing optimization firm had a follow-on investment of more than $100,000.

3.2.5 To what extent has the program improved employability of graduate students and postdoctoral fellows?

Key finding:
There is evidence that Mitacs has improved the employment prospects of its interns by providing on-the-job experience and networking opportunities. A number of companies and organizations have hired the graduate students and postdoctoral fellows involved in their Mitacs R&D projects.

Multiple lines of evidence showed that Mitacs programs have played an important role in improving the employment prospects of their interns. According to Mitacs exit surveys, the vast majority of interns, fellows and industry partners responding to surveys reported that participation in Accelerate and Elevate led to a more competitive skill set. This included improved communication, critical and creative thinking, and networking skills. Overall, 95% of Accelerate and Elevate interns/fellows responding to exit surveys reported that their career prospects had improved as a result of participating in the program.

For Mitacs international internships supported through Globalink, more than 90% of those responding to exit surveys said that participation in the program had improved their career prospects. Of note, more than 85% of Globalink participants indicated an interest in pursuing a career in R&D.

The majority of interviewees said that the on-the-job experience gained through Mitacs programs was the most important factor in improving interns' employment prospects. They also said that the program creates networking opportunities for interns, often leading to job opportunities with the industry partner or with other organizations. In the interviews and case studies, many of the industry partners reported having hired one or more students, or intending to do so when the interns completed their studies.

For example, in one project an intern worked for a small company developing 'bio-marker' analysis to identify indicators of oil exposure on marine organisms. The intern was the lead in carrying out the research and was hired by the company to manage the delivery of this new service. In another project, an intern worked with a company developing technology to analyze social media. In addition to being hired by the company, the student eventually became part-owner of a spin-off company.

Overall, the Mitacs' Accelerate outcome survey of industry partners found that 30% of companies responding to the survey had hired one or several interns and 26% of companies had hired other new employees based on their project experience. Among the Elevate fellows who answered the outcome survey, about half were working in academia and about a third were working in the private or non-profit sectors.

3.2.6 In what manner and to what extent have the internships led to increased retention of domestic and international graduate students and fellows in Canada after completing their studies?

Key finding:
Mitacs is helping to retain domestic and international graduate students and fellows in Canada after completion of their studies.

Program surveys provide evidence that Mitacs Accelerate, Elevate and Globalink programs are contributing to the retention of domestic and international graduate students and fellows in Canada after they complete their studies.

Accelerate
In the last two years (2014–15 and 2015–16), 95% of Accelerate interns who responded to exit surveys reported that they are more likely to stay in Canada postgraduation as a result of their participation in the program. Further, the Accelerate outcomes survey showed that 91% of former interns responding to the survey were still residing in Canada following their internship. This is significant given that foreign students made up more than a third (37%) of ISED-funded participants in the Accelerate program in 2015–16.

Elevate
According to exit survey results, an average of 75% of Elevate fellows who responded to the surveys said they were likely to stay in Canada post-fellowship. The 2016 Elevate Fellows Survey confirmed these results. It found that 76% of former program participants who responded to the survey were living in Canada.Footnote 20

Globalink
There is evidence that Globalink is encouraging international scholars to further their studies in Canada. Exit surveys showed that over 95% of the Global Research Internship respondents were more likely to pursue graduate studies or work in Canada after their studies as a result of their research placement. Outcome survey results demonstrate that this intention is reflected in reality; 22% of former Global Research interns who enrolled in a new degree are currently studying in Canada, compared with only 9% of an applicant control group (i.e. eligible program applicants that did not complete a Mitacs internship). Further, the majority of interns responding to the survey (72%), who are currently studying in Canada, have applied or intend to apply for permanent residency in this country.

3.2.7 How efficiently and economically is the program being delivered?

Key finding:
Mitacs administration costs are within an acceptable range and, as an organization, Mitacs is leveraging considerable resources. Mitacs has robust performance measurement systems in place. However it lacked a way to measure the program's contribution to academia-industry collaboration in the broader Canadian context. Given the substantial increases in ISED contributions over the next five years, it may be prudent to conduct an external review of the organization's systems and processes.

The evaluation assessed both program delivery by Mitacs and ISED's management and oversight of the program.

Efficiency and economy of Mitacs
The efficiency of third-party delivery organizations is commonly assessed by looking at the ratio of administrative expenditures to total expenditures. For Mitacs, the average ranged from 11% to 15% from 2012–13 to 2015–16 across its three programs. This is within the 15% limit for administrative expenditures stipulated in ISED's contribution agreements.

Mitacs made efforts to improve its efficiency during the evaluation period through upgrades to its information management systems. This included the development of an online tool to handle application forms, the review and matching of applications, and communications with students as well as a portal for universities to monitor the status of Globalink students coming to their universities. Further, Mitacs has 50 active Memoranda of Understanding with organizations across all disciplines and sectors, a key objective being to streamline the administrative process of internship delivery for participating industry partners.Footnote 21

When asked about Mitacs efficiency, a number of interviewees indicated that Mitacs business development officers are an important part of the service delivered by the program due to their proactive approach, and the support provided during the application process. It should be noted however that among the small sample of unsuccessful applicants interviewed for the evaluationFootnote 22, some were not aware of the presence of the business development officers at their academic institutions and/or being able to obtain assistance and support through them. Although data was not collected for this evaluation, moving forward it may be useful to examine the extent to which the presence of business development officers at different academic institutions influences the success of applications. Also, while some unsuccessful applicant interviewees were satisfied with the assistance these officers provided with their application, many of them were not.

According to Mitacs' exit surveys, most program participants were satisfied with Mitacs' application and review process. Further, Mitacs reporting showed that it was meeting its performance targets for the time needed to review applications. In the personal experience of interviewees the time required to process a Mitacs application ranged from four weeks to four months. A few interviewees said they felt the application and review process took too long.

As well, some industry stakeholders and unsuccessful Accelerate applicants also found the applications repetitive and difficult to follow. Unsuccessful applicants said they would have benefitted from information about application volumes and program success rates in order to gauge how competitive their application would be. They also would have liked more guidance on what constitutes a successful application, over and above meeting eligibility requirements (such as rating criteria in addition to the mandatory criteria). Most felt the review process could be improved by providing constructive feedback so they could understand why their application was unsuccessful.

Mitacs has put in place a robust system for performance measurement and the evaluation used Mitacs' exit and outcome surveys as a key source of evidence. However, Mitacs' does not have a way to measure the program's overall contribution to academia-industry collaboration in the broader Canadian context. Further work is needed to develop such tools. There may also be an opportunity to improve Mitacs' exit survey response rates, especially for the Elevate program where less than half of participants are responding to exit surveys. Further, in reporting on its output data on internships it would be preferable if Mitacs reported the actual number of internships that were carried out by its Globalink program rather than the number of matches.

While the evaluation revealed no major issues related to Mitacs administration of ISED-funded programming, moving forward it will be important to closely monitor Mitacs performance to ensure it has adequate resources, systems and processes in place to support the scaling up of its program due to the nearly four-fold increase in ISED contributions to Mitacs over the next five years.

Efficiency and economy of ISED's oversight
In funding Mitacs, ISED is employing a third-party delivery model. About a third of those interviewed pointed to third-party delivery as a strength, as it allows the program to be responsive to stakeholder needs and take advantage of opportunities; for example, by developing partnerships.

Another benefit of the third-party delivery model is the ability to leverage funding from other sourcesFootnote 23. Mitacs secured a total of $222 million from external sources from 2012–13 to 2015–16, meaning that for every dollar invested by the federal government, Mitacs has been able to leverage an average of $3.8 of other funding.Footnote 24 Of note, 44% of funding was provided by industry.

ISED oversight includes the development and implementation of the funding agreements, attending Mitacs Board of Directors meetings as an observer, reviewing Mitacs reports and providing ongoing analysis and advice to the Minister of ISED and Minister of Science in relation to Mitacs. Evidence collected as part of this evaluation suggests that ISED is managing these tasks efficiently. SIS estimates that approximately one full-time equivalent (FTE) is dedicated to core Mitacs oversight activities for this contribution, made up of both staff and management time.

4.0 Conclusions and recommendations

4.1 Relevance

  • The evaluation found that Mitacs responds to the ongoing need to increase Canadian innovation by using internships to facilitate linkages between academia and industry and increase international collaboration. Strong demand for Mitacs programs is reflected in the growing number of internships.
  • Mitacs clearly supports federal government priorities to invest in research and innovation to strengthen the Canadian economy. The program also seeks to foster the next generation of researchers through collaboration between the private sector and academia. Further, ISED's support for Mitacs is consistent with the federal government's responsibility to foster science and technology.

4.2 Performance

  • Evidence shows that Mitacs has achieved its expected results. In the short-term, Mitacs internships have contributed to collaboration and resource sharing between academia and industry. They have allowed for knowledge transfer to the private sector and enabled some interns to use their research to further their academic work.
  • The work of Mitacs interns has supported the development of new products, processes and services for industry partners. Mitacs' use of university-based business development officers is seen by stakeholders as an important contributor to program success.
  • The evaluation also found that over the medium-term Mitacs programs are helping to build research linkages between industry and academia in Canada, and between researchers in Canada and abroad that continue beyond the end of Mitacs projects.
  • The program has contributed to investments in industrial R&D by industry partners during and after the conclusion of their Mitacs projects.
  • There is evidence that Mitacs has improved the employment prospects of its interns by providing on-the-job experience and networking opportunities. A number of companies and organizations have hired the graduate students and postdoctoral fellows involved in their Mitacs R&D projects. Finally, Mitacs is helping to keep domestic and international graduate students and fellows in Canada after they complete their studies.
  • Mitacs administration costs are within an acceptable range and it has made efforts to improve efficiency through upgrades to its information management systems. Mitacs has robust performance measurement systems in place. However, it lacked a way to measure the program's overall contribution academia-industry collaboration in the broader Canadian context.
  • As an organization, Mitacs is leveraging considerable resources, averaging $3.8 dollars for every dollar contributed by ISED. As ISED's total contributions to Mitacs will be increasing by nearly four times over the next five years as compared to the evaluation period, it may be prudent for SIS to conduct an external review of the organization to ensure it has the people, systems and processes in place to successfully deliver the increased programming.

4.3 Recommendations

The findings of the evaluation led to the following recommendations.

  1. SIS should consider conducting an external review to assess whether Mitacs has the resources, systems and processes in place to successfully manage future growth.
  2. SIS should consider working with Mitacs to develop the measures against which to assess Mitacs'contributions to overall industry-academia collaboration in Canada.

Management Response and Action Plan

A - For inclusion in the report

The findings and recommendations of the Mitacs Report were presented to program management. Management has agreed with the findings included in this report and will take action to address all applicable recommendations by December 31, 2018.

B - For follow-up purposes - Detailed actions to address the recommendations in the report

Recommendation

Planned Action on the Recommendation

Responsible Official (position)

Target completion date

Recommendation 1
Science and Innovation Sector (SIS) should consider conducting an external review to assess whether Mitacs has the resources, systems and processes in place to successfully manage future growth.

Agree with the importance of Mitacs having the resources, systems and processes to manage its forecast growth.

ISED will seek from Mitacs a detailed plan for how it intends to manage future growth. This will include the controls and processes related to resources and systems to successfully deliver increased programming.

ISED will review Mitacs' plan, in consultation with the Audit and Evaluation Branch, and seek expert analysis and advice where gaps or opportunities are identified to strengthen existing or planned management controls and practices.

Director General,

Program Coordination Branch

December 31, 2018

 

 

Recommendation 2
SIS should consider working with Mitacs to develop the measures against which to assess Mitacs' contributions to overall industry-academia collaboration in Canada.

Agreed.

SIS will work with Mitacs to examine current approaches to assessing Mitacs' contributions to overall industry-academic collaboration in Canada and examine the feasibility of expanding upon and improving these metrics.

ISED's Performance Measurement Strategy will be reviewed and updated accordingly. SIS will consult with the Audit and Evaluation Branch (AEB) and Corporate Management Sector (CMS) in making revisions to the Performance Measurement Strategy.

Director General,

Program Coordination Branch

December 31, 2018