Management of Transfer Payment Programs

Audit and Evaluation Branch
Industry Canada

April 2008

Recommended for Approval to the Deputy Minister By the DAC on May 27, 2008
Approved by the Deputy Minister on May 30, 2008

This publication is available upon request in accessible formats. Contact:

Multimedia Services Section
Communications and Marketing Branch
Industry Canada
Room 264D, West Tower
235 Queen Street
Ottawa, Ontario  K1A 0H5

Telephone: 613-948-1554
Fax: 613-947-7155
Email: ic.cmb-creative.ic@canada.ca

Permission to Reproduce

Except as otherwise specifically noted, the information in this publication may be reproduced, in part or in whole and by any means, without charge or further permission from Industry Canada, provided that due diligence is exercised in ensuring the accuracy of the information reproduced; that Industry Canada is identified as the source institution; and that the reproduction is not represented as an official version of the information reproduced, nor as having been made in affiliation with, or with the endorsement of, Industry Canada.

For permission to reproduce the information in this publication for commercial redistribution, please email: copyright.droitdauteur@pwgsc.gc.ca

Cat. No. Iu4-133/2008E-PDF
ISBN 978-1-100-10023-4
60476

Aussi offert en français sous le titre Gestion des programmes de paiements de transfert.

Table of Contents

• 1.0 Executive Summary
  • 1.1 Introduction
  • 1.2 Main Findings
  • 1.3 Recommendations
  • 1.4 Statement of Assurance
  • 1.5 Audit Opinion
• 2.0 About the Audit
  • 2.1 Managing Industry Canada Transfer Payment Programs
  • 2.2 Audit Objectives
  • 2.3 Audit Scope
  • 2.4 Audit Approach and Methodology
• 3.0 Findings and Recommendations
  • 3.1 Capacity to Deliver
  • 3.2 Risk Management
  • 3.3 Compliance with Terms and Conditions
  • 3.4 Operational and Financial Monitoring
  • 3.5 Information for Management
• 4.0 Appendix A–Summary of Transfer Payments Programs
• 5.0 Appendix B–Detailed Audit Criteria
  • Capacity to Deliver
  • Risk Management
  • Compliance to Transfer Payment Program Terms and Conditions
  • Operational and Financial Monitoring
  • Information for Management
• Management Response and Action Plan

1.0 Executive Summary

Industry Canada is responsible for the delivery of several transfer payment programs, with annual expenditures of almost $900 million. As such, it is essential that the delivery of such programs be supported by a robust management framework.

Industry Canada has such a framework encompassing several key elements, including:

• a Centre of Excellence, the Program and Services Board (the Board);
• Industry Canada Program Policies and Procedures Manual;
• management controls and delegation of financial assistance authority;
• program terms and conditions; and
• bulletins and training for program managers and staff.

In our opinion, through this framework, the Department has been effective in providing oversight in the selection, design and implementation of transfer payment programs, in ensuring a systematic approach to project risk management and in establishing a strong due diligence process during the application process.

However, opportunities exist to strengthen or clarify certain elements of the framework, notably the need for more timely Post-Approval Reviews of smaller projects and the absence of a comprehensive strategy to deal with information systems to support the management of grants and contributions at a departmental level.

1.1 Introduction

The audit of the Management of Transfer Payment Programs was carried out in accordance with the approved Internal Audit Plan for 2005–06.

The audit was department-wide in scope and encompassed the responsibilities assigned to all managers involved in the delivery, management and oversight of transfer payment programs. The audit included both grant and contribution programs but excluded transfers to foundations. The audit covered programs in existence and projects that were in place during the fiscal year 2005–06.

1.2 Main Findings

An Effective Centre of Excellence: The department has established an effective centre of excellence for grants and contributions through the Programs and Services Directorate which provides critical value-added oversight in the selection, design and implementation of transfer payment programs. However, opportunities exist to further strengthen or clarify the Board's oversight role.

Grants and Contributions Management Systems Landscape: The Department has not yet developed a comprehensive strategy to deal with information systems to support the management of grants and contributions at the department level.

A Working Capacity to Manage Transfer Payments: Program staff has the knowledge, training and time to properly assess applications and recipients.

Systematic Project Risk Management: Programs have developed systematic means of assessing project risks. However, there is no assurance that program level risks are considered from a corporate perspective and reflected in the Corporate Risk Profile.

Strong Due Diligence during the Application Process: Programs have a due diligence process in place to approve recipient applications that address the eligibility criteria contained in the programs terms and conditions.

Tracking and Monitoring of Repayable Contributions: A variety of mechanisms exists to track, monitor and recover repayable contributions within the department.

Weak Links between Project Risks and Monitoring: Not all programs link the risks of a project to their project monitoring strategy.

Improved Performance Management Required: Ongoing program performance information is not systematically identified, collected, analyzed and appropriately reported.

Various Approaches to Conducting Recipient Audits: While recipient audits are conducted in every program; there are differences in the approaches used by the programs.

Evaluations Consistent with RMAF Requirements: Evaluations of programs are undertaken in accordance with approved RMAFs.

1.3 Recommendations

• The Chief Financial Officer, through the Programs and Services Board, should:
  • ensure that Post-Approval Reviews are undertaken and tabled in a timely manner. The PAR exercise should be undertaken using a risk-based approach to recognize the existence of mechanisms put in place by programs to monitor the approval process;
  • clarify its role with respect to performance monitoring;
  • establish a policy with respect to Quality Assurance Programs; and
  • reconsider the Board's role with respect to the review of contracts. Considerations should be given to the establishment of a separate contract review mechanism with suitable representation.

• The Chief Financial Officer should assume accountability as process owner for departmental grants and contributions until a permanent process owner is determined. In so doing, the Chief Financial Officer should:
  • through the ITSMC, and with the involvement of all sectors (program and corporate), articulate a vision and a transition strategy for a new departmental G&C Management System; and
  • ensure that any replacement system include the ability to track and report on program benefits.

• The Chief Financial Officer should, through the Integrated Risk Management Framework, provide policy direction requiring programs to regularly update their program and project risk profiles.
• The Chief Financial Officer should establish policy direction requiring programs to link their monitoring strategies to the identified risks and establish tracking mechanisms to account for monitoring activities.
• The Chief Financial Officer should establish policy direction with respect to requirements for benefits tracking.

1.4 Statement of Assurance

In my professional judgment as Chief Audit Executive, sufficient and appropriate audit procedures have been conducted and evidence gathered to support the accuracy of the opinion provided and contained in this report. The opinion is based on a comparison of the conditions, as they existed at the time, against pre-established audit criteria. The opinion is applicable only to the entities examined and within the scope described herein.

1.5 Audit Opinion

In my opinion, the Management of Transfer Payment Programs reflects some weaknesses, with low to moderate risk exposures related to the governance, control and risk management processes relative to how the department manages its Transfer Payment Programs that require management attention.

Richard Willan
Dirigeant principal de la vérification intérimaire Industrie Canada

Date

2.0 About the Audit

2.1 Managing Industry Canada Transfer Payment Programs

Policy

The Treasury Board Secretariat's (TBS) Policy on Transfer Payments (Policy) is the key authority with respect to the management of Industry Canada's (IC) transfer payment programs. ¹ The Policy is supplemented by the IC Program Policies and Procedures Manuals.

Industry Canada Roles and Responsibilities

Framework

Total budgeted transfer payments for FY 2006–07 exceed $600 Million (see Appendix A for a summary of transfer payments over $5M for 2005-06 and 2006–07). The management framework for transfer payment programs within Industry Canada encompasses TBS policy requirements and includes:

• IC Program Policies and Procedures Manuals;
• IC management controls and delegation of financial assistance authority;
• Program terms and conditions;
• Results-based Management and Accountability Frameworks (RMAFs) and Risk-Based Audit Frameworks (RBAFs) developed by the programs and the implementation of such frameworks;
• Support from the Programs and Services Board and Directorate (The Board and The Directorate), the Program Policy and Management Directorate (PPMD), and the Chief Information Officer; and
• Bulletins and training for program managers and staff.

Program Management

Responsibility for the management and delivery of IC transfer payment programs rests with departmental sectors and program managers who design and implement procedures and controls that promote and ensure the appropriate compliance to program terms and conditions as well as to TB and IC policies.

Departmental sectors are responsible for the delivery of their respective transfer payment programs and for establishing requisite delivery mechanisms. Program managers are responsible for the development of program terms and conditions, result-based management accountability frameworks and risk-based audit frameworks (RMAFs/RBAFs) which require TB approval, and for managing programs in compliance with these requirements.

Programs and Services Board (Board)

The Board serves as a forum for senior departmental officials to provide corporate direction in the selection, design, implementation and performance monitoring of financial assistance programs within the Department. The Board is supported by a secretariat which operates within the Programs and Services Directorate.

The Board is primarily responsible for:

• establishing criteria for the review of new financial assistance programs and business service proposals, assessing new initiatives and proposed modifications to existing programs and services against these criteria and formulating departmental recommendations on such initiatives to the approval authority;
• providing policy direction on governmental and departmental policies, operational procedures and performance monitoring which relate to transfer payments programs and business services;
• reviewing recommendations on large projects and requesting modifications when necessary before consideration by the approval authority, and making decisions on substantive amendments submitted under departmental programs².

In its terms of reference, the Board also identifies a number of areas which it is concerned with, including performance monitoring and informatics systems relating to programs and services.

Programs and Services Directorate (Directorate)

The Directorate contributes to program design, program integrity and provides secretariat services to the Board. Its key functions include:

• Ensuring that departmental transfer payment programs are designed and administered taking into consideration relevant governmental and departmental policies, priorities and lessons learned (this includes the conduct of environment assessments in compliance with the Canadian Environmental Assessment Act when such assessments are required);
• Design and oversight of the implementation of tools (policies, directives, procedures and training modules) to support officers in program delivery;
• Acting as a centre of expertise through the provision of expert advice on program delivery;
• Providing advice and assistance to programs in the preparation of authority documents such as: TB Submissions; Memoranda to Cabinet; RMAF/RBAFs; Terms and Conditions; and MOUs;
• Providing interpretations on matters regarding governmental and departmental policies, guidelines and directives and the amendment or issuance of policies;
• Conducting project integrity reviews on all projects above $500,000 prior to presentation to The Board;
• Conducting a post-approval review of projects below $500,000 every other year. The primary objective of this post-approval review is to attest that these projects were subject to a due diligence process by the programs prior to their approval; and
• Reviewing and posting to the Public Registry, environmental screenings carried out by the various responsibility centres of the Department.

Chief Informatics Officer (CIO)

As the Department's principal information technology (IT) advisor, the CIO offers a wide range of IT products, services and support to IC sectors. To this end, the office of the CIO maintains a central data repository–Contribution Management Information System (CMIS)–related to transfer payment programs delivered by the department. CMIS serves as the financial system for the management of multi-year commitments for Grants and Contributions. It interfaces with the Integrated Financial and Materiel System (IFMS). CMIS also serves as a management information system to program managers and is often used to respond to Access to Information (ATI) requests. A few years ago, additional reporting functionalities were added through the creation of the Grants and Contribution Reporting System (GCRS).

Within the Comptrollership and Administration Sector; Financial and Materiel Management Directorate (CAS-FMMD), the CMIS Team is responsible for the maintenance of the CMIS and manages access to the system. Together, the CIO and CAS-FMMD have designed the security protocols and established the access profiles for CMIS. The CIO provides accounts to users. Controls over access to CMIS are a shared responsibility between program managers and CASFMMD.

Program Policy and Management Directorate (PPMD)

PPMD is part of the Policy and Sector Services Branch, Industry Sector and works with the Comptroller, Legal Services and program authorities to ensure that: IC transfer payment programs meet the requirements of the TB Policy on Transfer Payments regarding the repayment of contributions; recipients of IC financial contributions continue to fulfill their contractual obligations to repay part of or the entire contribution received; and that economic objective commitments established at the time of approving the contribution are achieved.

Regional Offices

Departmental regional offices of the Regional Operations Sector assist in the delivery of some transfer payment programs. These include: the Aboriginal Business Canada (ABC) Program (transferred to INAC on December 1, 2006), the Canada-Ontario Infrastructure Program (COIP), the Federal Economic Development Initiative in Northern Ontario's (FedNor) Eastern Ontario Development, Northern Ontario Development and Community Futures Programs.

Regional offices may also facilitate the delivery of transfer payment programs under the responsibility of other sectors, such as Canadian Apparel and Textile Industries Program (CATIP), Structured Financing Facilities Program (SFF) and the Broadband for Rural and Northern Development Pilot Program (BRAND). A specific arrangement typically defines the roles and responsibilities of the regional offices when they are involved in the delivery of another sector's program.

2.2 Audit Objectives

The audit of the Management of Transfer Payment Programs was conducted in accordance with the approved Internal Audit Plan for 2005–06. The objectives of the audit are:

1. To assess the adequacy of the design and operation of the Department's framework for managing transfer payment programs; and
2. To ensure that IC is effectively discharging its responsibilities with regard to the Result-Based Management Accountability Framework and Risk-Based Audit Framework for the various transfer payment programs it manages.

2.3 Audit Scope

The audit was department-wide in scope and encompassed the responsibilities assigned to all managers involved in the delivery of transfer payment programs. The audit included both grant and contribution programs but excluded other transfers such as transfers to foundations. The audit covered programs in existence and associated projects that were in place during the fiscal year 2005–06. Records of Decisions of the Board meetings and minutes of other departmental committees were also reviewed.

During the course of the audit, Aboriginal Business Canada (ABC) was transferred to the Department of Indian and Northern Affairs. ABC was examined as part of this audit and key findings were communicated to the program. These are not included in this report.

2.4 Audit Approach and Methodology

To conclude on the two overall objectives audit criteria were developed for the following areas: Capacity to Deliver; Risk Management; Compliance with Program Terms and Conditions; Operational and Financial Monitoring; and, Information for Management. These audit criteria are presented in Appendix B.

The following approach and methodology was used to conduct the audit:

• Reviewed relevant documentation on the management of transfer payments within Industry Canada. The auditors reviewed relevant documentation in order to gain a sound understanding of IC policies and procedures as they relate to grants and contributions, controls, reporting and oversight mechanisms.
• Established audit criteria to undertake the audit (Appendix B). Specific audit criteria were developed to assess the appropriateness of the grant and contribution management framework and to ensure that key elements of this framework are in place in HQ and in the regions and working as intended.
• Interviewed key individuals from various Programs, the Programs and Services Board and Directorate, representatives from the Resource Management and the Financial and Materiel Management Divisions of the Comptrollership and Administration Sector (CAS) in their capacity to manage CMIS and interfaces to IFMS and the Chief Informatics Officer who has a supporting role with respect to CMIS.
• Conducted site visits to regional offices and one Alternative Service Delivery organization. The auditors visited the Ontario and Prairie and Northern Regional Offices as well as the FedNor office in Sudbury to assess management controls as well as to review a sample of grant and contribution files. In addition, an Alternative Service Delivery organization, which received financial assistance from both FedNor and ABC, was visited. In its relationship with ABC this organization solicits, approves and manages contributions on behalf of the department. In its relationship with FedNor, it acts as an aboriginal Community Futures Development Corporation, lending money within the community it is serving.
• Conducted a review of selected grant and contribution program files. The auditors reviewed a sample of contribution projects from the following six programs for detailed review. These include a mix of large and small programs, either centrally delivered or decentralized:
  • Technology Partnerships Canada (TPC);
  • Aboriginal Business Canada (ABC);
  • Canada-Ontario Infrastructure Program (COIP); Canada-Ontario Municipal and Rural Infrastructure Program (COMRIF) and Canada Strategic Infrastructure Fund (CSIF);
  • Northern Ontario Development Program (NODP), a FedNor program;
  • Canadian Apparel and Textile Industries Program (CATIP); and
  • Broadband (BRAND).
• Validated results from sample review. Audit results were documented and provided to the various Programs for review and comment. The auditors also debriefed key departmental officials on the results of the audit and to seek feedback.
• Current dated the results of the audit. The audit report was prepared in early 2007 based on field work completed during the fall of 2006. Given that the reporting strategy was changed to present results of audits of key transfer payment programs prior to tabling this audit report, additional work was undertaken during the fall of 2007 to current date this audit report. Interviews were conducted with key senior managers to take into consideration the latest developments with respect to observations being raised in the audit report.

3.0 Findings and Recommendations

Findings arising from the audit are presented in this section. In the absence of a designated functional authority for all matters pertaining to the management of transfer payments, ensuing recommendations have been directed to the Chief Financial Officer (CFO) who also serves as the Chair of the Programs and Services Board.

3.1 Capacity to Deliver

Finding 1: an Effective Centre of Excellence

The Board reviews recommendations on large projects, requests modifications before consideration by the approval authority, and recommends approval of substantive amendments to projects. This follows a rigorous review of Project Summary Forms by the Directorate. Where required, clarifications are sought from programs before they are submitted to the Board for review.

Our review of the Board Records of Decisions noted that its activities primarily involve the review of new programs, projects above $500,000 or substantive amendments to projects. Every project above $500,000 is reviewed by the Board. For fiscal year ABC 2006–07, 53 projects of more than $500,000 were reviewed and approved (or recommended for approval to the appropriate authority) by the Board, representing 86% of the value of approved IC transfer payments that year. As of December 2007, 24 projects of more than $500,000 were reviewed and approved by the Board, representing 51.7% of the value of approved IC transfer payments for the year.

The role of the Board pertaining to their review of major expenditures has been recognized amongst departments as a good practice and the department has been cited as a model to other departments and agencies.

Use of PAR exercises as an assurance mechanism for due diligence for smaller projects

Projects below $500,000 are not subject to a pre-approval integrity review by the Directorate nor are they tabled at the Board for review and approval. However, based on a 1994 Ministerial decision to delegate authority for approval to responsibility centres managers, every other year, at the direction of the Board, the Directorate is required to conduct a Post-Approval Review (PAR) of projects below $500,000 to provide assurance to the Board that programs have exercised due diligence prior to these projects being approved. These projects have traditionally represented less than 25% of the value of the Department's transfer payment portfolio but with changes to programs, this ratio is increasing. They however, constitute the greatest number of grant and contribution projects approved in the Department in any given year.

The audit noted that the last PAR results tabled at the Board were for the 2001–02 fiscal year. The tabling of PAR results for fiscal year 2003–04 has been delayed, in part due to scheduling challenges at the Board but also because some programs had indicated disagreement with results of the PAR exercise, which noted increased deficiencies over previous PAR exercises. Auditors also noted that the initiation of the most recent PAR, for fiscal year ABC2005–06 was delayed and that no decision has yet been made as to whether and when it will be conducted.

The conduct of PAR exercises was a condition imposed on the Department in return for receiving increased delegation limits from the Minister. As such, it is important that the process continue in a timely manner. The PAR exercises are also considered by the Board as a key element of the grant and contribution management framework, as expressed by the Chief Financial Officer in a recent presentation to the Audit Committee. Auditors are concerned that undue delays in initiating or tabling the PAR may lead to the report becoming less relevant to programs, and to corrective measures not being noted or implemented in a timely manner.

Performance Monitoring

The Board's mandate makes reference to it providing direction with respect to performance monitoring. In Appendix 4.2b of the Programs and Services Board Terms of Reference, it states that: "With regard to performance monitoring, the PSB will be concerned with: the monitoring of financial assistance programs… through the review of objectives, achievements…, the review of annual reports, evaluation studies and action plans developed to address observations and recommendations of Central Agencies." This notwithstanding, the audit noted that the Board has not articulated how this role is to be discharged, nor does it regard this element to be part of its role.

Some programs have established quality assurance programs (QAP) with varying degree of sophistication. Auditors noted that a QAP typically provides assurance of compliance with terms and conditions of the program and with IC and or TB policies. The decision to have a QAP is at the discretion of the program and those programs that do have a QAP have yet to implement a complete process which includes reporting and action plans. There is no formal policy on quality assurance programs to stipulate when such programs should be in place or how the results should be communicated.

While the Board has occasionally reviewed departmental internal audit reports, and some reports of the Office of the Auditor General (OAG), it does not do so routinely. We found that the Board is not informed of whether programs are working as intended. The Board does not review departmental program evaluation reports unless it is part of a TB submission for the renewal of a program, and does not receive annual reports from programs or reports summarizing the results of quality assurance exercises that programs conduct from time to time.

While the Board undertakes excellent work with respect to the due diligence process leading to the recommendation of new programs, major projects or their amendments, its role could be enhanced to provide increased oversight of performance monitoring, thus allowing the Board to apprise how well the departmental policies and procedures are complied with and improving its ability to discharge its governance role.

PSB's Other Duties

In addition to its role with respect to transfer payments, PSB serves as a contract review mechanism by reviewing sole source contracts over $25,000, ACAN submissions and (with some exceptions) proposed competitive contracts valued in excess of $84,000 (known as the Early Warning System). During 2007–08, approximately 35% of projects presented to the Board represented contracts. Although an important task, this responsibility represents an added workload to the Board's primary oversight role for grants and contributions (i.e. comparing project greater than $500,000 with contracts often less than $100,000) and may take away from its other mandated responsibilities. The auditors noted that in other departments and agencies, the activities of a contract review mechanism are often carried out by contract specialists and managers below the ADM level and feel that a similarly represented mechanism could be established in the Department.

Recommendations

• The Chief Financial Officer, through the Programs and Services Board, should:
  • ensure that Post-Approval Reviews are undertaken and tabled in a timely manner. The PAR exercise should be undertaken using a risk-based approach to recognize the existence of mechanisms put in place by programs to monitor the approval process;
  • clarify its role with respect to performance monitoring;
  • establish a policy with respect to Quality Assurance Programs; and
  • reconsider the Board's role with respect to the review of contracts. Considerations should be given to the establishment of a separate contract review mechanism with suitable representation.

Finding 2: Grants and Contributions Management Systems Landscape

The G&C Management System Background

Over the years, the Department has developed a number of systems aimed at supporting both its financial as well as program needs with respect to the management of grants and contribution programs. The CIO has identified eleven systems that contribute to the family of Grants and Contributions Management Systems.

Diagram A   Current G&C Systems Landscape
(Extract from 2007 CIO deck: Grants and Contributions Sustainability Strategy)

Figure 1: Diagram A—  Current G&C Systems Landscape

[Description of Figure 1]

Some of the systems are accessible to all programs, such as Contribution Management Information System (CMIS) and Grant and Contribution Reporting System (GCRS) while others are accessible only to those programs that have developed them, such as the ITO Data Warehouse and e-claims (FedNor). Diagram A shows the current family of Grants and Contributions Management Systems.

Past Efforts Toward an Integrated G&C Management System

For several years the Department has favoured a shift to an integrated Grants and Contributions Management System (G&C Management System) and successive efforts were launched to that end. Some efforts, such as CMIS, introduced in 1998, and the GCRS, introduced a few years later, were successful. Other efforts have not been as successful, notably the 2001 GX project. Also in 2007, a Request for Information (RFI) was developed by the CIO and the G&C Working Group to initiate an investigation of products available in the marketplace to replace CMIS, which has been identified as having a remaining useful life of approximately four years. The RFI was subsequently cancelled. In other instances, some programs worked together to develop sub-systems meant to address their particular requirements. This was the case with the e-claims applications which was developed by FedNor and Aboriginal Business Canada. In other instances, organizations with larger programs have developed specific applications to facilitate the administration of their programs.

These initiatives were launched in the absence of a clear departmental vision and strategy for an integrated G&C Management System and of how it objectives were to be achieved.

A 2004 BearingPoint Study acknowledged that the Department required a central information management system to manage and monitor grants and contributions, but concluded that a lack of IM/IT governance was a significant factor in the collapse of the 2001 GX project. That report concluded that senior management are generally not involved in planning or decision-making and that there is little awareness of, or challenge to, sector-based IM/IT investments.

The 2005 Corporate Risk Profile identified that there was an important stewardship risk associated with the management and control framework of Gs&Cs awarded by IC, resulting in sectors having to keep multiple "books", redundancy of effort and increased potential for errors. The Corporate Risk Profile also identified a risk with respect to the corporate information technology management stating that decentralized IT functions may result in integration and coordination difficulties as well as problems and challenges from a department-wide IT management perspective.

The IT Governance Structure in the Department

Departmental IT governance is established through the Information Technology Governance Committee, which was recently restructured and renamed the Information Technology Strategic Management Committee (ITSMC). The ITSMC mandate is to provide strategic direction and oversight on IT activities and policies and to prioritize departmental IT investment projects. A G&C Steering Committee and a Working Group were established in 2005 to explore options to a CMIS replacement. The former Information Technology Business Planning Committee, now replaced by the Project Oversight Committee, also played a role in the governance structure, with an objective to providing a common voice for G&C stakeholders with a focus on business transformation and automation. However this governance structure is not easy to operate given that different programs remain very autonomous in how they organize and fund IT support for the programs they manage. The involvement of the governance structure is limited to department-wide applications, of which there are few. A review of the minutes of these committees for the past two years shows that no discussion or decisions were made with respect to an integrated G&C management system or for the replacement of CMIS.

The Program and Services Board Terms of Reference also identifies a role for the Board when it states that: "…the PSB will be concerned with: … the review of any informatics system having a significant impact on financial assistance programs and business services and of all informatics system proposals relating to programs and services".

Although efforts have been made to structure a governance environment for IT decisions, confusion in who the process owner is for grants and contributions and who should sponsor changes in systems still remains a barrier.

Sustainability of Current G&C Management System Environment

Recent studies undertaken by the CIO have revealed that there is still time to develop a comprehensive strategy to deal with information systems supporting the management of grants and contributions at the departmental level. Current systems are sustainable, with mitigation strategies, for the next four to five years. However, the results of the sustainability assessment study and strategy have yet to be tabled at the ITSMC for discussion and endorsement.

In the meantime, the CFO envisioned that during this period of time, the department will be able to move from its current IFMS SAP version 4.7 to version 6.0, opening more avenues to the department in terms of its future choices for a comprehensive grants and contributions management system. A government-wide forum of SAP users, the IFMS Core, is also assessing solutions aimed at responding to departments' needs with respect to comprehensive grants and contributions systems. This information should be valuable to the department for making its decision with respect to a comprehensive G&C management system.

Impact on Programs and Staff

Amongst the G&C systems landscape, CMIS is the key system as the department relies on it to track grant and contribution applications and payments (multi-year commitments) on approved projects. CMIS is a system serving both financial control and other G&C management needs. The system, populated by programs is owned by the Comptrollership and Administration Sector (CAS) since a re-organization in October of 2003. The audit noted that CMIS has limitations that diminish its usefulness as a tool to support program staff in the management of grant and contribution projects. CMIS has a slow response time and, although access to it is limited, staff responsible for data entry often encounters considerable difficulties in using the system. Response time can be as long as fifteen to twenty minutes during peak periods. The system also has weak functionalities with respect to capturing data on program benefits, which limits its usefulness for reporting purposes.

In the decade that followed the introduction of CMIS and through the launch of several complementary systems developed by programs, the department has not yet articulated a vision for a Grants and Contributions Management System. This has had an impact on the efficient delivery and administration of the department's transfer payment programs. In the auditors' opinion, the momentum currently exists for establishing a vision for a G&C Management System and for articulating a transition strategy to attain the vision.

Recommendations

• The Chief Financial Officer should assume accountability as process owner for departmental grants and contributions until a permanent process owner is determined. In so doing, the Chief Financial Officer should:
  • through the ITSMC, and with the involvement of all sectors (program and corporate), articulate a vision and a transition strategy for a new departmental G&C Management System; and
  • ensure that any replacement system include the ability to track and report on program benefits.

Finding 3: A Working Capacity to Manage Transfer Payments

As part of examining the capacity of program staff, the auditors assessed whether sufficient resources exist to support program delivery, training is available and information and tools exist to support project officers and managers in the carrying out their responsibilities.

Interviews with various program officials determined that staff is provided with the information, tools and resources to do their job well. Interviews also confirmed that program managers and project officers clearly understand their respective responsibilities relating to program delivery.

A set of IC policies, training courses and materials allow program managers and officers to deliver programs in compliance with the TB Policy on Transfer Payments as well as current departmental policies.

We noted that the Directorate offers a Project Assessment and Program Delivery course and that an Environmental Assessment course is also available to departmental staff. There has been considerable demand for the training and at times, the Directorate has had difficulty coping with requests. The audit also noted that other programs have developed their own training programs tailored to their own processes and forms. All in all, project officers and program managers have received helpful training on the management of grant and contribution programs.

3.2 Risk Management

Finding 4: Systematic Project Risk Management

Maintaining the Department's Risk Profile

The Corporate Risk Profile (CRP) outlines and communicates potential high risks that could hinder the achievement of the department's strategic outcomes and/or program activities and related risk mitigation strategies. The development of a CRP is an important element of a systematic Integrated Risk Management process. In IC, responsibility for development of the CRP rests with the Corporate Planning and Governance Branch of the Comptrollership and Administration Sector (CAS). A first CRP was done in 2005.

In 2007, the Corporate Planning and Governance Branch launched an initiative to develop an integrated risk management framework, including updating of the Corporate Risk Profile. The framework known as the " Integrated Risk Management (IRM) Framework " was tabled at the Comptrollership and Administration Sector Advisory Committee (CASAC) in December of 2007. The process is expected to be implemented in the Spring of 2008.

Auditors reviewed the results of the 2005 CRP to assess whether risks associated with transfer payment programs were adequately reflected in the CRP and noted that transfer payments were reflected in two areas:

• as part of the Partnerships–Outreach and Communications risk area. Primary emphasis was on risks associated with the Technology Partnerships Canada program, notably risks associated with media coverage of TPC and difficulty in delivering the program; and
• as part of Stewardship and Financial Management risk area. The primary risk relates to the need for an integrated financial management system to support the management, stewardship and control framework associated with departmental G&C programs.

Program and Project Risks

Auditors also examined how program and project risks are assessed. All programs examined have established mechanisms to identify and assess program and project risks and have developed formal approaches to assess project risks during the application process. Programs are required to develop a Risk Based Audit Framework (RBAF) to explain how risk concepts are integrated into the strategies and approaches used for managing programs that are funded through transfer payments. Information included in an RBAF includes key inherent risk areas faced by the program, an assessment of each specific risk in terms of its likelihood and impact on the achievement of the transfer payment program objectives, a description of existing measures and proposed incremental strategies for managing specific risks, and an explanation of monitoring, recipient auditing, internal auditing, and reporting practices and procedures. The TBS Risk Based Audit Framework (RBAF) Guide provides program managers with guidance on the development of program RBAFs.

At the program level, most programs identify their risks and associated mitigation strategies at the time of designing their program or when the program is renewed. This exercise is done as part of the development of the RBAF, which is required by TB for approving or renewing programs. However, few programs have established a systematic approach for regularly reviewing and updating their program risk profiles, thus possibly precluding the development of appropriate strategies to mitigate any newly identified risk areas. We noted that some programs have also developed formal means of updating their project risk profile at least annually.

We recognize the efforts of the Corporate Planning and Governance Branch, CAS, for leading the CRP initiative for the department. However, more guidance is required to ensure that programs regularly assess their project and program-related risks and establish appropriate mitigation strategies where they are deemed necessary. Given the size, number and diversity of transfer payment programs in the department, without such a requirement, the risk remains that the CRP may not be fully communicating all potential risks that could hinder the achievement of the department's objectives.

Recommendation

• The Chief Financial Officer should, through the Integrated Risk Management Framework, provide policy direction requiring programs to regularly update their program and project risk profiles.

3.3 Compliance with Terms and Conditions

A variety of mechanisms exist to promote compliance with program terms and conditions. As part of its management framework for transfer payment programs, IC has developed a checklist that guides all aspects of project management. Some programs have also developed quality assurance mechanisms aimed at ensuring that projects selected complied with Programs' Terms and Conditions and policies in place. The variety of programs examined provided the auditors a firsthand appreciation of how transfer payment programs are managed. The auditors assessed the mechanisms for selecting the applicant, the content of the contribution agreement and performance monitoring exercised by Programs. A sample of 60 projects files were selected at random from the six programs and regions visited.

Finding 5: Strong Due Diligence During the Application Process

Each program examined had established mechanisms to facilitate the application process. In some cases, a pre-application screening process had been developed to facilitate the application process and ensure that applicants met the basic eligibility criteria.

The department acts as an intermediary for Infrastructure Canada under the COIP, COMRIF and CSIF programs and a Memorandum of Understanding has been signed to clarify the roles and responsibilities of both departments. Where the department co-shares the delivery of a program with a province (i.e. COMRIF), we found that substantial effort had been devoted to harmonize the application, claims processing, and monitoring activities.

Interviews with staff revealed knowledge of their programs and an understanding of who is eligible for funding, under what conditions and for what purposes. Letters of Offer and contribution agreements developed are in line with approved templates previously reviewed by Legal Services. Project Officers understand that they cannot deviate from the approved templates when completing a letter of offer or a contribution agreement.

Finding 6: Tracking and Monitoring of Repayable Contributions

The Program Policy and Management Directorate (PPM), Industry Sector, tracks, monitors and recovers repayable contributions for most departmental contribution programs. In so doing, PPM follows-up with recipients of repayable contributions in order to facilitate repayment of the contribution or parts thereof when conditions for repayment are met, and maintains a tracking system for such purpose. For some programs, formal arrangements exist for the provision of these services; for others, similar agreements do not exist, such as: CATIP, Softwood Lumber and the Canada-QuC)bec Agreement, for which it also offers recovery services. An existing MOU with ITO was not renewed in 2007–08.

In 2005–06, the sales audit function was transferred from PPM to ITO but the responsibility to manage the tracking, monitoring and recovery of repayable contributions and analysis of sales reports remains with PPM. PPM remains responsible for conducting sales audits for all other programs for which it manages collections.

3.4 Operational and Financial Monitoring

Finding 7: Weak Links Between Project Risks and Monitoring

As noted, programs are required to explain their planned monitoring activities in the program's RBAF document. While most programs have developed a means for assessing risks at the project level, few link the results of their risk assessments to their project monitoring strategy. The audit also noted that contribution agreements often identify monitoring requirements that are not always carried out (i.e. maintaining separate bank account; providing a financial statement at year end).

Programs have improved their financial monitoring practices when processing claims, with more attention paid to reviewing progress reports and linking these reports to the financial activities. Project Officers and Payment Monitoring Officers select a sample of invoices for verification during claim processing rather than verifying all invoices.

Most projects do not have a precise operational (performance) monitoring strategy and when operational monitoring is carried out it is usually on an informal basis. We noted that FedNor is implementing a new approach to operational monitoring and has linked its project monitoring strategy to the identified risks and now tracks the monitoring activities carried out against that strategy.

When site visits are undertaken, they generally consist of informal meetings aimed at obtaining information on progress, or in examining whether environmental conditions were respected. Monitoring activities aimed at understanding the internal financial controls and examining financial systems supporting the management of the funds received by recipients, conducting a review of expenditures and/or file reviews of the recipient are rarely undertaken. Information on monitoring activities typically remains in Project Officers' files and is not reported further.

Larger programs reported that they have adequate resources to manage the delivery of their programs while smaller programs indicated that they have too few officers to carry out monitoring activities.

Recommendation

• The Chief Financial Officer should establish policy direction requiring programs to link their monitoring strategies to the identified risks and establish tracking mechanisms to account for monitoring activities.

3.5 Information for Management

Finding 8: Improved Performance Management Required

As previously noted, CMIS has weak functionalities with respect to capturing data on program benefits, which hinders the reporting of program performance. We noted that there is no common process within the department to track benefits and there is no entity assigned responsibility to enforce the collection of benefits-related information. This affects the ability of the department to report on program results in a systematic fashion.

A review of the mechanisms used by programs to collect benefits information noted that while some programs have developed individual applications to track benefits, there is no common effort to develop this capacity at the departmental level.

Recommendation

• The Chief Financial Officer should establish policy direction with respect to requirements for benefits tracking.

Finding 9: Various Approaches to Conducting Recipient Audits

The audit noted a variety of approaches used to carry out recipient audits. An organization such as ITO, which delivers several large programs including TPC and SADI, has a structured recipient audit program, with dedicated staff and systems to support the management and conduct of its cost, sales and compliance audits. Other, generally smaller programs have adopted a less structured approach for recipient audits and typically only undertake audits if there is sufficient funding to do so. In almost all instances, recipient audits are contracted out to consulting firms.

Follow-up on recommendations arising from recipient audits also varies from program to program. In general, programs do not roll-up the results of recipient audits to report on what works and does not work with respect to compliance by recipients of their contribution agreements. Further, there is no mechanism by which programs can share information on their recipient audit practices from a lesson learned standpoint.

Finding 10: Evaluations Consistent with RMAF Requirements

Under the TB Policy on Transfer Payments, an RMAF is required for program approval of terms and conditions for grants to a class of recipients or for contributions and must be presented to TBS for review and approval as part of a related Treasury Board Submission. An RMAF provides Program Managers with a road map to plan, monitor, evaluate and report on the results throughout the lifecycle of a program, policy or initiative. The Government of Canada's Evaluation Policy (April 2001) also encourages the development of an RMAF. The RMAF integrates the evaluation function within the context of results-based management and supports managers and decision-makers in objectively assessing program and policy results. More recently, and as part of the actions arising from the implementation of the Federal Accountability Act, all departments are required to conduct a review every five years of the relevance and effectiveness of each ongoing program for which it is responsible.

In response to these requirements, Program Managers have requested that Audit and Evaluation Branch–Evaluation Directorate, undertake evaluations of their programs in line with the approved RMAFs. To date, the Audit and Evaluation Branch has been able to accommodate this increased demand for evaluations.

4.0 Appendix A–Summary of Transfer Payments Programs Over $5m for 2005–06 and 2006–07
(based on the Reports on Plans and Priorities for 2005–06 and 2006–07)

5.0 Appendix B–Detailed Audit Criteria

Capacity to Deliver

Sub-objective

The department has the capacity to effectively deliver and administer its transfer payment programs.

Criteria

• There is a centre of knowledge for Gs&Cs in IC to assist with program management.
• Available resources (human, physical, financial and other) match the workload of programs.
• The Contribution Management Information System (CMIS) contributes to the efficient delivery and administration of transfer payment programs.
• Program staff has the knowledge, the training and the time to properly assess applications and recipients.
• Program staff has the information, tools and essential resources to do their job well (policies/procedures).
• Responsibilities of Program Managers and Project Officers are clear.

Risk Management

Sub-objective

IC has an adequate and appropriate framework for managing risks to its portfolio of transfer payment programs.

Criteria

• The corporate risk management strategy takes into account the risks relating to delivering transfer payment programs.
• IC has developed a framework for managing transfer payment programs that take into account the risks relating to delivering transfer payment programs.

Sub-objective

IC Transfer payment programs have developed implemented and utilized an adequate and appropriate program risk management framework.

Criteria

• Intended program objectives (results) are stated in clear and measurable terms in programs' RMAF.
• Unanticipated negative results (risks) have been defined in programs' RBAF.
• Programs have developed in their RBAF risk mitigating strategies commensurate to defined risk tolerance level and have implemented the strategies.

Compliance to Transfer Payment Program Terms and Conditions

Sub-objective:

Transfer payment programs are:

• Managed and delivered in keeping with the specific provisions of the approved terms and conditions for the program; and
• Duly approved when a substantial change occurs.

Criteria

• Conditions imposed by TB as part of approving the terms and conditions of a program are met and/or complied with.
• IC systems, procedures and resources for ensuring due diligence in approving transfer payments and verifying eligibility and entitlement and for the management and administration of the programs are in place.
• Rules for eligibility and/or for deciding on which projects to fund are clearly stated and complete and are directly related to the expected program objectives.
• Staff understand who is eligible for funding, under what conditions, for what purposes and in what amounts.
• Selected projects are consistent with the approved program terms and conditions.
• There is an appropriate process for drafting, revising, signing and approving all contribution agreements and their amendments.
• Agreements for approved project have objectives, deliverables and expected results are consistent with TB approved programs' terms and conditions.
• Both RMAF and RBAF are resourced and implemented as approved.
• Suitable mechanism exist to track and monitor recipient of repayable contributions.

Operational and Financial Monitoring

Sub-objective:

Policies and procedures exist and are utilized to ensure that monitoring is made on the basis of the terms set out in the contribution agreement and the risks of the project.

Criteria

• Risks of the project are assessed as part of the application review.
• Documentation on the departmental files supports the risk analysis and the monitoring activities performed (progress reports; review of Financial Statements; site visits; cash flow requirements; post final payment monitoring).
• Appropriate actions are taken when monitoring activities revealed a situation requiring a managerial intervention.

Information for Management

Sub-objective:

Parliament, IC senior management and program managers receive timely, sufficient and appropriate program performance information to make informed decisions.

Criteria

• Ongoing program performance information is identified, collected, analyzed and appropriately reported.
• Recipients of contributions are subject to an independent audit of the compliance to the terms of the contribution agreement, including the achievement of planned results and this in accordance with the strategies developed in the Program's RBAF.
• Evaluations are performed as per approved RMAF.
• Program management uses findings from ongoing performance information, recipient and internal audits and evaluations.
• The Departmental Report on Plans and Priorities includes supplementary descriptive material, such as stated objectives, expected results and outcomes and milestones for achievement.
• The actual results of the program b€" compared to expected results–are adequately reported to the Canadian public and to Parliament through the Departmental Performance Report.

Management response and action plan

¹ The major types of transfer payments are grants, contributions and "other transfer payments" as defined by the Policy on Transfer Payments. This audit covers only grants and contributions, sometimes referred to collectively as 'transfer payments'. Examples of 'other transfer payments' are transfers to other levels of government such as Equalization payments or Canada Health and Social Transfer payments. (Back to text)

² The Board does not review recommendations nor make decisions on amendments submitted under the Aboriginal Economic Programs. These are reviewed and recommended by the National Aboriginal Economic Development Board. Note that responsibility for these Programs was consolidated in Indian and Northern Affairs in December 2006. (Back to text)

Figure 1: Diagram A—Current G&C Systems Landscape

Diagram A is the Current G&C Systems Landscape, an Extract from 2007 CIO deck "Grants and Contributions Sustainability Strategy". The diagram shows the current family of Grants and Contributions Management Systems. The background is split in half, the left side identifies the "Back Office" (Corporate Finance) and the right side identifies the "Front Office/Program Office".

The diagram consists of many bubbles that represent the agents, the systems and the programs. In the Back Office we find the FRS, IFMS (SAP), Proactive disclosure, CMIS, and Oracle. In the Front Office we find the Lotus Notes, eGC, GCRS, IT Data Warehouse, ITO AIU Automation, PPM tracker, FedNor forms, e-claims (FedNor), and approximately 15 other organisations.

There are arrows between the bubbles, both in the back and front office, showing the links between the systems and the organisations.

Back to Figure 1