Table of Contents

1. 1.0   Executive Summary
   1. 1.1   Audit objective
   2. 1.2   Scope and methodology
   3. 1.3   Findings
      1. For ICMS
      2. For THS
      3. For both ICMS and THS
   4. 1.4   Conclusion
2. 2.0   Introduction
   1. 2.1   Background
   2. 2.2   ICMS
   3. 2.3   The Interpreters Program
   4. 2.4   Temporary or Casual Help Services
   5. 2.5   Contracting at the IRB
   6. 2.6   Scope
   7. 2.7   Objective of the audit 3
   8. 2.8   Methodology
   9. 2.9   Audit Criteria
   10. 2.10   Sample selection
       1. For ICMS
       2. For Temporary Help Services
3. 3.0   The Procurement Initiation Process
   1. Observations
   2. ICMS
      1. Lack of a proper commitment certification as required by section 32 of the FAA
      2. Most Statements of Work and deliverable definitions were vague and lacked clarity
      3. Work started prior to the award of the contracts
   3. Temporary Help Services
      1. Commitment certification under section 32 of the FAA showed some weaknesses
      2. Statement of Work or task description was missing
   4. Recommendations
4. 4.0   Bid Solicitation and Contract Award Process
   1. Observations
   2. ICMS
      1. The sourcing strategies to solicit bids either competitively or non-competitively were not in compliance with policy
      2. When competitive bids were requested, the procurement process showed various weaknesses
      3. The contracting authorities were not always exercised according to the delegation of signing authorities
   3. Temporary Help Services
      1. Selection of suppliers not supported and/or appropriately documented
      2. The contracting authorities were not always exercised according the delegation of signing authorities
   4. Recommendations
5. 5.0   Contract Administration
   1. Observations
   2. ICMS
      1. Contract amendments evidence missing
      2. Contract administration showed various weaknesses
   3. Temporary Help Services
      1. Call-up amendments showed various weaknesses
      2. Contract administration showed various weaknesses
      3. Unacceptable staffing arrangements were observed
   4. Recommendations
6. 6.0   Audit Trail
   1. Observations
   2. ICMS
   3. Temporary Help Services
   4. Recommendations
7. 7.0   Conclusion : Internal Controls
   1. Recommendations
8. Appendix A Audit Objectives and Criteria
9. Appendix B Contracting Practices Review Checklist
10. Appendix C Management Response and Action Plan

Acronymes

Acronymes	Description
ACAN	Advance Contract Award Notice
ASC	Audit Services Canada
CPSB	Corporate Planning and Services Branch
CRC	Contract Review Committee
CSA	Corporate Security and Administration Directorate
FAA	Financial Administration Act
GCRs	Government Contracts Regulations
GOL	Government on Line
IAD	Immigration Appeal Division
ICMS	Integrated Case Management System
ID	Immigration Division
IPS	Informatics Professional Services
IRB	Immigration and Refugee Board of Canada
NAFTA	North American Free Trade Agreement
PAM	Procurement and Assets Management
PMO	Project Management Office
PWGCS	Public Works and Government Services Canada
RFI	Request for Information
RFP	Request for Proposal
RPD	Refugee Protection Division
SO	Standing Offer
SOW	Statement of Work
TBS	Treasury Board of Canada Secretariat
THS	Temporary Help Services

1.0  Executive Summary

1.1  Audit objective

The audit objective was to determine to what extent management complied with applicable regulations, policies and procedures when awarding contracts and to examine the framework or processes put in place by IRB management for contracting services to develop ICMS between February 2003 and December 2006 and to acquire THS.

The audit sought to ensure that

• the contracting process was open to fair competition,
• the contracting principles of best value were followed,
• Canadas interests were protected when contracting for services,
• contracts were properly managed, and
• contract files were fully documented to maintain an audit trail.

1.2  Scope and methodology

On November 17, 2006, the IRB mandated ASC to perform an audit of its contracting practices in three areas: the ICMS project, the Interpreters Program and THS. The Interpreters Program was subsequently de-scoped from the review. This report therefore covers ICMS and THS. The period under review included ICMS contracts issued from 2003 until November 20, 2006, and THS call-ups for fiscal years 2005-2006 and 2006‑2007.

Although a definitive figure could not be established, it is estimated that 173 contracts were issued for ICMS for a total value of $28,601,326.39, including amendments and taxes. Close to 60 THS call-ups were issued by the IRB during fiscal year 2006-2007 for a total value of $1,614,921.05, and 30 call-ups in 2005-2006 for a total value of $355,205.00. Before ASC's engagement in the audit process, the IRB's Audit and Evaluation services conducted a preliminary survey of 50 ICMS contracts, which led to establishing a sample of contract files that appeared more significant than others in terms of contract duration and dollar value. ASC concentrated its efforts on 11 ICMS files out of a wider sample of 50 files for a total contract value of $2.6 million. For THS, 15 procurement files were reviewed for fiscal years 2005-2006 and 2006-2007 with a total value of $779,787.45.

The contract files were examined against a checklist based on detailed criteria for the analysis of each phase of the procurement process applied in the federal government, which includes procurement initiation, sourcing of potential bidders, bid solicitation phase, bid evaluation, contract award, contract administration and contract file closeout. Work for this audit was carried out between December 2006 and March 2007.

Criteria for this audit were developed in support of the audit objectives, and were based on TBS Contracting Policy, the GCRs and IRB Contracting Policy.

1.3  Findings

Major findings include:

For ICMS:

1. Certification of availability of unencumbered funds has not been conducted in compliance with section 32 of the FAA.
2. Professional services contracts issued for the development of ICMS could lead to a perception of an employer/employee relationship.
3. Some contracts showed evidence that the contractor started work prior to the award of contracts.
4. The procurement processes in support of some competitive and non-competitive contracts were not in compliance with TBS Contracting Policy.
5. Some amendments included an increase in the level of effort or an extension to the contract period that essentially gave the contractor more time and money to deliver the work than originally defined.
6. Contract administration exhibited absence of tracking deliverables as payments were made based on time sheets only and not on actual deliverables and several files did not totally follow requirements certification under section 34 of the FAA.

For THS:

1. THS files were poorly documented or not documented at all with respect to sourcing and selection of suppliers. Most files did not contain appropriate support as to how suppliers were selected and why lower priced suppliers were not selected, as required by PWGSC from users of its THS SOs.
2. The THS files contained either no work description at all or a list of experience, knowledge and ability requirements instead of a work description that would have allowed THS firms to submit best-suited resources.

For both ICMS and THS:

1. A number of ICMS and THS contracts and contract amendments were not approved or signed by an incumbent holding the appropriate authorities as per the instrument of delegation of signing authorities in effect at the time the documents were signed.
2. The absence of proper documentation on several files was also noted.

1.4  Conclusion

The contracting practices applied to the sample files examined for ICMS and THS lacked consistency and in many instances were not fully compliant with the GCRs and Treasury Board Contracting Policy. Although management strived to award and manage its contracts in relation to best value, open access, fairness and transparency, the general quality of contract file documentation does not consistently demonstrate achievement of these objectives.

While this audit was under way, the CPSB took several steps to reinforce procurement management practices, most notably in the area of delegation of financial and contracting authorities. A global assessment of the management control framework for contracting activities is required, however, where roles, responsibilities and accountabilities for contracting and related financial activities should be clarified and clearly communicated to all parties concerned, including IRB regional offices. To ensure that internal controls are operating as intended, monitoring mechanisms should be established, particularly where there are significant residual risks.

In my professional judgment as Chief Audit Executive, sufficient and appropriate audit procedures have been conducted and evidence gathered to support the accuracy of the conclusions provided and contained in this report. The conclusions are based on a comparison of the conditions, as they existed at the time, against pre-established audit criteria listed in Appendix A. This conclusion is based on the sample of files examined. I can express no opinion on the entire population of ICMS and THS files for the period from February 2003 to December 2006. Furthermore, it is to be noted that there may be other evidence not obtained during the course of this audit that could affect the nature of the findings on these files. The evidence was gathered in compliance with Treasury Board Policy of Internal Audit (2006), directives and standards on internal audit.

Chief Audit Executive

________________________________
[Signature]
Mary Ellen McGuire

2.0  Introduction

2.1  Background

The IRB is Canada's largest independent administrative tribunal that renders decisions on immigration and refugee matters, efficiently, fairly, and in accordance with the law. It is comprised of three Divisions: the Refugee Protection Division (the RPD), the Immigration Appeal Division (the IAD) and the Immigration Division (the ID).

To achieve its mandate, the IRB relies on Governor-in-Council appointed decision-makers and public service employees. It also relies on contracting services to achieve major projects and program delivery objectives in different areas such as information technology, interpretation of proceedings, translation of documents and transcription of proceedings. Depending on fluctuating work volumes and priorities, it also hires temporary or casual help from time to time in a wide variety of positions within the organization.

Since 2003, the IRB's main contracting activities were in support of three areas: the development of ICMS to replace the System for Tracking Appellants and Refugees (STAR), Interpreters Services and THS.

2.2   ICMS

Until April 2007, the IRB's case management system was operating on outdated software that increased the risk of unrecoverable errors and business interruption. STAR tracked paper files as they moved sequentially through a manual case handling process and did not allow for the matching of relevant case law and country of origin documentation. Consequently, in order to ensure continuous, cost-effective, and high levels of productivity, the need for a new and integrated case management system (ICMS) was identified to support and automate the IRB's re-engineered processes and in turn improve processing time, reduce pending inventory, promote consistency in decision-making and enhance the protection of refugees and the overall security of Canadians.

The IRB's move to develop a web-based ICMS largely reflects the Government On-Line strategy, which promotes the introduction of information management of all types toward electronic access, storage, security and maintenance over the Internet. With the ongoing ebb and flow of refugee population and the increasing volume of immigration appeals, there is a greater need to optimize storage, improve access to and protection of information, facilitate the sharing of information between the regions and allow parallel flows to increase the speed of processing cases. The ICMS project was proposed as a means of addressing these pressing needs.

The ICMS project was intended to be achieved in a series of major deliverables for each of the Divisions that were called project milestones. Stage 1 consisted of providing the RPD with ICMS; Stage 2 for the IAD and Stage 3 for the ID.

Only Stage 1 was completed and delivered in April 2007. Stage 1 was rolled-out in several consecutive Releases. Release 1 (2004) pertained to claim identification and preparation; Release 2 (2005) for research processes; Release 3 (2005) for Electronic Personal Information Forms and Release 4 for automating the business processes of the RPD. Stages 2 and 3 were de-scoped.

Since the beginning of the ICMS project in 2003 and as of November 20, 2006, 173 contracts had been issued for the ICMS Project for a total value of $28,601,326.39, including amendments and taxes. This information was obtained from PAM and was not audited by ASC external auditors.

2.3  The Interpreters Program

The IRB's Interpreters Unit provides interpretation services to people who appear before the IRB in the RPD, the ID and the IAD. Over 90% of IRB proceedings require interpretation services. The IRB provides accredited interpreters with an orientation to IRB processes. Interpreters are employed on a contractual basis and are not IRB employees.

The IRB provides interpretation in over 260 languages in some 40,000 to 50,000 proceedings a year. Quality interpretation is a prime consideration for quality decisions and the IRB has developed a rigorous testing and accreditation system to ensure that interpreters at the IRB are well qualified.

On October 4, 2006, the IRB launched a RFI addressed to the members of the interpreter services industry and the general public to seek information to assist in the development of a new procurement strategy for interpretation services.

The RFI lists areas of interest related to interpretation services such as qualifications, interpreter scheduling and procurement options, on which the IRB is looking for information and feedback. The information gathered through the RFI process will be reviewed internally and may be used to assist with the development of procurement documents.

2.4  Temporary or Casual Help Services

The IRB occasionally requires the support of different categories of resources to assist in the delivery of its various programs. Depending on the nature of the requirement, the IRB will obtain temporary help either through the use of SOs put in place by PWGSC or by using casual employment.

During the last two fiscal years (2005-2006 and 2006-2007), the IRB issued for $355,205 and $1,614,921 of call-ups respectively against existing SOs to obtain various categories of resources.

2.5  Contracting at the IRB

Before June 2006, the procurement function at the IRB was performed by the Finance, Administration and Security Directorate at the CPSB. Contracting authorities were decentralized throughout the organization as managers could, and still can, sign contracts in accordance with their delegated authorities.

As of April 2006 the PAM, established in the Corporate Security and Administration Directorate (the CSA) at the CPSB, began to establish proper processes and procedures for contracting for goods and services. The PAM is comprised of a manager and three officers and is charged to develop processes for the entire procurement cycle. To that end, the Directorate was developing a new contracting policy that will include the role and responsibilities of the Contract Review Committee (the CRC) and clear delineation of financial, procurement and contracting authorities. Procurement is centralized in the National Capital Area and regional offices must send all requirements for $25,000 and above to headquarters.

At the beginning of the ICMS project, the IRB awarded professional services contracts for project resources using, as often as possible, the IPS tool provided by PWGSC. IPS can be used for requirements valued at less than the NAFTA threshold, which fluctuates but is usually around $84,000. It provides a list of professional services suppliers from which client departments can select suppliers from which to solicit bids. The suppliers are not pre-qualified by PWGSC and the client department is responsible for the procurement process. IRB personnel interviewed during the course of this audit mentioned that, at the beginning, the procurement was done somewhat loosely. There were no formal selection criteria and no evaluation grid. Selection of a contractor was mostly based on subjective criteria or simply on judgment.

Following this first period, the IRB benefited from the dedicated services of a PWGSC Procurement Officer for a period of one year. The PWGSC Officer was co-located with the ICMS PMO and developed a more formal procurement process for the IRB, which included putting several GOL professional services contracts in place for the ICMS Project.

The GOL Supply Arrangements are established by PWGSC, which provide a list of pre-qualified suppliers in different areas of expertise. Client departments submit a requisition and SOWs to PWGSC, which runs a competition process between pre-qualified suppliers. The fact that suppliers are already pre-qualified accelerates the procurement process and allows a reduction in the bidding period. While PWGSC handles the solicitation, the client department is responsible for the SOW and evaluation of technical requirements.

To help with the development of SOWs and the technical evaluations, the IRB hired three consultants through sole source contracts to form an evaluation team. While PWGSC did not participate in the technical evaluations, a PWGSC representative usually attended the interviews led by an IRB employee from the Project office and not by a consultant. Upon the departure of the PWGSC resource, the ICMS Project Office Administrative Support Officer assumed her role.

For THS, call-ups/contracts were predominately issued in headquarters and against PWGSC SOs. IRB Regional offices used very few THS. THS call-ups were issued by the Finance, Administration and Security before the establishment of the PAM. Currently, PAM only processes headquarters requirements.

2.6  Scope

• The contracting period under examination was from 2003 to November 20, 2006, for ICMS and fiscal years 2005-2006 and 2006-2007 for THS.
• After discussion with the Director, Corporate Planning and Management Practices, it was determined that the first area subject to the audit would be the ICMS Project, in order to provide the IRB Chairperson with the audit results by mid-January 2007.
• The audit of contracting practices related to the ICMS Project pertained only to the procurement processes followed to issue and administer IT contracts. The pertinence and quality of the ICMS system itself were not part of this audit.
• The activities handled by PWGSC on behalf of the IRB fall outside the scope of this audit as this audit pertains only to the IRB's contracting practices.
• The audit focussed only on THS obtained through procurement activities. Casual help was not part of this audit as casual help is obtained through human resource activities and the staffing process. As such, casual help falls outside the scope of this review, which is limited to procurement activities.
• In addition to ICMS and THS, the scope of the audit originally also included the Interpreters Program. After discussion with IRB internal audit representatives, it was determined that it would be premature to audit this sector of activities given the ongoing initiative to redefine the procurement process in this area. Therefore, this portion of the audit was consequently de-scoped.
• Insofar as this audit is to verify compliance with contracting policy, it covers the following procurement phases:
  1. contract initiation process;
  2. sourcing of potential bidders;
  3. bid solicitation process;
  4. bid evaluation and contractor selection;
  5. terms and conditions of the contract, the role of contracting authorities and the issuance of contract amendments; and
  6. contract administration.

2.7  Objective of the audit

The audit objective was to determine to what extent management complied with applicable regulations, policies and procedures when awarding contracts and to examine the framework or processes put in place by IRB management for contracting services to develop ICMS between February 2003 and December 2006 and to acquire THS.

This objective includes ensuring that

• all important contract planning issues were covered and documented during the procurement initiation process,
• the contract bidding and award processes were conducted following an open and fair competitive process and, in case of non-competitive contracts, fully justified and supported by regulations and policy provisions,
• proper financial controls were applied, contract progress was monitored throughout contract duration and contract amendments were supported by appropriate justifications,
• contract closeout was achieved when all of the contractors' obligations were met, and
• all file documentation was gathered to ensure that a complete audit trail was maintained.

For each of these sub-objectives, a series of detailed criteria were developed to test and validate if they were met when processing each contract requirement subjected to the audit. These sub-objectives and related audit criteria are expanded in the following sections of this report. They are also summarized in Appendix A.

2.8  Methodology

In order to get preliminary information on the proposed audit, for both ICMS and THS, ASC external auditors met with the following IRB representatives:

• the Director, Corporate Planning and Management Practices;
• the Senior Advisor, Audit and Evaluation;
• the Manager, Procurement and Asset Management; and
• the Chief, Contracting and Procurement.

These meetings were aimed at validating the IRB's needs, clarifying concerns and planning the audit work on site. Concerns centred mainly on professional services contracts awarded for the ICMS Project and the length of time certain contractors were employed by the ICMS PMO.

This audit covers the contracting process from the initiation of the requirement to the issuance and management of the contract. Tests on financial signing authorities were limited to confirming that the incumbent had a valid financial signing authority for the period under review regardless of funding, area of responsibility and budget.

Before ASC's engagement in the audit process, the IRB's Audit and Evaluation Senior Advisor conducted a preliminary survey of close to forty ICMS contracts, which led to establishing a sample of contract files that appeared more significant than others in terms of contract duration and dollar value. The ASC examined this sample of files on a priority basis.

In addition, ASC external auditors reviewed pertinent reference documentation, which includes the following:

• the policy on the completeness, accuracy and authority of IRB financial transactions and related information;
• the delegation of financial signing authorities for the IRB, effective April 1, 2003, 2004, 2005 and 2006;
• specimen signature cards; and
• the Draft IRB Contracting Policy, dated November 29, 2006.

Throughout the ICMS review, interviews were conducted with IRB employees to gather information and to obtain clarifications, opinions and facts on ICMS. The following IRB representatives were met:

• the ICMS Project Manager;
• the ICMS Project IT Coordinator;
• the ICMS Project Architecture Team Leader; and
• the ICMS Project Office Administrative Support Officer.

Towards the end of the audit work, the audit team briefed the Director, Corporate Planning and Management Practices and the Senior Advisor, Audit and Evaluation, on the preliminary results of the review.

2.9  Audit Criteria

For each of the audit objectives, a set of criteria were developed based on applicable regulations and policy. Most important referenced regulations are the FAA and the GCRs. Applicable policies are those produced by central agencies such as Treasury Board and PWGSC, which include the Contracting policy and also internal IRB policies.

The audit criteria were developed to ensure that contract procurement was

• open to fair competition,
• aimed at obtaining best value for the money spent,
• effected by protecting Canada's interest through proper contractual terms and conditions,
• properly managed to ensure that goods and services were delivered and paid according to the terms of the contract, and
• fully documented and an audit trail was maintained to ensure that decisions were justified.

These criteria are included in the checklist attached as Appendix B, which was used to review each contract file.

2.10  Sample selection

For ICMS:

• As of November 20, 2006, 173 contracts had been issued for the ICMS Project for a total value of $28,601,326.39, including amendments and taxes.¹ As the primary concern of IRB Management was on professional services contracts, the IRB provided ASC with several professional services contracts files, which constituted the sample. Time only allowed the review of 11 files, of which 6 were entirely processed by the IRB and 5 were PWGSC GOL contracts selected for the audit examination. For these 5 files, only the activities under the responsibility of the IRB are included within the scope of this audit. The procurement activities conducted by PWGSC fall outside the scope of the audit.
• The 11 files reviewed totaled $2,563,702.00, including amendments and taxes. It is possible that this value is higher since one file in particular showed indications of three contract amendments, without the amendment documents being on file. The value of these amendments is therefore not included in this amount.

For Temporary Help Services:

It is estimated that sixty THS call-ups were issued by the IRB during fiscal years 2006-2007 for a total value of $1,614,921.05, and thirty call-ups in 2005-2006 for a total value of $355,205.00.² ASC external auditors reviewed a total of fifteen files from those periods with a total value of $779,787.45, including amendments and taxes. All fifteen call-ups were issued against a PWGSC THS SO. Of those fifteen call-ups, six call-ups appear to have been issued on a sole source basis due to their low dollar value and nine were issued after soliciting resumes from several THS suppliers.

The table below illustrates the size of the sample compared to the size of the population for each category of contracts (ICMS and THS). While this audit was conducted in accordance with standard audit practices, the information is based on a selected sample. There may therefore be other evidence not obtained during the course of the audit that could affect the nature of our conclusions.

Population vs. Sample

Category	Population		Sample
	Number of files	Value ($)	Number of files and % of population	Value ($) and % of population
ICMS	173	$28.6 M	11 (6%)	$2.6 M (9%)
THS	90	$2  M	15 (17%)	$0.8  M (40%)

3.0  The Procurement Initiation Process

The objective of a sound procurement initiation process is to ensure that, prior to the procurement process, sufficient funds are available and can be committed as required by section 32 of the FAA, which requires that certification is provided to that effect. To test this, the external auditors expected to find an internal document showing a signed certification under that section or some other evidence demonstrating that funds were encumbered before the date on which the corresponding contract was issued to the contractor. The encumbered funds must be sufficient to cover the value of the contract. The impact of an inadequate commitment translates, firstly, in generating an incomplete financial picture of the organization at any given time and, secondly, in impeding a sound decision process because the financial information base is inaccurate.

Furthermore, SOWs or a detailed description of required goods or services must be written to precisely describe the requirement and not to be unduly restrictive in order to open to a fair competition. These are the prerequisites to the solicitation and the contract. They should be clear and complete so that suppliers can submit appropriate bids. This also helps avoid misunderstandings, disputes and poor or non-performance by the supplier.

When the requirement is for the services of individuals, policy requires that no employer-employee relationship is created. Other considerations such as ownership of intellectual property resulting from the work or security requirements are also to be considered at this stage.

ICMS

a)  Lack of a proper commitment certification as required by section 32 of the FAA

Two of the eleven files reviewed were not properly authorized under section 32 of the FAA. In one procurement file, the certification was signed by an individual without delegated authority. In the second file, insufficient funds were encumbered to cover the value of the ensuing contract. In addition, it should be noted that in six files, the specimen signature card for the signatories did not include an end date. There is therefore no evidence that the signatories had the delegated authority to sign when they did.

It should also be noted that in two additional files, while the certification was signed before the contract was let, it was signed only after the procurement process had taken place (after the winning contractor was selected). There was therefore no internal requisition and approval to proceed with the procurement process beforehand.

b)  Most Statements of Work and deliverable definitions were vague and lacked clarity

In the 6 files entirely processed by the IRB, SOWs included the following statements:

• "The resource will receive specific tasks assignments from the Team Leader";
• "Other related duties as determined by Management"; and
• "The willingness of the resource to work overtime is not mandatory but may be requested" (sic).

In addition, there were no timelines associated with the deliverables listed.

A professional service contract should clearly define the work that the consultant must deliver within a very specific timeframe. It is then left to the consultant to determine how they will manage their time and other workload to deliver the contracted work in time. The risk associated with statements such as those referred to above is that the Crown can be seen as directing how and when the consultant works, which can lead to a perception of an employer/employee relationship. The absence of timelines also removes the obligation on the contractor to deliver specific work by the end of the contract period and may lead to contract extensions to perform the work or to the work never being completed.

In addition, two of the six IRB files included a two-page Contract Request Form where the second page addresses an employer/employee relationship. Both forms provided contradictory information. They indicated that there would be employer/employee relationship elements in the requested contracts, however both concluded with an incorrect indication that the contract duration would be less than twenty weeks (both contract periods were extended beyond twenty weeks) and that none of the employer/employee relationship elements listed apply, when in fact more than one was checked as applying.

With respect to the SOWs submitted by the IRB to PWGSC, ASC external auditors could not comment on the appropriateness of the evaluation criteria pertaining to the level and depth of experience required. The deliverables, however, have no timelines associated with them. In addition, the contract requisitions include several option years, which means that the contractor has several option years to complete the work. This gives the impression that the contracts were in fact for long-term temporary help rather than professional services contracts. The risk here is again one of an employer/employee relationship. The responsibility for verifying whether deliverables were in fact delivered resides primarily on the IRB delegated authorities who initiated the procurement process to ensure that contractors, through appropriate supervision and oversight, deliver the expected services related to the project at hand.

In at least three cases, the SOW submitted by the IRB to PWGSC was almost identical to the SOW on a previous IRB-issued contract and the existing contractor was also the selected contractor following the PWGSC competitive process. This resulted in certain resources being on the ICMS project for several years. In one particular case, a specific resource has been present since July 2003 and the firm has received in excess of $800,000.00 for the services of that resource. Although one cannot automatically conclude that these resources were favoured, the situation does draw attention and could warrant further investigation.

c)  Work started prior to the award of the contracts

Two procurement files showed evidence that the contractor started work prior to the award of contracts. In one case, the contract was signed after the start date of the contract and there is no evidence of correspondence giving the contractor the authority to proceed before the contract signing date. In another case, there was a replacement of personnel and the replacement started working before the contract amendment was issued. Both cases are evidenced by the invoices submitted by the contractors.

The fact that contractors start work prior to the issuance of an official written contract signed by both parties constitutes a risk to the Crown. The risk evolves from the pre-contractual arrangements that are generally verbal and therefore incomplete since, most likely, the terms and conditions of the written contract are not formally agreed upon by the various parties. In that context, if something goes wrong before the contract is signed by both parties, the contractor may invoke that the Crown is liable for the incidents that occurred before the contract was officially signed and this could degenerate into litigation where each party tries to demonstrate that the other party was at fault.

Temporary Help Services

a)  Commitment certification under section 32 of the FAA showed weaknesses

Section 32 signatories had delegated authorities; however, the specimen signature cards provided did not have an end date, except for acting appointments. This introduces a weakness in the control framework, since it is unknown during exactly which period the delegated authorities can be exercised.

In addition, in three cases, the signature required by section 32 on the internal requisition is not dated, which means it is impossible to determine exactly when the requisition was signed and to confirm whether or not the person signing had the delegated authority to commit funds.

b)  Statement of Work or task description was missing

Although a generic description is provided by PWGSC for the different labour categories, PWGSC requires that the work required by the client department be confirmed with the supplier. The THS SO stipulates that the client must provide a task description to the suppliers, as well as all relevant details, such as competency sought to allow suppliers to match the need of a specific job/task with the skills and experience of available temporary employees. Providing a SOW or task description also helps avoid misunderstandings and disputes as to the work and duties that will be expected from the temporary employee. It also reduces the risk of non-performance.

• Sole source files (6):
  • The services required on five files were administrative support services. No SOW or work description appear to have been communicated to the selected firms.
  • The sixth file was for Human Resources consulting services. The IRB internal client provided a SOW, which does not appear to have been communicated to the selected supplier.
• Competitive files (9):
  • On seven files, only a list of experience, knowledge and abilities was provided to suppliers. No work or task description was provided.
  • Another file contained a detailed SOW, but that document was not provided to the suppliers invited to submit resums. Instead, an e-mail listing experience requirements that mirrored the tasks listed in the SOW was provided to suppliers. None of the resums submitted were found acceptable and the IRB had to re-solicit resums.

To help suppliers identify appropriate resources, the IRB should provide SOWs or work or task descriptions to suppliers and not just a list of experience, knowledge and ability requirements, which could be construed as evaluation criteria. The SOW or work or task description should also be attached to the call-up document to avoid misunderstandings as to the work that is expected and reduce the risk of poor or non-performance by the temporary employee.

4.0  Bid Solicitation and Contract Award Process

The objective of a sound bid solicitation and contract award process is to ensure an open and competitive bidding process and that all contracts are awarded competitively unless sole source contracting is justified by one of the exceptions covered by the GCRs.

A prime criterion is to verify whether there were clear bid selection and evaluation procedures and that the proper terms and conditions were used both at bid solicitation and contract award stages.

A second criterion is to verify whether or not only IRB personnel holding delegated contracting authority approved contracts up to the amount permitted under their delegation.

A third criterion is to verify whether the contract price in contracts issued without competition was justified and properly supported as required by TBS Contracting Policy.

ICMS

a)  The sourcing strategies to solicit bids either competitively or non-competitively were not in compliance with policy

Of the six files processed entirely by the IRB:

• Two were awarded non-competitively, however the sole source justification as documented on file is not valid in accordance with Treasury Board Contracting Policy and the ACAN process was by-passed. The sole source justification invoked is that only one person or firm is capable of performing the work. Since these sole source contracts were awarded to keep existing resources on the project until PWGSC processed the requirements competitively, this defeats the argument that only one person or firm is capable of performing the work. Furthermore, it should be noted that Treasury Board Contracting Policy states that this exception should be invoked only where patent and copyright requirements, or technical compatibility factors and technological expertise, suggest that only one contractor exist. It should not be invoked simply because a proposed contractor is the only one known to management.
• Four were awarded on a competitive basis. However, the files do not contain information on how the source list was established or how the requirement was advertised. One of them referred to a separate folder for the selection process, but the separate folder could not be found. Personnel interviewed during the course of the audit mentioned that the IRB used the PWGSC IPS system to establish a source list but, since the files contain no information, it is impossible to determine whether or not the sourcing strategies were in compliance with policy for these four files.

b)  When competitive bids were requested, the procurement process showed weaknesses

Of the four competitive files processed entirely by the IRB:

• The Selection Process File is missing for one file. It is therefore impossible to comment on the procurement process phase.
• Two files did not contain a copy of the RFP, or information on how the source list was established and the requirement advertised. It appears that only an e-mail was sent to potential bidders with a copy of the SOW. If that is the case, the risk is that the bidders would not have known what terms and conditions would apply to the resulting contract. While the SOWs on file list the desired skills and experience, it is unclear what evaluation criteria and selection methods were used and if the bidders were aware of them before bidding. The files also do not contain evaluation tabulations or reports, only a summary as to why the winning resource was selected. Another concern on these two files is that the bids submitted do not contain a level of effort or number of days but the subsequent contracts do include such information. The external auditors could not ascertain whether the RFPs included such information or how and when the level of effort to which the bidders agreed to in their bids was determined (e.g. the contractor agrees to meet all terms and conditions called for in the SOW).
• While the fourth file does not contain a copy of the RFP or information on how the source list was established and the requirement advertised, the SOW on file includes the evaluation criteria and selection method. While these are more detailed and formal than on previous files, the following weaknesses were noted:
  • The evaluation included mandatory criteria, point rated criteria, an interview and a written test. The point rated criteria section required a pass mark of 2/17. However, the first point rated criterion required a minimum of two years' experience and two years of experience was worth two points. This is in fact a mandatory criterion and is in essence preventing bidders from obtaining the minimum two points pass mark with another point rated criterion.
  • The document states that the interview (which will be used to clarify the experience and to determine the depth of experience) and the written test will be evaluated based on pre-determined criteria, which are not published. As well, the use of interviews and written tests are reminders of a staffing process rather than a supplier selection process and this could introduce a perception of an employer/employee relationship. Furthermore, the fact they could result in additional information being added to the bid after the closing date introduces the possibility of bid repair, which is not permitted as it jeopardizes the integrity of the procurement process. A bid should be complete and entirely submitted upon bid closing.
  • The financial proposal instructions limit the per diem to be quoted at $650 for budgetary constraints reasons. Although budget limits are often used in research and development RFPs, here it could in fact prevent a true competition as all bidders could have quoted the ceiling per diem rate.
  • To be considered responsive, a bid has to meet all mandatory criteria and the minimum pass marks for the point rated criteria, interview and written test. But contractor selection was based on the resource that scored the highest on the interview questions and the written test only. The basis of selection did not take into account the points scored in the point rated criteria section (which renders that section useless and unfair to bidders) or the price. The price was not evaluated and served only as a potential tie-breaker, which means that the IRB could pay significantly more for a given resource only because its score was one point higher than the next bidder. As well, the emphasis was placed on the interview and written test, which resembles more a staffing process than a supplier selection process.
  • The evaluation report indicates that several resources were withdrawn by various bidders during the procurement process. There is no correspondence or evidence to that effect on file, however, to ensure that a proper audit trail was maintained and to support the evaluation and contract award decisions.

c)  The contracting authorities were not always exercised according to the delegation of signing authorities

Three files processed by the IRB showed that the ensuing contracts were not signed by an authorized incumbent in compliance with the delegation of signing authorities documents. To conduct this test, we examined the type of contract issued, the date of signature, the delegation of signing authority grid, the area of authority and the individual signature specimen card that specifically states the delegated authorities to which the incumbent is entitled. Each of the files examined showed contravention in at least one of these areas:

• For three files, the contract value was higher than the delegated authority of the person who signed the contract on behalf of the IRB.
• For three files, either there is no specimen signature card covering the period in which the contract was signed or there is no end date on the specimen signature card. In the latter case, it is impossible to determine if the signatory held contracting authority at the time of contract signature.

Temporary Help Services

a)  Selection of suppliers not supported and/or appropriately documented

All 15 files reviewed were for THS call-ups issued against a PWGSC THS SO. It is a condition of the PWGSC THS SO that the supplier offering the lowest rate be selected and that rationale be provided if a supplier other than the one providing the lowest rate is selected. Files must be documented in support of the selection.

• Six call-ups appear to have been issued on a sole source basis due to their low dollar value (less than $25,000). There is no explanation on file to confirm this, however. It is further unknown how the firms were identified and selected. With the exception of one file, there is no THS supplier list on file; it is unknown if the lowest priced firm in the labour category was selected. Additionally, there is no documentation on file to support the rate paid, again with the exception of one file.
• Nine call-ups were awarded on a competitive basis, i.e. where several firms identified after a search in the THS database were invited to submit resums.
  • None of the files contained detailed notes or documentation to explain why lower priced firms were bypassed and on what basis the selected firm was chosen. In some files, resums are very briefly annotated with the word no or not enough experience, but some have no annotations at all and there is no summary of the evaluation.
  • In one file, some resums were rejected for not meeting a knowledge requirement that was never communicated to the suppliers and/or for insufficient experience when the level and depth of experience was not specified.
  • In one file, the successful resource's resum is not on file and there is no evidence that it was submitted. Only a brief description seems to have been submitted by the firm in an e-mail. Although the text of this e-mail refers to an attached resum, there is no indication on the e-mail that there was indeed an attachment.
  • On another file, only a copy of the e-mail that was sent to suppliers inviting them to submit resums was found. No resums were found on file. It is unknown how the resource was found and selected.

It appears that the selection process followed by the IRB was to select the resources with the most experience or the best fit rather than selecting the lowest priced resource meeting the minimum requirements identified, which does not meet the intent of the THS SO. Caution should also be exercised when rejecting resources to make sure that it is on the basis of requirements that were well defined and communicated to the suppliers. There should also be an evaluation summary on file that indicates why lower priced resources were rejected. Without this, the IRB could find itself in the position of being unable to explain and justify why a given resource was rejected.

b)  The contracting authorities were not always exercised according the delegation of signing authorities

To conduct this test, we examined the type of contract issued, the date of signature, the delegation of signing authority grid, the area of authority and the individual signature specimen card that specifically states the delegated authorities to which the incumbent is entitled.

• For two files, the person who signed the call-ups as contracting authority did not have the delegated authority to do so.
• For ten other files, the signatories had delegated authorities; however, the specimen signature cards provided did not have an end date, except for acting appointments. This introduces a weakness in the control framework, since it is unknown exactly during which period the delegated authorities can be exercised.
• On one other file, it was impossible to identify the signature and therefore to determine whether or not the person had delegated authority.

5.0  Contract Administration

The objective of sound contract administration is to ensure that the contract is performed as set by its terms and conditions and that payment is made according to the payment conditions clearly stated in the contract. In addition, when contracts need to be amended, the amendments are to be issued for valid reasons and after being approved at the proper level as set by the delegation of authorities.

The criteria used to validate this objective include confirmation that deliverables have been received and that invoiced and paid amounts are in compliance with the basis of payment of the contract and that certification under section 34 of the FAA has been applied before payment.

ICMS

a)  Contract amendments evidence missing

With respect to the five files processed by PWGSC:

• For two files, the files contain indications that amendments were requested, but the files contain either unsigned copies of the amendments or no copies at all. There is therefore no evidence that amendments were really issued.
• For four files, the contract amendments or amendment requests included an increase in the level of effort to pay for overtime, when overtime was not included in the contract's basis of payment, or an extension to the contract period that essentially gave the contractor more time to deliver the same work. The amount of the contract was also increased to cover the additional period.

With respect to the six files processed by the IRB:

• For one file, amendments were issued to extend the period in which the contractor had to produce the deliverables and the contract value was increased to cover the additional period, despite the fact that the contractor had agreed to produce the work within the original contract period. No supporting justification was found on the file.
• In another case, the contract was amended to avoid losing the expertise of the contractor while PWGSC processed a requirement for a new contract. It should be noted that the contract's SOW included the transfer of skills and knowledge to the IRB.
• On the two files where contract amendments were issued, the person who signed section 32 did not date his signature so it is impossible to determine whether or not he had the delegated authority to do so, and the person who signed the contract amendments did not have the delegated authority to do so (that person had already surpassed his authority at contract entry, therefore he did not have the authority to further increase the contract value).

b)  Contract administration showed weaknesses

• None of the eleven files reviewed contained any indication that the deliverables listed in the contracts had been received. The only documents found on these files are copies of invoices based on time sheets and the time sheets themselves. IRB personnel interviewed during the course of this review confirmed that payments were based on time sheets and not on contract deliverables.
• In the case of one file, it is impossible to determine whether or not the invoiced amounts are in accordance with the contract's basis of payment since copies of the amendments are not on file.
• In the case of two files, it is impossible to determine whether or not the invoiced work is within the contract period since the files do not contain copies of all of the PWGSC amendments.
• In the case of two files, the total amount paid under the contract surpasses the contract value. In another file, this is impossible to determine since the file does not contain a copy of all of the amendments.
• Only two of the eleven files reviewed were compliant with respect to certification under section 34 of the FAA. All of the other files presented one or a combination of the following weaknesses:
  • signatures were not dated. It is therefore impossible to determine if the individual had the delegated authority when he/she signed; and
  • there is no evidence that the person signing under section 34 had the delegated authority to do so at the time as there is no specimen signature card for those dates.

Following the review of the eleven ICMS contract files and the submission of a preliminary report, an additional review was conducted on one particular case.

In this case, the services of three to four Senior Project Managers were required for Application development. From this unique requisition, PWGSC awarded three contracts, each to a different firm and each for one Project Manager.

Only one of the three required files was located in the IRB. Based on correspondence found in that file, all three contracts seem to have been awarded at the same time for similar periods. The contract found on file was awarded on March 22/04 for a first period ending March 31/04 and with three option years. The original resource on this contract provided services until the summer of 2005.

In this case, there is no documentation or explanation on file as to why the original resource stopped providing services under this contract. The only information found on file is an amendment to the contract dated October 11/05 for the replacement of the original resource by the resource already providing services under one of the two other contracts awarded by PWGSC for this requirement.

Again, there is neither documentation nor correspondence explaining the reasons behind this change of personnel, nor the impact on the contract under which the replacement resource was apparently already providing services. Based on an email from PWGSC in March 2004, the rate of the contract found on file was higher than the rate of the contract under which the replacement personnel was already providing services. When the resource on this latter contract replaced the original resource on the other contract, it was at this contract's higher rate.

Although it is not uncommon for a consultant to change firms, what is of interest here is that the replacement personnel were already providing the same services to the IRB through another firm at a lower rate. Also of interest is what happened to the contract under which this resource was already providing services? Did the contractor also provide replacement personnel? Did the IRB end up with only two programmers? In the absence of documentation, it is impossible to conclude.

Temporary Help services

a)  Call-up amendments had shortcomings

• One file appears to be incomplete in terms of documentation. It is therefore impossible to determine if amendments were issued.
• Six files contain call-up amendments that were issued to extend the period of work beyond the 20 week limit for THS. IRB internal policy stipulates that when doing so, a higher level of approval must be sought. In three cases, this was not done.
• Again, except for acting situations, specimen signature cards provided in support of section 32 of the FAA and contracting delegations do not have end dates. This introduces a weakness in the control framework since it is unknown exactly during which period the delegated authorities can be exercised and one cannot confirm that the signatories had delegated authority.

b)  Contract administration showed weaknesses

• In the case of five files, the period worked was shorter than the one stipulated in the call-up and/or the expenditure was lower than the estimated cost indicated on the call-up. While client departments can put a stop to a THS call-up at any time, the call-up should be amended to adjust (reduce) the time period and estimated cost. A call-up is a contract. By not amending the call-up, the contractual agreement still exists and the IRB exposes itself to the risk of the contractor claiming the entire amount originally estimated.
• In the case of another file, a call-up was awarded to a firm for the period from July 7, 2006, to March 30, 2007. On December 18, 2006, a GOL contract was awarded to the same firm, for the same resource for the period of December 18, 2006, to March 31, 2007, plus two option periods. While the invoices submitted for work performed under the THS call-up cover the period of July 7, 2006 to December 31, 2006, only, the THS call-up was never amended to reduce the period of work to end on December 31, 2006. This should have been done to avoid confusion, duplication and for the reason stated above.
• One file does not contain a time sheet for the first month. It is therefore unknown when the work started and if it started prior to contract award or not.
• There was over-expenditure on one file. While the amount over expended is minimal ($83.65), it may point to weaknesses in financial control mechanisms. Indeed, the payment summaries found on the financial file contain data entry and calculation errors that may have contributed to the over-expenditure.
• On one file, the period of the call-up ends November 11, 2006, however some work was conducted in December 2006 and invoiced.
• In the case of 5 files, specimen signature cards provided in support of delegation under section 34 of the FAA do not have end dates. This introduces a weakness in the control framework since it is unknown exactly during which period the delegated authorities can be exercised and one cannot confirm that the signatories had delegated authority. In addition, four of these five files contain invoices where the section 34 signature is not dated, which means that it is impossible to determine exactly when the invoice was signed and, again, to confirm whether or not the person signing had the delegated authority to do so.

c)  Unacceptable staffing arrangements were observed

It is a condition of the PWGSC THS SO that Federal Government Departments and Agencies are not to initiate casual, term or permanent staffing arrangements with a pre-qualified Supplier's temporary help service employee(s). Such action could create a liability for the federal government department or agency. Within industry, a Supplier is in a position to invoice when this type of arrangement is agreed to by all parties.

However, in two cases, the IRB hired the THS employees of a pre-qualified supplier as casual employees after having contracted for their services under the THS SO. These employees were again contracted for under the THS SO after the end of their casual employment.

6.0  Audit Trail

The objective is to ensure that all elements of the contractual process from initiation of the requirement until complete delivery of goods and services are completely and properly documented in the procurement files. This entails that all decisions taken during the procurement process are documented to ensure that a complete audit trail is maintained.

The criterion is completeness of documented evidence to support each phase of the procurement process.

ICMS

In addition to the missing documentation and information mentioned throughout this report, the following faulty documentation and records management were noted:

• one invoice was found on a previous contract file;
• one file contained invoices that did not pertain to the contract on file but to a previous contract with the same resource for the similar services; and
• one file contained an invoice that was paid under a different contract.

A specific file could not be found while others were difficult to locate. The specific file appeared to be for services identical to those of other existing service contracts.

Temporary Help Services

• One file contains only one invoice for three weeks of work. The call-up period, however, is for 3 months and there is no indication that the work stopped after three weeks. In the absence of the other invoices, it is impossible to determine if the amounts invoiced and paid were in accordance with the call-up's terms and conditions.
• One file contains an amendment that relates to a different file.
• One file contains only the documentation that refers to another file.

7.0  Conclusion: Internal Controls

The contracting practices applied to the sample files examined for ICMS and THS lacked consistency and in many instances were not fully compliant with the GCRs and Treasury Board Contracting Policy. Although management strived to award and manage its contracts in relation to best value, open access, fairness and transparency, the general quality of documentation of these contract files does not consistently demonstrate achievement of these objectives. As stated previously, no opinion can be provided on the entire population of ICMS and THS files for the period February 2003 to December 2006.

Based on the audit results, it is deduced that important aspects of the management control framework were weak, inefficient or absent. They pertain to, among other things, a poor quality of contract file documentation, inadequate risk management, unclear roles and responsibilities, lack of contract expertise or training, and the absence of control and monitoring mechanisms to ensure quality of contracting processes.

---

Appendix A Audit Objectives and Criteria

Objectives and Criteria

Main Objective
The audit objective was to determine to what extent management complied with applicable regulations, policies and procedures when contracting for services, and to examine the framework the IRB had in place for contracting services to develop ICMS, and for THS between February 2003 and December 2006.
SUB-OBJECTIVES	DETAILED CRITERIA
1.   PROCUREMENT INITIATION All important issues of contract planning are covered and documented.	Funds were encumbered as per s. 32 of the FAA. A SOW describes the requirement in sufficient detail to allow bid solicitation and is not unduly restrictive. The SOW does not open to employer/employee relationship. No work has started before contract award. Ownership of intellectual property and security requirements were addressed.
2.   CONTRACT BIDDING AND AWARD PROCESS The bidding and award process opens to fair competition and favours obtaining best value for money spent.	Sourcing strategies used favoured competitive procurement. When sole source procurement was conducted, it was in compliance with the exceptions provided by the GCRs. Bid selection and evaluation methods were detailed in the solicitation document and were applied integrally. An evaluation report supports the selection of the best bid. When a sole source contract was issued, contract price was supported to demonstrate that it was fair and reasonable. The contract was signed by an authorized IRB employee. Contract terms and conditions protect Canada's interests.
3.   CONTRACT ADMINISTRATION Contract is managed to ensure that goods and services are delivered and paid according to the terms of the contract.	Deliverables were provided by the contractor as stated in the contract. The amounts invoiced by the contractor were in compliance with the basis of payment of the contract. Payments were certified as required by s. 34 of the FAA by an authorized IRB employee. When contract amendments were issued, they were supported and were issued before contract expiry date by an authorized IRB employee.
4.   CONTRACT CLOSEOUT Responsibilities and obligations were discharged by both parties to the contract and the whole procurement process was fully documented to ensure that a complete audit trail is available.	All services and deliverables were provided. All of the contractor's obligations were discharged. The department has paid all claims as per the basis of payment stated in the contract. All elements of the procurement process are documented and filed to ensure a complete audit trail is maintained.

---

Appendix B Contracting Practices Review Checklist

---

Appendix C Management Response and Action Plan

Endnotes

1. Unaudited data obtained from a spreadsheet maintained by the IRB's PAM.
2. Unaudited data obtained from the IRB's Corporate Finance.

About PDF