ASSESSING EFFICIENCY, EFFECTIVENESS AND QUALITY:
AN EVALUATION OF THE ADR PROGRAM OF THE IMMIGRATION APPEAL DIVISION OF THE IMMIGRATION AND REFUGEE BOARD
FINAL REPORT
by Leslie Macleod
Executive Summary
March 2002
Background
The Immigration and Refugee Board (" IRB" or "Board") is Canada's largest administrative tribunal, mandated to meet Canada's obligations relating to immigration and refugee matters. One of the three divisions through which the Board currently carries out its mandate is the Immigration Appeal Division (" IAD"). The IAD is an independent tribunal that hears and decides various categories of appeals. This Evaluation reviews the ADR Program established by the IAD that applies to sponsorship appeals.
Canadian citizens and permanent residents are entitled to seek reunification with close relatives from abroad. For the sponsorship to be successful, both sponsors and the relatives that seek entry to Canada must meet certain requirements. If a sponsorship application is refused by a visa officer, the sponsor (Appellant) may appeal in an effort to have the family member (Applicant) approved for admission to Canada. The parties to the sponsorship appeal are the Appellant and the Minister of Citizenship and Immigration. The Minister is represented by Minister's counsel; the Appellant may be represented by counsel as well at hearings and in the ADR process.
IADADR Program
Consistent with the federal Department of Justice's policy that promotes "an accessible, efficient and fair system of justice", the IAD established a Working Group in July of 1997 to study the feasibility of introducing ADR into the IAD's adjudication system. After a year of consultation, planning, and development the IAD's ADR Program was launched in Toronto in July of 1998; it was implemented in Vancouver in April of 2000 after further study, refinement, and consultation.
The ADR Program has evolved over time. The IAD has made a number of changes to it in response to reviews of its effectiveness and particular circumstances that have arisen. Five primary types of sponsorship appeals have been eligible for ADR: adoption, marriage and fiancé(e), financial, medical, and criminal inadmissibility. Adoption cases were excluded in Toronto after initial indications that they rarely resolved through mediation and they were never included in Vancouver. The case types currently covered differ to some extent in Toronto and Vancouver. The Program is mandatory; opting out is permitted in limited circumstances. Parties are also allowed to opt in. Mediations are conducted by Dispute Resolution Officers (DROs), most of whom are tribunal members of the IAD.
Evaluation Objectives and Methodology
The IAD sought an independent Evaluator to conduct a comprehensive review of its ADR Program. It specified that it wanted both quantitative and qualitative analyses. The IAD's ADR Program was evaluated in relation to how well it met its three major objectives of providing a quality alternative to the adversarial hearing process, increasing the speed of dispute resolution, and improving the efficiency of dispute resolution, including reducing the financial and emotional costs of dispute resolution.
To fulfill the evaluation mandate, the Evaluator consulted with key stakeholders, observed mediation sessions, reviewed Program documents, administered questionnaires, analyzed data, and conducted research. The Evaluator used the information collected through these activities and its knowledge of the conflict resolution field to propose recommendations to the IAD.
Review of Qualitative and Quantitative Data
Consultations
Consultations with stakeholders uncovered a number of important issues. The Advisory Committees in Vancouver and Toronto debated fundamental aspects of the ADR Program, including whether or not the process used by the IAD was really mediation - a point raised by all the other groups as well. CIC senior management explained that while headquarters supported ADR, there was resistance at the levels of manager, Minister's counsel, and visa officers. It was suggested that it would be helpful to have selection criteria for Minister's Counsel doing ADR. Minister's counsel, particularly in Vancouver, questioned the utility of ADR and suggested that the benefits of early resolution could be achieved more cost effectively through other means. In Toronto, Minister's counsel pointed to a number of benefits of mediation including the participation of DROs and improvements in their relationships with Appellants' counsel.
Appellants' counsel in Toronto and Vancouver supported the concept of ADR while citing certain concerns. Counsel in both locations said that Minister's counsel play a critical role in the process because they have the power to decide whether to recommend that appeals be allowed. The view in Toronto and Vancouver was that Minister's counsel do not exercise their discretion equally well. Appellants' counsel and Minister's counsel all agreed that not all cases are suited for ADR. There was also general support for opting-out and for members taking an evaluative approach to mediation.
Consultations with DROs in Toronto and Vancouver revealed that there are significant differences in the ways they perform their roles. For example, some are evaluative and others are facilitative. DROs also spoke of the need for Appellants' counsel to be better educated about ADR and to improve their competence. While they agreed that some cases were not appropriate for mediation, they thought that the parties alone should not be able to opt-out. The DROs felt that more cases could be mediated with existing resources. DROs questioned CIC's and Minister's counsels' commitment to ADR, citing a reluctance to exercise their discretion. As did other groups, DROs said that there was scope to expand ADR to additional categories of cases. All the groups agreed that the ADR Program offered certain advantages to Appellants; even if the case did not resolve, Appellants were more prepared for the hearing to come. A number of stakeholders, however, raised a concern that failed ADR cases take longer to reach a hearing than do cases never streamed into ADR and that, in Toronto, failed ADR cases generally take longer to hear. IAD tribunal members in Toronto supported the ADR Program and its goals and expressed an interest in learning more about it.
IAD management recognized the unique nature of the IAD's ADR Program and spoke of their commitment to improving it. They emphasized the importance of ensuring that the public interest was well served by ADR through operating the Program in a transparent way, making clear recommendations to members for approval of consents to allow appeals, and excluding cases for which mediation is not appropriate.
Questionnaires
The Evaluator administered confidential questionnaires to DROs, Minister's counsel, Appellants, and Appellants' counsel during mediations conducted in a single period of six weeks. The questionnaires were specifically designed for each group; to address common issues, many of the questions were the same or similar. The survey included 100 cases from Toronto and 20 from Vancouver. Responses were received from virtually every participant, affording the survey a very high degree of reliability.
Many key points emerged from the survey. Approximately 40% of the mediations resulted in the appeal being allowed and 4% resolved through withdrawals. In 40% of the cases a hearing was scheduled. The remaining cases did not proceed or were withdrawn. A very high percentage of the each group of respondents (from 81% to 94%) found the mediation process to be fair and worthwhile and each group noted beneficial features of ADR such as providing useful information. Most mediations took approximately one hour to complete. Appellants participated actively in 85% or more of the cases although their participation was largely limited to answering questions. The performance of DROs was extremely highly rated; Minister's counsel and Appellants' counsel were subject to more critical ratings and commentary. DROs expressed the most concern about the insufficiency of information received prior to mediation. One of the consistent themes was that not all cases should proceed to mediation.
Databases
The Evaluator analyzed the separate databases maintained by the IRB and the IAD to gather information relevant to the mediation of sponsorship appeals. Key performance factors included the disposition rate of sponsorship appeals and how the average sitting times and processing times for mediation sessions and hearings compared. 3,347 cases disposed of in the period from January 1, 2001 to September 30, 2001 were selected for analysis.
Sponsorship appeals accounted for nearly 82% of the caseload during that period. Toronto processed approximately 50% of the sponsorship appeals in Canada; Vancouver processed about 25%. The most prevalent types of sponsorship appeals were marriage/fiancé(e) cases (about 43% of the total) and financial cases (almost 23% of the total). In both Toronto and Vancouver, approximately 13% of sponsorship appeals in the ADR Stream are disposed of pre- ADR and about another 52% resolve through ADR - for a combined resolution rate of nearly 65%.
ADR provided an advantage in shorter sitting times - averaging less than one hour as compared to 2.4 hours (Toronto) or 3.3 hours (Vancouver) for non- ADR cases that went to hearing. However, cases that were not successful at ADR in Toronto took 3.7 hours in hearing - longer than cases that never went to ADR.
Cases that resolve at ADR are processed more quickly than those that go directly to hearing. In Toronto, ADR resolutions are achieved in an average of 3.5 months as compared to 5.8 months for cases that go to hearing without mediation, providing a 2.3 month advantage. However, cases that fail at ADR take an average of 10.6 months. In other words, they take 4.8 months longer than those cases never streamed into ADR. In Vancouver, cases that resolve at ADR do so in an average of 3.4 months and non- ADR cases are resolved through the hearing process in an average of 4.5 months, a 1.1 month advantage. Cases that fail at mediation take an average of 6.9 months. Accordingly, they take 2.4 months longer than cases never streamed into ADR.
Key Conclusions
Having considered all the information available to it, the Evaluator came to the following key conclusions. In relation to the ADR Program and process, it was concluded that:
- The IAD employs a unique ADR process.
- Participants generally have positive experiences in the mediation process.
- The contribution of DROs to the ADR Program is highly valued.
- The ADR Program enjoys varying levels of support amongst stakeholders.
- Minister's counsel are perceived to be resistant to the ADR Program.
- Appellants' counsel do not meet a sufficiently high level of competency.
- Cases that resolve at ADR produce time savings. Cases that do not resolve at ADR take longer to reach a hearing than do cases not streamed into ADR.
- The hearings associated with failed ADR cases take longer, on average, than other hearings in Toronto.
- ADR offers cost advantages to Appellants in certain circumstances.
- The ADR Program is a cost effective alternative to hearings.
- The resolution rate of cases within the ADR Program compares favourably with other similar Programs.
The Evaluator also concluded that the IADADR Program would benefit from a number of programmatic changes to address the following concerns:
- Cases are not exclusively randomly selected for mediation.
- Parties should be able to confer prior to mediation.
- Mediation should be used for cases that will likely benefit from it.
- The ADR Program is not operating at capacity.
- There is concern that the public interest is not well served in some individual cases.
- Failure to disclose pertinent information in advance of the mediation hampers resolution possibilities.
- ADR has the potential to improve an Appellant's access to justice but it does not always do so.
Summary of Recommendations
After identifying and discussing critical issues relating to the IAD's ADR Program, the Evaluator offers recommendations for consideration. The recommendations have been grouped into categories reflecting their connection to the efficiency, effectiveness, and quality of the ADR Program. The Evaluator recognizes that there is overlap between and amongst these categories.
Efficiency
- Reduce the time to hearing for failed ADR cases.
- Failed ADR cases should be heard no longer than two months later than those cases not streamed into ADR.
- Consider guidelines for postponements in mediation.
- To address the significant postponement rate, postponement guidelines should be considered and postponements should be monitored.
- Optimize the use of resources for ADR.
- The IAD and CIC should optimize the use of financial and human resources within the ADR Program.
- Improve integration and monitoring of ADR within case management.
- ADR needs to be identified as a distinct process within case management and ADR cases should be better tracked and monitored.
Effectiveness
- Review case types appropriate for ADR.
- Toronto and Vancouver should continue to review the case types most likely to benefit from ADR.
- Determine and communicate selection criteria for cases.
- In keeping with transparency, selection criteria should be consistent, clear, and communicated to the parties.
- Assign tribunal members to the DRO role.
- Given the evaluative nature of the current ADR Program, tribunal members are most likely to be and be perceived to be effective DROs.
- Develop a defined opting-out mechanism.
- The Program should provide for opting-out in certain circumstances on mutual consent with the approval of a designated IAD official.
- Refine and communicate opting-in mechanism.
- The IAD should communicate the ability to opt-in on mutual consent for case types outside the program and on the application of either party for case types included.
- Consider replacing "mediation" with another label.
- Because the IAD's unique process lacks many of the features of a classic mediation, another name should be considered.
- Continue to hold joint DRO monthly meetings.
- The meetings should address key practice issues including opening statements, confidentiality, caucusing, and providing evaluations.
Quality
- Develop selection criteria for DROs and continue to support their role.
- Specific selection criteria should be used for the purpose of assigning tribunal members to ADR.
- Enhance ADR training for DROs and Minister's counsel.
- The training should include an increased emphasis on IAD-specific issues and relevant practice issues.
- Support Appellants' counsel through training.
- The IAD should provide training materials to counsel and support ADR training for them.
- CIC to develop strategies and policies.
- CIC should develop litigation and mediation strategies and prepare policies to guide Minister's counsel in the exercise of their discretion at ADR.
- Continue consultation with stakeholders.
- The ADR Advisory Committees should be expanded and include additional representatives with substantive and practical experience in ADR; they should meet quarterly.
- Deliver consistent opening statements.
- DROs should consistently address, through opening statements, matters including the roles of the participants, confidentiality, caucusing, and potential outcomes.
- Produce a document regarding confidentiality.
- The IAD should draft a document outlining confidentiality parameters to be provided to Appellants prior to the mediation and addressed in the DRO's opening statement.
- Address confidentiality of documents produced at mediation.
- If a case does not resolve at mediation, DRO's should ask whether the Appellant wants to have the documents produced for mediation remain on the docket.
- Explain the confidentiality rules pertaining to caucus.
- DROs should explain how they intend to deal with confidential information obtained in caucus.
- Address the issue of notes taken at mediation.
- DROs should explain that participants may take and retain notes for their individual purposes; the IAD should develop a policy to address issues such as access to and retention of notes.
- Provide an explanation for not recommending consent.
- Minister's counsel should return from caucuses to the plenary session and provide, as appropriate, an explanation to the Appellant if they do not consent to recommending that the appeal be allowed.
- Clarify procedures relating to Summaries of Agreement.
- The IAD should determine issues including who should draft Summaries of Agreement and the nature of information that should be contained in them.
- Hold a session for interpreters.
- Interpreters should be provided with an understanding of key ADR terms.
- Improve ADR facilities.
- The IAD should explore improving ADR facilities and provide appropriate technology.
- Modify the proposed ADR Rule.
- A number of amendments to the Rule should be considered, including a provision that the DRO who conducts the ADR session will not hear the case.
- Consider expanding ADR
- The Report offers areas for potential expansion and suggests that the implications of impending immigration legislation be considered.
Conclusion
The IAD's ADR Program contributes significantly to the efficiency, effectiveness, and quality of administrative justice at the IAD. It provides Appellants with a process that is generally viewed as fair and worthwhile and often produces savings in terms of costs and time. Its informality reduces the tension often felt by participants in adversarial proceedings. The ADR Program has not, however, reached its full potential. The Evaluator has provided a series of recommendations that call upon each of the stakeholders in the Program to make changes that would improve its efficiency, effectiveness, and quality.
Related Links