Introduction
The IDLO Health Law Program (2009 – 2012) includes three objectives to build country capacity to respond to HIV:
1. Legal empowerment: Increase community legal awareness, and strengthen and expand HIV-related legal services.
2. Law enforcement: Sensitize police, judges and other government officials so that they apply laws in a manner consistent with public health goals.
3. Law reform: Build lawmakers’ capacity for reform based on evidence, international law and best practices.
As of December 2010, pilot projects to strengthen HIV-related legal services had been initiated in Benin, Burkina Faso, China, Egypt, Guatemala, Indonesia, Nepal, and Papua New Guinea. Country projects have up to two years to complete the pilot phase. During the pilot phase IDLO provides technical support that enables the partner organization to set up a monitoring and evaluation system for the project, to conduct the mid-term review and to conduct the final evaluation.
The HIV-related legal services project in Papua New Guinea is financially supported by Australian Agency for International Development (AusAID). The two-year project aims to build the capacity of legal professionals, strengthen community groups and establish legal services for PLHIV and key populations.
The mid-term review and the final review will be conducted by a team of stakeholders, as explained below. The IDLO Health Law Program MRI officer will facilitate the evaluation process and provide the final report. The role of the lead evaluator is to:
• Facilitate the evaluation process
• Provide necessary technical guidance to the evaluation team
• Ensure the evaluation is conducted in an impartial, independent manner
• Advise IDLO of any necessary actions to be taken based on the findings of the evaluation
• Provide a detailed report to IDLO as specified below
The purpose of the mid-term review is to assess progress made during the first year and make recommendations for the future planning and implementation. The review will include a strong emphasis on considerations relevant to sustaining the project in future years.
Objectives of the review
1. Identify what the project stake-holders achieved during the period to be reviewed.
2. Identify successes and gaps in achieving the project objectives.
3. Make recommendations for year 2 and beyond.
Deliverables by the evaluation team
1. Review plan submitted two weeks prior to the review exercise.
2. A report on findings and recommendations.
23
3. Short (2-5 paragraphs) narratives of key achievements which can be used in communications materials.
Methodology
The review follows a participatory approach from the outset. The review team comprises representation from all levels of stakeholders. Six team members contribute to the review process. The team follows the following methodology for the review:
1. Document review
• Project proposal/ annual plans
• Policies and procedures for legal services
• Environmental scan report
• Quarterly reports
• Training reports
• Other
2. Data collection from different groups:
• Implementers
• Beneficiaries (PLHIV/ key populations who received legal services from the legal service, trained lawyers, trained communities etc)
• Other similar service providers
• Government (eg. National HIV/AIDS Program, Ministry of Community Development, OPS)
• Other (PNG Law Society etc)
Data collection will be done through focus group discussions when and where possible.
3. Analysis of data/ findings, recommendations
4. Presentation to the stakeholders on the last day of field work 5. Writing the report
The review team initiates communication at least two weeks before the review start date.
During this period, sharing of the review objectives (to come to a common understanding about the review) and finalization of the methodology are particularly important. A common understanding of the availability of the review team members is also important.
The review team will emphasize focus group discussions (FGD) along with other
participatory approaches and tools as a data collection method (please see the attached guidelines for FGDs).
Sampling: The review team needs to collect information from the legal service staff,
beneficiaries of the project, other legal service providers in the project location, government (National HIV/AIDS Program, OPS), PLHIV networks, implementing partner, IDLO, and technical advisors. An appropriate number of people from each group, according to the size of the population, should be taken as a sample.
Focus issues:
• Has the project been effective in achieving intended results?
• What lessons have been learned?
• What steps can be taken to improve prospects for the project’s sustainability?
24
Detailed review SAMPLE questions: Detailed review questions:
1. Project design:
1.1 How was the project designed?
1.2 Should the project design be adjusted to reflect new information about needs and work to be done to achieve intended results?
1.3 Are the objectives still relevant and valid, or should they change to reflect changing contexts?
2. Project implementation and achievement of results:
2.1 Is the legal aid project appropriately established in terms of accessibility and availability?
2.2 Has the project achieved intended outputs and outcomes during the review period?
Taking into account data on the
a. Type and number of legal issues received? Types of legal services provided? Areas that need improvement? Type and nature of workshops/ events held and
effectiveness of them?
b. How government sector got involved in the project/ contribution to the enabling environment?
c. How policy makers/ parliamentarians got involved in the project/ contribution to the enabling environment?
2.3 Has the project made efficient use of available resources, and what steps could be taken to improve efficiency in future?
2.4 Has the project addressed gender factors in its work?
2.5 Is the project reaching populations most in need of HIV-related legal assistance and support?
2.6 How effective is the networking with community groups and their involvement in the project?
2.6 Has the project addressed/promoted participation by people living with HIV and most-at-risk populations in its work?
2.7 What lessons can be learnt from client demographic and satisfaction data, and feed-back from beneficiaries about the nature and quality of services provided?
3. Project management:
3.1 What is the project execution strategy? Should there be any changes to the project execution strategy?
3.2 Has the management structure and management approach of the project been effective in achieving its midterm targets?
3.3 What adjustments to internal management should be taken to improve the project’s effectiveness in achieving its objectives in Year 2?
4. Technical capacity:
4.1 Do the project staff have sufficient knowledge of HIV related legal services?
4.2 Do the project staff receive sufficient technical inputs?
4.3 What are the future needs to improve the technical capacity?
5 Monitoring:
5.1 What are the regular monitoring/ reporting measures?
5.2 Are the regular monitoring/ reporting measures workable/ practical?
25 6 Sustainability:
6.1 What is the comparative advantage of the project compared to other service providers?
6.2 What lessons have been learnt to inform steps to be taken to support the project’s sustainability including seeking contributions from domestic funding sources in future years?
Composition of the review team: field work takes five days including presentation of draft findings to local stakeholders.
• Draft review concept note available by 25th April
• Nominations for the review team and confirmation from them by 15th May
• First information to the review team by 10th June
• Relevant documents to be sent to team member by 15th June
• Draft meeting schedule to be shared with local partner and IDLO by 15th June
• Finalized meeting schedule to be sent to team members by 23rd June
• Review from 27 June to 1st July
1st day morning – Review team meet and finalize the schedule
1st day afternoon – Review team goes through reports and documents. Meeting with the project staff
2nd day – data collection from beneficiaries (FGD)
3rd day morning – data collection from beneficiaries (pro bono lawyers/ peer counselors etc) data collection from government, PLHIV networks
3rd day afternoon – data collection from other service providers 4th day morning – data collection from the local partner
4th day afternoon – Meeting with bar association. Analysis of information 5th Day – key findings/ presentation of draft findings to stakeholders
• Draft report is due by 1st August follow up actions to be implemented by the projects.
1. Introduction (1/2 page) 2. Executive summary (1 page) 3. Purpose of the review (1 page) 4. Methodology (1 page)
5. Findings (up to 6 pages)
6. Recommendations/ follow-up actions (2-3 pages) 7. Annexes as needed (2-3 pages)
26