Consultant, DRIVE Project Baseline Evaluation at Catholic Relief Services (CRS)

Catholic Relief Services (CRS) is an International non-governmental organization supporting relief and development work in over 100 countries around the world. CRS programs assist persons based on need, regardless of creed, ethnicity or nationality. CRS works through local church and non-church partners to implement its programs, therefore, strengthening and building the capacity of these partner organizations is fundamental to programs in every country in which CRS operates. CRS re-established presence in Nigeria in 2000 and currently focuses on vulnerable children, agriculture, health and HIV, extractives and governance, and peace building programming, Health Systems and Institutional Strengthening Programs.

We are recruiting to fill the position below:

Job Title: Consultant, DRIVE Project Baseline Evaluation

Location: Jere LGA - Maiduguri, Borno

Background

• The DRIVE project is a strategic initiative led by Catholic Relief Services (CRS), aiming to enhance resilience and promote sustainable recovery among “vulnerable but productive” populations in Maiduguri Municipal Council (MMC) and Jere Local Government Area (LGA) in Borno State, Nigeria. Active in Nigeria for over two decades, CRS has established a robust presence in the Northeast since 2014, implementing critical interventions to support crisis-affected communities. With funding from the Bureau for Humanitarian Assistance (BHA), CRS has been instrumental in delivering humanitarian aid and facilitating early recovery efforts in the region.
• Currently, CRS is implementing two major programs in Borno State: the THRIVE program in Jere LGA and the Improved ALERTS program in Gubio, Kaga, and Magumeri LGAs. Through a collaborative approach that emphasizes partnerships with community leaders, Area Based Committees (ABC), and local government authorities, CRS has achieved significant community acceptance and engagement in Jere, enabling the effective delivery of high-quality, responsive programs.
• The proposed DRIVE project is designed to build upon the insights gained from previous CRS initiatives and the successes achieved through over nine years of humanitarian and early recovery programming in Northeast Nigeria. It aligns closely with the Borno State development plan and integrates valuable lessons learned from past projects. It integrates valuable lessons learned from past projects such as THRIVE and aligns with the Borno State development plan. DRIVE will provide context-specific early recovery interventions with the goal of sustaining early recovery in three wards in Jere LGA (Mashamari, Dusuman, and Mairi) and two wards in MMC (Maisandari and Dala).
• DRIVE will coordinate with the government to operationalize targeted, integrated early recovery interventions for “vulnerable but productive” households. These are households who have working-age members but who are in poverty due to multiple factors which limit their access to income opportunities. These households lack the necessary skills and resources for gainful engagement in income generation. Following BHA guidance, the project will prioritize households already receiving social protection assistance in targeted locations, including those enrolled in the government-led social safety nets program overseen by the Borno State Operations Coordinating Unit, under the supervision of the National Social Safety-Nets Coordination Office (NASSCO). Specifically, it will prioritize households classified as "vulnerable but productive." In the initial stages, CRS will collaborate with NASSCO to devise targeting strategies. These households are primed for early recovery interventions. By layering and integrating proposed interventions, their coping capacity will be enhanced, leading to self-reliance. Over the two-year project, it is estimated that approximately 132,000 unique participants will be reached across three sectors.
• The overarching goal of DRIVE is to empower households and communities, fostering sustainable progress in livelihoods and recovery in MMC and Jere. The project's Theory of Change posits that “If households improve their livelihoods, enhance their ability to manage shocks, and engage with markets; and if they reduce their vulnerabilities to hazards and increase their preparedness; and if they receive support to reduce trauma, foster trust, and build and manage relationships for socially cohesive communities, then conflict-affected households will be more self-reliant and live in dignified environments”
• The project will deliver an integrated, multi-sectoral, and coordinated package of life-sustaining and life-building early recovery interventions to “vulnerable but productive” households in MMC/Jere LGAs through three sectors. The project will target at least 22,000 households, including settled internally displaced people (IDP) and host community members.

Based on this available information, the project intends to carry out a multi-sectoral baseline study to measure the project indicators before starting activities.

Purpose of the Evaluation
Purpose of the Baseline:

• The baseline evaluation aims to establish initial metrics that will serve as benchmarks for assessing the contributions of the DRIVE Project in supporting “vulnerable but productive“ households through the package of life-building and early recovery interventions in MMC/JERE LGAs.
• Specifically, this evaluation will focus on evaluating the initial status of the pervasive food insecurity and limited income access opportunities, and the capacity of the target population to manage shocks and engage with markets,
• The baseline will provide essential data to measure progress and impact throughout the project's implementation.

CRS and Donor Evaluation Requirements
CRS Requirements:

• As per CRS MEAL Policies and Procedures1, Terms of Reference must be developed for all evaluation events (MPP 3.2).

BHA requirements:

• BHA requires2 that all awards that are 18 months or longer in duration, must conduct a baseline study (within 90 calendar days of the award start date) and a mixed-method final evaluation (Exceptions to this requirement can be requested for responses immediately following a sudden-onset disaster (e.g. hurricane, earthquake, tsunami, flood, cyclone, etc.). The baseline may be conducted either by an internal team led by an experienced team leader or a third-party firm/independent Consultant. The team leader must be external to the organization.
• Final evaluations that have a quantitative baseline survey must include a probability sample at the participant or population level in the activity area that is consistent with the baseline survey and be designed to produce values that will be compared to results from a baseline survey so that change can be measured and tested for difference. Final evaluations seek to answer a breadth of questions, which go beyond only measuring the final indicator values at the end of an activity. Partners are encouraged to ensure that data collection for baseline and endline surveys occur at the same point in time/season (during the year), to the greatest extent possible.
  • https://crsorg.sharepoint.com/sites/Monitoring-Evaluation-Accountability-and-
  • Learning/SitePages/Policy3_Evaluation.aspx
  • https://www.usaid.gov/sites/default/files/documents/BHA_Emergency_ME_Guidance_February_2022.pdf.

Evaluation objectives

• The baseline evaluation will establish initial measurements for the indicators outlined in the DRIVE Project’s Indicator tracking table (ITT).
• Additionally, it will gather comprehensive data on household characteristics, providing valuable insights into the demographics and socio-economic conditions of the targeted populations.
• Furthermore, the baseline will facilitate the assessment of resilience-related indicators tracked internally by CRS through its Global Results framework.

Specific objectives
The baseline has five specific objectives which are:

• Establish preliminary values for the key outcome indicators associated with the DRIVE Project. This includes setting or validating target levels for indicators related to disaster risk reduction (DRR), livelihoods, social cohesion, and resilience, providing a foundation for future assessment and tracking.
• Collect detailed demographic and socio-economic information on households within the targeted populations in MMC and Jere LGAs. This will encompass data on household composition, socio-economic status, livelihoods, vulnerabilities to shocks, and access to essential resources, to inform program interventions across sectors such as agriculture, financial services, and vocational training.
• Assess the current understanding, attitudes, and practices regarding disaster risk reduction and resilience within the community. This includes examining household preparedness, the adoption of DRR measures, and identifying barriers that prevent the implementation of effective resilience-building strategies.
• Measure the resilience of households and their ability to recover from shocks. This involves identifying existing coping mechanisms, understanding how households have responded to past crises, and evaluating their capacity to manage future risks and uncertainties.
• Assess the current economic activities, income sources, and employment opportunities in the target communities, including agricultural and non-agricultural livelihoods. This includes identifying barriers to market access, financial services, and employment, as well as opportunities for improving household income and livelihood recovery.

Table 2: List of Indicators that requires a Baseline Value:

The consultant is required to refer to the Indicator Performance Tracking Table (IPTT) to identify the appropriate disaggregation for each of the indicators specified in the table.

Evaluation Methodology
Evaluation Design:

• The baseline evaluation will adopt a mixed-methods approach, integrating both quantitative and qualitative data collection techniques to ensure a comprehensive understanding of the project context.
• The survey will employ a one-stage sampling methodology using simple random sampling

The methodology will consist of the following components:
Beneficiary Household Surveys:

• Three beneficiary household surveys will be conducted to gather data on key indicators at the household level.
• These surveys will utilize standardized data collection instruments to ensure consistency between the baseline and final evaluations.
• The household surveys will enable the detection of statistically significant changes in key indicators over the course of the project.
• For the Economic indicator (E02), information such as household income from livelihoods, will be established based on data collected with all potential project participants during registration, making it a census.
• Two other beneficiary household surveys will be carried out:
  • Data required to compute baseline values of the indicators A2, A10, and A15 will be collected on a sample of selected beneficiaries for the project’s agricultural activities. The sampling frame will be comprised of selected only project participants with an agricultural profile (access to farm/land, having farmed or made backyard activity lastly, etc.)
  • For indicator D05, the perception of the likelihood of being severely affected by a specific hazard will be assessed over a survey with sampled selected households after registration irrespective of the sector they are supported with.

Key Informant Interviews (KIIs):

• KIIs will be conducted with a range of community stakeholders, including local leaders, government representatives, and other key actors.
• These semi-structured interviews will provide qualitative insights that complement the quantitative data, enriching the understanding of community dynamics, challenges, and opportunities for enhancing resilience.

Focus Group Discussions (FGDs):

• FGDs will be held with diverse community members (e.g., men, women, youth, and vulnerable groups) to explore collective perceptions, local practices, and social dynamics related to DRR, resilience, and livelihoods.
• These discussions will identify social barriers, enabling factors, and community-driven solutions to enhance disaster preparedness and response strategies.

Sampling Strategy for the Multi-Sectoral Baseline Surveys
Overall Sampling Approach:

• The baseline evaluation will adopt two distinct sampling approaches based on sectoral needs and data availability:

ERMS, DRRP, and Agriculture (participant-based): ERMS (E02):

• The sampling frame will be all the households qualified as "vulnerable but productive" coming from two sources: i) the participants (households) list of the National Social Safety-Nets Coordination Office (NASSCO), ii) - if the target is not reached - list of potential participants coming from the general population in the targeted communities pre-identified by the Area Based Committees following the project’s eligibility tool.
• During registration, all will be questioned about their current livelihood (if any) and net income. No sample for Economic Recovery and Market Systems (ERMS) sector will be taken.
• Agriculture (A2, A10, and A15) and DRRP (D05): Stratified Sampling will be used to identify respectively selected participants considered for the Agricultural project’s services only, and selected participants in all sectors combined. To allow the samples to reflect their actual representation in the population, each project’s implementation localities (eleven) will constitute a stratum. The households from each sub-group will be chosen by simple random sampling and the size of the sub-groups in the overall sample will be proportional to the entire target population. This method is appropriate since it is the RedRose application that will be used to register the selected participants making a list of uniquely identifiable beneficiaries or households available on the one hand, and there will be a very reasonable logistical effort of data collection associated with this technique on the other hand since all the implementation localities will be nested in Jere and MMC.

Sample Size Calculation:

• The sample size will be determined ensuring adequate power for statistical analysis and detection of significant changes over time.
• Following the BHA guidance, we will use the following formula ensuring statistical robustness to detect changes in livelihood activities, agricultural productivity, and economic resilience by comparing the values of indicators collected at two points (now at baseline level and after the project is completed):
• ninitial = Dest*Z1-∝2P(1-P)+Z1-βP1, est1-P1, est+P2, est(1-P2, est)δ2
• Dest = 1
• δ = P1, est- P2, est=minimum effect size to be achieved over the LOA
• P1, est represents the estimate of the true population P1 at baseline = 0.5
• P2, est represents the estimate of the true population P2 at endline = 0.6
• P= P1, est+ P2, est2
• Z1-∝ is the one tailed critical value from the inverse normal distribution corresponding to a confidence level 1-α ; With α=0.05 and Z0.95=1.64.
• Z1-β is the one tailed critical value from the inverse normal distribution corresponding to a power level of 1-β ; With 1-β=0.8 , Z0.8=0.84.

With these parameters, the theoretical sample size. is 305 households. By application of a 90% Probability of getting at least one measurement unit per Household, an adjusted sample size for finite population and factoring a 20% non-response rate, the final sample size for each sector is 423 ~ 425 households.

Key Informant Interviews (KIIs)

• In addition to the household surveys, at least eleven KIIs will be conducted in total with the following stakeholders, considering their respective roles in the DRIVE project’s strategy and implementation.
• Additional key informants may be included based on specific community contexts and emerging relevance.

Focus Group Discussions (FGDs)

• FGDs will be held with key community members, ensuring representation from different demographic groups (e.g., men, women, youth).
• These discussions will capture in-depth insights into local knowledge, practices, and attitudes towards disaster preparedness, resilience-building, and livelihoods, which will complement the findings from household surveys and KIIs.

Data Collection Methodology

• Household survey questionnaires will be designed to gather data on the indicators pertinent to the various DRIVE sectors. Trained enumerators will be employed to collect quantitative information.
• The surveys will be conducted using electronic devices (tablets or smartphones) equipped with the CommCare application.
• Enumerators will receive training on the study methodology, the survey instruments, and the procedures for obtaining informed consent.
• The ICT4D team at CRS will provide specific training on the use of these electronic devices to ensure consistency and minimize input bias. Once data collection is complete, the information will be exported through API/Odata format and prepared for analysis.
• The DRIVE MEAL team will supply the consultant with relevant datasets and methodological details used during participant registration.
• This information will assist in generating baseline values for additional indicators outlined in the Indicator Tracking Table (ITT).

Data Analysis

• The data analysis will encompass both the primary data collected through household surveys and the secondary data provided by the DRIVE MEAL team. Quantitative analysis will be performed using statistical software such as SPSS or STATA, focusing primarily on descriptive statistics. The consultant will examine subgroup data to identify patterns, consistencies, and discrepancies among different demographics, such as gender and age groups.
• Results will be disaggregated by location, gendered household types, and any other attributes specified in the Performance Indicator Reference Sheets (PIRS). The consultant will present the findings in various formats, including tables, graphs, and charts, alongside a comprehensive analysis syntax that can be independently applied to the dataset for verification or further analysis.
• In addition to quantitative analysis, a qualitative data analysis approach will be employed to analyze data obtained from key informant interviews and open-ended survey questions. Software such as MaxQDA, NVivo or Atlas.ti will be used to facilitate thematic analysis, allowing for the identification of recurring themes, concepts, and insights that emerge from the qualitative data. The qualitative findings will be integrated with the quantitative results to provide a comprehensive understanding of the context and implications of the baseline assessment.
• Furthermore, the consultant will be responsible for developing a Power BI report that showcases a visual snapshot of the key results from the baseline assessment, enhancing the communication of the findings to stakeholders.

Evaluation Team Composition

• The evaluation will be conducted by at least two evaluators: a Lead Evaluator and one or more Co-evaluators. The Lead Evaluator will oversee the entire evaluation process in collaboration with CRS staff, ensuring that all steps are executed effectively and according to the evaluation’s goals. The MEAL Coordinator/Manager will be responsible for the overall coordination of the evaluation, ensuring adherence to quality standards and compliance with CRS and USAID evaluation guidelines. The evaluation team will work closely with the enumerators, selected and trained by CRS, to ensure accurate baseline data collection.
• Clear roles and responsibilities for each team member will be outlined in a separate section below to facilitate effective collaboration among all stakeholders involved in the evaluation.

Roles and Responsibilities
Evaluation Consultant’s Responsibilities:
The evaluation consultants will be responsible for the following:

• Review project documents, records and existing dataset and prepare an inception report that details the evaluation matrix and methodology, comprehensive and costed plan for the evaluation tasks;
• Work with the DRIVE Program and MEAL team to design the baseline evaluation survey instruments, including the household questionnaire, a plan for the supervision and application of quality control checks during the data collection;
• Design a detailed sampling plan including sampling methodology;
• Collaborate with the DRIVE team to develop the protocols and instructions along with a training and instruction manual for the study enumerators;
• Lead the training of data collection teams in all components of the assessment and sampling methodology (i.e., data collection, questionnaire design, sampling, data analysis and tabulation);
• Design of a database (or databases) of the household questionnaire data (must be submitted in a non-proprietary format);
• Develop a draft report highlighting the assessment objectives, methodologies, data collection techniques, main findings.
• Design a detailed final report of the evaluation that incorporates all comments raised from the program staff, stakeholders, CRS regional technical advisors as well as USAID.

CRS Staff/DRIVE Team:

• Develop and share the methodological details and datasets from the Participant registration exercise with the consultant
• Prepare and share all relevant secondary data required for desk review including planning documents, MEAL plan and ITT.
• Review and approve the inception report within 5 days after the inception meeting.
• Review and provide feedback on data collection tools.
• Provide feedback and recommendations on draft evaluation report
• Provide the needed number of enumerators for the evaluation
• Co-facilitate/co-manage the enumerators’ training
• Facilitate community entry at intervention communities
• Process the payment of the evaluator after submission and approval of each deliverable agreed
• Regularly update evaluator on security context and associated risks for completing the evaluation.

Evaluation Team Deliverables

• All reports and other deliverables must be completed in the language that was originally agreed, be free of typos or grammatical errors, and be ready for publication. This means the document contains no factual errors or inaccuracies and citations are properly used.
• Inception report (including evaluation matrix and methodology)
• Final copies of all quantitative data collection tools;
• Completed copies of survey questionnaires, consent forms, and qualitative notes;
• Detailed syntax of the analysis steps with clear reference to the dataset versions on which the syntax can be verified
• Draft and final reports for the evaluation following the outline and pages limits defined in the Reporting and Dissemination Plan section of this TOR;
• Soft copies of presentations and the Power BI dashboard developed and delivered during dissemination events

Reporting and Dissemination Plan
Evaluation report:

• The full report should not exceed 25 pages, excluding title page, list of acronyms, table of contents and annexes. The report should be structured in accordance with the following guidelines:

Cover Page:
List of Acronyms:

• Table of Contents which identifies page numbers for the major content areas of the report.
• Executive Summary (1 page) should provide an accurate reflection of the most critical elements of the report, including the evaluation purpose and questions, background information, methods, limitations, findings, recommendations, and conclusions. It should only include information found elsewhere in the evaluation report.
• Tabular summary of results presenting baseline findings in table form against the program indicators included in the IPTT.
• Introduction: Describe the project’s scope, locations, timing of baseline data collection. This should include the objectives of the study
• Methodology: sampling method including strengths and weaknesses of method used, confidence intervals, inclusion/role of stakeholders and staff, rough schedule of activities, description of any statistical analysis undertaken, including justification and software package used. The discussion of any random sampling used should include details on how the random respondents were identified and invited to participate. This section should also address constraints and limitations of the evaluation process and rigor. The methodology section should also include a detailed description of data collection techniques used throughout the evaluation.
• Detailed Findings: A presentation of the findings in response to baseline questions, and the program objectives, as appropriate. Baseline values must be presented in quantitative format as required by the IPTT’s disaggregation, complemented by descriptive analysis.
• Synthesis and Programmatic Implications: Additional analysis of the data and results, drawing out programmatic and organizational recommendations, lessons learned, and best practices for planning or modifying program design. Everything presented in this section must be directly linked back to the information presented in the Results section of the report. Recommendations must be relevant to the program and context and must include concrete and realistic steps for implementing or applying the recommendation. Recommendations that are not directly linked to Results can be included in an Evaluator Comments section of the report.
• Report Annexes:
  • Updated Indicator Performance Tracking Table (IPTT)
  • List of stakeholder group consulted with number, type and date of interactions - Data collection instruments in English, including developed and used
  • Photos
  • TOR
  • Other special documentation identified as necessary or useful.

Dissemination Plan – Key Audiences and Uses

• All evaluation findings will be communicated to key stakeholders.
• The table below includes a summary of how findings will be communicated to different stakeholders.
• In addition, as per the MPP 3.6 CRS will post the baseline report to Gateway within 90 days of the end of data collection activities.

Evaluation Calendar

Ethical Considerations
Evaluators will obtain written or verbal informed consent from all adult respondents prior to data collection. The consent process will include clear communication of the following:

• Purpose and Objectives of the Evaluation: Respondents will be informed about the evaluation's goals and how their participation contributes to achieving these objectives.
• Voluntary Participation: It will be emphasized that participation is voluntary, and respondents may withdraw at any time without penalty or consequence.
• Data Collection Details: Respondents will be informed about the specific data being collected, including the types of questions asked.
• Use of Data: Respondents will be assured that their data will only be used for the stated evaluation purposes.
• Data Retention: Respondents will be notified of the duration for which their data will be stored.
• Data Sharing: Respondents will be informed about who will have access to their data, including any third parties involved in the evaluation process.
• Confidentiality Assurance: Measures to ensure the confidentiality of the respondents' information will be clearly communicated, including the protection of personally identifiable information (PII).
• PII will be stored securely on encrypted servers, and it will not be included in the final report or shared through insecure channels. All data will be managed in compliance with CRS’s Responsible Data Values and Principles. The consultant will ensure that no data collected for this evaluation is used for any purpose other than the objectives specified in this ToR. Data collected will be transferred to the CRS/DRIVE project team, and the consultant will not retain or use the data beyond the stated evaluation scope.

Lines of Communication

• The MEAL Coordinator will serve as the main point of contact for the consultancy.
• The consultant(s) will work closely with the DRIVE Project Director, DRIVE Project & MEAL Managers/Officers.

Budget:

• The total budget to submit should include consultant’s fee, lodging (accommodation), DSA, transportation and enumerators’ allowances and all other associated field cost related to this assignment.
• All applicable taxes will be deducted at source in accordance with extant financial laws and regulations in Nigeria. CRS will not bear any financial liability in excess of the above amount submitted.

Required Qualifications
Lead Evaluator:

• Post graduate qualification in Development studies, Monitoring and Evaluation, Statistics, Sociology or International Development, PHD will be an added advantage.
• Extensive knowledge of evaluating multisectoral projects in emergencies and good understanding of the emergence response in the Northeast.
• At least 5 years’ experience in conducting evaluations in complex environments, experience evaluating USAID/BHA funded projects. Experience working in Nigeria Northeast states, preferably BAY states will be an added asset.
• Extensive experience in evaluating multi-sectoral programs implemented through multiple partners.
• Excellent oral and written communication skills in English and fluency in Hausa and Kanuri.
• Extensive knowledge of qualitative and quantitative data collection methods and analytical methods and techniques.
• Experience in generating strategic, informative, and action-oriented recommendations to CRS project staff and strategic management.

Application Closing Date
25th December, 2024; 5pm (West Central Africa).

How to Apply
Interested and qualified candidates should send their full applications to: [email protected] using “CRSNGBHA-20012025-01-Application DRIVE Baseline Evaluation” as the subject of the mail.

Note

• To Apply, applicants must write and submit a brief (maximum 3 pages) approach paper outlining how they understand the TOR and proposed methodology. In addition, applicants should submit a professional profile highlighting key areas of expertise and experience and how they match the required skills specified in these terms of reference. We also request links to at least three recent examples of relevant work, and the contact details for three references.
• In addition to the above, applications must include an Indicative budget in line with the expected deliverables and should show all related costs including travel costs and anticipated overhead costs.