ToR – Final Evaluation of the Improved MSc in Cultivar Development for Africa (IMCDA)

1. Background and Context

The Alliance for a Green Revolution in Africa (AGRA) is a dynamic, African-led partnership working across the African continent to help millions of smallholder farmers out of poverty. Under its new strategy (2016-2020) AGRA is seeking to catalyze an agricultural transformation in eleven (11) key focus countries. AGRA has been implementing the Improved MSc in Cultivar Development for Africa program (IMCDA) to improve smallholder farmer productivity and income by training plant breeders to better serve seed companies and public institutions by designing and operating state of the art breeding pipelines, increasing genetic gains and accelerating adoption of improved varieties in Sub Saharan Africa (SSA). This program is part of AGRA’s capacity building efforts in the agriculture sector.

AGRA’s Extension & Capacity-building program sits within the Program Development and Innovation (PDI) component.  In the past 10 years AGRA’s capacity-building programs have seen the successful training of over 1,100 post-graduate students in the areas of plant breeding and seed systems, integrated soil fertility management, and, agricultural and applied economics and policy.  AGRA has also over the years trained 260 field and  laboratory technicians, as well as over 1000 seed enterprise personnel on how to better manage their seed enterprises for greater profits.
 
 2. Program Components

The Improved MSc in Cultivar Development for Africa (IMCDA) was established through funding from the Bill & Melinda Gates Foundation (BMGF) to build scientific capacity that more effectively delivers improved varieties to farmers. The primary goal of this five-year program launched in November 2013 is to improve the rate of genetic gain in varieties adopted by farmers, through the training of a cadre of skilled professionals responsive to farmers’ needs and market demands. The specific objectives are:

  • To develop relevant web-based curricula for the training of industry-ready plant breeders for both private and public sectors in SSA, with a strong focus on cultivar-development-pipeline management and optimization;
  • To train 90 plant breeders at the MSc level from 16 sub-Saharan African countries using modern technologies;
  • To develop cultivar development programs at three partner universities that will serve as training platforms for students;
  • To establish MSc student internship programs inside established seed companies; and,
  • To teach modern technologies in plant breeding to mid-career public and private sector plant breeders.

The project’s approach was for Iowa State University’s Agronomy Department, a leader in plant breeding education with experience in developing web-based teaching material, to work with three African universities serving as regional centres: University of KwaZulu-Natal (UKZN) in South Africa, Makerere University in Uganda, and Kwame Nkrumah University of Science and Technology (KNUST) in Ghana, to develop web-based training material covering modern breeding approaches and technologies that can be used by any institution in the world. The program targeted to train 90 MSc students at each of the 3 African universities, in cohorts of 30 students each, with the first intake in August 2014.  Iowa State University (ISU) received a separate grant from BMGF for the development of the elearning modules for use by the IMCDA and for the eventual use of faculty from interested universities at the conclusion of the project. AGRA on the other hand was awarded a separate grant for the implementation of the IMCDA.

3. Objectives of the final evaluation

The main objective of this final evaluation is to assess the overall performance of the program in relation to its objectives, and to provide input into the new strategic direction of another phase or program sustainability beyond the life of the project.  The evaluation will assess what has worked, what did not work very well, key lessons learned and areas requiring emphasis or adjustment, in the event that a second phase is granted. The evaluation will essentially cover the following four components of the IMCDA program:

1. Programmatic impacts – assessment of what outputs have been delivered, what changes (outcomes, both intended and unintended) have occurred among the beneficiaries, relevance of program interventions, cost-effectiveness of investments and the sustainability of the achievements to-da

2. Program delivery mechanisms – review of the approaches adopted by the program to achieve program objectives, such as:

  • Integration of e-learning modules into conventional teaching methods, research and student internships.
  • Role of Plant Breeding E-learning for Africa (PBEA) in developing e-learning modules, how useful these have been and how well they are integrated into teaching and learning
  • Role of PBEA Professional Learning Communities in enhancement of course delivery
  • Breeding programs at each university and whether these are forward looking projects that can be effectively used for training, research and product development.
  • Mentorship by supervisors and visiting professors
  • The suitability of the advisory committee and the significance of their contribution to AGRA and project coordination teams.
  • Mid-Career plant breeder’s training – has it had an impact. What is anticipated to change in plant breeding research?

3. Project management by university coordination teams – timeliness, implementation of breeding programs, student intake, supervision and completion.

4. Program management for results – evaluate program management processes; assessing their adequacy, efficiency and effectiveness. These will include program structure; staffing; grant-making processes; quality of grants funded; level of integration and alignment with other AGRA programs and the level of engagement of AGRA staff in overseeing the implementation of the program.

4. Scope of the evaluation

The final evaluation will assess the program performance since its inception in, 2013 to date. The MSc in Cultivar Development for Africa program has been implemented by three (3) universities in Africa and two (2) universities in the US.   The key organizations critical to this project are listed below in broad categories:

a. Universities involved in the training

  1. University of KwaZulu-Natal, South Africa
  2. Makerere University, Uganda
  3. Kwame Nkrumah University of Science and Technology, Ghana
  4. Iowa State University
  5. University of California, Davis

b. Partners of these universities involved in training

  • Seed companies
  • National Agriculture Research Institutes
  • International agricultural research institutes

c. Donors

The partner universities where students were trained are the University of KwaZulu-Natal in South Africa serving southern Africa (Tanzania, Mozambique, Malawi and Zambia), Makerere University (Uganda) serving eastern Africa (Ethiopia, Kenya, Tanzania, Rwanda and Uganda) and Kwame Nkrumah University of Science and Technology (Ghana) for the West African region (Burkina Faso, Mali, Nigeria, Ghana, and Niger). The students were to be recruited from all 13 countries in which AGRA is supported by BMGF. Student recruitments will not be equivalent across all countries; rather, plant breeder gaps in each country will be addressed. More than 30% of students were recruited from AGRA’s Priority 1 countries: Ghana, Mozambique, Mali, and Tanzania.

The purpose of the funding was to cater for the following aspects:

  1. Develop a model MSc Plant Breeding program that emphasizes product development
  2. Support Iowa State University’s participation in curriculum improvement, web-based curriculum development, student and staff exchange and collaborative training of MSc students and staff teaching skills improvement through the Plant Breeding Elearning for Africa (PBEA) and PBEA Professional Learning Communities.
  3. Recruit additional university staff in order to enhance course delivery and supervision;
  4. Establish an internship program ensuring greater breadth of student exposure to the seed industry, and increased demand for trained plant breeders within those companies. The expectation would be enhanced capacity for in-house development of improved varieties by seed companies;
  5. Establish breeding programs within the universities for purposes of training and product development
  6. Train university staff in new approaches to cultivar development, breeding information management, pipeline optimization, and teaching
  7. Plant breeding research at universities will expose students to product-focused cultivar development programs, while simultaneously developing new varieties
  8. Sponsor active plant breeders to mid-career plant breeder training to expose them to modern trends in plant breeding

Key Evaluation Questions:

4.1 Quality and relevance of program design

  1. What were the major changes of program design, threats and opportunities which may have occurred since the inception of the IMCDA, and which may have had implications on program performance?
  2. Assess program risks and test the underlying hypotheses and assess how these were managed.
  • What were the program strategies and how they respond to emerging risks to the program?

4.2 Program performance

  1. Assess the extent to which program outputs, outcomes and objectives were achieved at all levels.
  2. What were the positive and negative, direct and indirect, planned and unplanned results of the program?
  • How effective was the program, sub-programs and project activities?
  1. What are the  strategic  importance  of  the  program’s  achievements  and  how  it interfaces with AGRA’s new strategy?
  2. What are the challenges, lessons learned and best practices in program, sub-program and project implementation?

4.3 Efficiency in resource utilization

  1. Conduct a cost effectiveness analysis of program results to assess the quality of results/deliverables against the cost of achieving these results in comparable contexts?
  2. What is the potential and options for future sustainability, scalability and replicability?

4.4 Management issues

Assess the overall organizational structure of MSc in Cultivar Development for Africa, including staff configuration, execution modalities, and quality of grant making and management approach, arrangements for implementing grants, capacity gaps, utilization of the expertise of the MSc in Cultivar Development for Africa team and AGRA program officers.

4.5 Lessons learned

What key lessons have been learned from MSc in Cultivar Development for Africa program?

5. Methodology

While a detailed survey design and methodology to achieve the objectives of the final evaluation shall be defined by the consultant, and approved by AGRA, the following is broadly suggested as a guide. The Evaluation methodology will be a combination of qualitative and quantitative approaches for data collection and analysis. Essentially, this will entail: Desk review  of  key  program  documents  and  program  implementation  progress  reports  from grantees, and results. Such documents shall include but not limited to the background program documents; MSc in Cultivar Development for Africa proposal), AGRA sub-award agreement with sub-recipients, grant proposals; progress reports from grantees and reports from AGRA to BMGF, and other key documents related to the program. Besides the review of relevant literature related to the assignment, the consultants shall also undertake primary data collection using structured interviews with key informants, notably; project administrative and technical personnel, program partners, AGRA program staff and management; targeted Focus Group Discussions (FGDs), as appropriate; and site visits, observations, and other qualitative data collection methods as appropriate.

To enrich the qualitative data obtained through the above qualitative methods, the Consultants shall also engage in quantitative data collecting by administering questionnaires among graduates who have been admitted to the program. This should be a representative sample from all the 3 universities supporting the program.  To enhance likelihood of achieving the evaluation objectives stated above the consultants shall work closely with the IMDCA staff and AGRA’s M&E unit to ensure effective coverage of the projects graduates and countries. The methodology for the evaluation will be determined by the evaluation team to provide robust findings and conclusions that best address the objectives listed above and rigorously answer the evaluation questions. The consultants will develop and institute quality control mechanisms to ensure robust and valid data is collected, aggregated and analyzed for reporting.

6. Deliverables

The consultant shall be expected to prepare and submit to AGRA a set of key reports in the course of undertaking this assignment. These reports shall be reviewed and accepted by AGRA before payment is approved. The following have been identified as key reports to be submitted

6.1 Inception Report/Evaluation Plan – This shall be prepared and submitted within two weeks after the signing of the contract. The consultant will prepare this after reviewing key technical documents and after discussion with the AGRA team. The inception report shall focus on: the understanding of the Terms of Reference and scope, the relevant methodology to be adopted, the evaluation design and key questions, data collection plan and methods, data analysis plan, limitations of the evaluation and the work-plan for the assignment. The inception report shall be reviewed by both the M&E and the program’s team within 5 days after submission, and shall have to be approved before proceeding to the next phase. A Revised Evaluation plan should be submitted by evaluators to AGRA 5 days after receiving comments.

6.2 Progress brief – While there is no formal progress report required during the assignment implementation, between inception and draft report submission, the consultant will be expected to regularly (bi-weekly) share with AGRA, key emerging issues and trends to avoid surprises or misconceptions by either party.

6.3 Draft Report/Presentation – This shall be prepared and submitted to AGRA towards the end of the assignment. The draft report shall require feedback in form of comments, questions and inputs from AGRA. To facilitate this the consultant shall also arrange to present the report to a wider AGRA audience for validation.

6.4 Final Report – submitted to AGRA on, or before the expiry of the assignment contract. Any valid extension may be mutually agreed between the consultant and AGRA, provided it carries no extra cost to the latter. The following will also be expected from the consultant:

  • A master copy of the final report in print and electronic form suitable for reproduction, submission of the final report, after incorporating the comments/inputs on the presented draft repor The final report shall include actionable recommendations
  • All data-sets (in SPSS/Stata) and questionnaires used during the assignment shall be a property of AGRA, and shall be the responsibility of the consultant to deliver to AGRA electronically
  1. Management and Reporting

To ensure consistency, enhance utilization and avoid any surprises, the consultant shall in the course of the evaluation, be required to provide a bi-weekly update to AGRA by email. A draft report shall be submitted 10-days before the end of the contract.  Reporting shall be directly to the assigned AGRA M&E officer with oversight of the Senior Program Officer for Extension & Capacity Building.

8. Timing

The evaluation is expected to be completed within a 6 0-day period, effective from the date the contract is signed.

8. In-house resources

Access to project documents, progress and evaluation reports, program databases, contacts for grantees, financial records and other program related files, depending on the consultant’s requirements will be availed. Access to the IMCDA team and other AGRA staff shall be arranged upon request.

10. Criteria for selection of Consultant

The consultant will be selected through an open and competitive process and will be based on their proven experience, qualifications and ability to deliver a high quality product in a timely and efficient manner. Particular qualifications and experience of the lead consultant team members include:

Team leader:

  • The lead consultant must possess a minimum of an MSc in Agricultural Sciences, Social Sciences, Economics or a related field, and with at least 10 years of relevant experience
  • At least 5 years’ experience conducting rigorous, evidence-based evaluations utilizing a mixed methods approach

Consultancy team:

  • A member of the team should possess a postgraduate degree in business management with specialization in strategic management, development studies with a bias towards sustainable development.
  • Collectively the consultancy team should have 10 years’ experience in conducting survey fieldwork (data collection, validation, entry and analysis) in sub-Saharan Africa
  • An understanding of academic research methods
  • Recent experience with results-based management evaluation methodologies
  • Experience in leading teams in field (training, field logistics, human relations, teamwork)
  • Demonstrable ability and experience in working with communities and the team’s capacity to undertake the study in Africa
  • Excellent writing skills, with publication record in one discipline related to the assignment
  • Demonstrable analytical skills
  • Excellent knowledge of English. Knowledge of French and Portuguese is an added advantage

11. Scoring

Evaluation scoring     

No. Description Max. Score
1. Combined experience of the team in conducting rigorous, evidence-based evaluations utilizing a mixed methods approach 20
2. Adequacy and quality of the proposed methodology, and work plan in responding to the Terms of Reference (TORs):
(i)                 Technical approach, methodology and work plan 10
(ii)               Organization and staffing (competence and experience) 5
3. Key Experts’ qualifications and competence for the Assignment:
(i)        Team leader’s academic qualification and general team qualifications 10
(ii)      Experience in similar survey fieldwork and analytical skills (data collection, validation entry and analysis) 10
(iii)     Excellent report writing (structure, flow and coherence) 10
(iv)     Excellent knowledge of English and a good command of French and Portuguese within the team 5
Sub-total technical score 70
Financial proposal 30
TOTAL 100

 
12. Budget

The Consultant will submit a budget breakdown. The budget presented should include fees, travel and subsistence, reproduction charges, and unit costs should be calculated as a per day tariff (e.g. fees).

Travel will be required to the three regional centres and selected collaborating institutions within Africa.

13. Proposal Submission

Interested and qualified consultants are invited to submit their proposal (s) comprising of the following:

  • A description of the evaluation plan including details of the proposed methodology, sampling, study design; analysis and reporting, and milestones for the evaluation and a timetable of activities.
  • Methodology and work-plan for performing the task
  • Team composition and tasks assignment
  • Detailed reference list indicating the scope and magnitude of similar assignments
  • Relevant services undertaken in the past three (3) years
  • Detailed budget
  • Description of the pay schedule for the review
  • Past performance summaries (at least three brief descriptions of past or current contracting mechanisms for assignments similar in size, scope and complexity to this tender) and list of references that demonstrate performance in conducting similar evaluations
  • 2-3 previous evaluation reports and list of previous reports
  • CVs conforming to the qualifications listed above for persons to manage and conduct the evaluation. Only CVs strictly relevant to the assignment shall be accepted.

Technical and Financial proposals will need to be submitted as separate documents and clearly marked as such. Financial proposals MUST be password protected. Financial proposals will not be opened until the conclusion of the technical evaluation and only for those proposals that are deemed qualified and responsive to the TOR’s.

14. Clarifications

Questions and/or clarifications with the subject line reading “Request for Clarification – IMCDA Final Evaluation” may be submitted to procurement@agra.org by April 23, 2019.

15. Submission

All interested consultants or consultancy companies are asked to submit their proposal by close of business on 30 April, 2019 at 17:00 Hours East African Time (GMT +3) to the following email address: procurement@agra.org.

For more information on this request, applicants can visit www.agra.org

AGRA
Menu