By Carrie Hasselback, Deputy Director, Monitoring, Evaluation, and Learning
Whole-of-project evaluation (WOPE) is a term that we are starting to hear more and more but is often misunderstood. Is it just a trendy new term for a performance evaluation? Or is it something different that we haven’t quite figured out yet?
As the Monitoring and Evaluation Support for Collaborating, Learning, and Adapting (MESCLA) team in Honduras gears up to start a WOPE of USAID/Honduras’ citizen security development objective, Chief of Party Pat McLaughlin sat down with us to discuss WOPEs and how to approach them.
Carrie Hasselback: What is a whole-of-project performance evaluation? How does this differ from an activity-level performance evaluation?
Pat McLaughlin: The “whole-of-project” concept might be confusing if you aren’t familiar with USAID’s lexicon. Many people think this is just a fancier new way of referring to an activity (i.e. an intervention generally carried out through a contract or grant). However, for USAID, a project is actually a set of activities that are intended to achieve a higher result, often aligned with an Intermediate Result or even a development objective in the CDCS results framework.
In fact, in Honduras the whole of project evaluation is more related to the CLA strategy across a development objective. The WOPE will support USAID/Honduras’ CLA approach by helping identify assumptions or gaps in the project and inform course correction and future project design. It should also contribute to the wider evidence base on effective development practices.
CH: Why are WOPEs important?
PM: Because a WOPE evaluates an entire set of activities that support a development objective rather than a single activity, it can show how progress toward the objective is greater than the sum of its parts. Even if a single project in a portfolio isn’t performing well, a WOPE might show that the project together with other activities with which it has synergies is in fact achieving progress (or not). This can help improve the coordination between projects and activities across a portfolio and provide valuable lessons on how to develop comprehensive portfolios that effectively achieve development goals.
CH: Within USAID, how common are WOPEs?
PM: Based on new requirements that went into effect in 2017, each USAID mission must conduct at least one whole-of-project performance evaluation within their CDCS timeframe (which is generally five years). So the concept is a little newer and there isn’t a lot of experience in the field, but we will definitely start seeing it more and more.
CH: What are the key considerations when deciding to proceed with a WOPE? What does a WOPE look to achieve?
PM: Ultimately USAID will make the best decision for proceeding with a WOPE based on their learning and accountability needs. But there are definitely other factors to consider. For example, it’s helpful to have an updated evidence base, activities with clear synergies or interdependent implementing mechanisms, and teams implementing activities that regularly reflect on progress and use that knowledge to adapt. MEL platform contracts, like MESCLA, can help USAID assess projects’ suitability for a WOPE and guide the decision process.
CH: MESCLA, our MEL platform in Honduras, is preparing to conduct a WOPE. Can you walk us through the steps for getting this in place and performing a WOPE?
PM: The Honduras team wanted to do something a little bit different by focusing on the effectiveness of the CLA strategy across the mission’s development objectives. After securing buy-in from USAID/Washington, USAID/Honduras approached the MESCLA team, which facilitated sessions to refine initial ideas about learning questions and methodology, and then drafted the scope of work for USAID/Honduras to review. The timing of information collection was discussed to ensure time for participation from the Development Objective technical team at USAID as well as implementing partners and government stakeholders, and to ensure that the findings would inform the preparation of the upcoming CDCS.
CH: What are some of the challenges associated with WOPEs?
PM: Timing is consistently a challenge. The guidance is that all evaluations “should be timed so that their findings can inform decisions such as, but not limited to, course corrections, exercising option years, designing a follow-on project, or creating a country or sector strategic plan.” And I think that anyone who has any interest in evaluations would agree with this and wants to see the findings utilized. However, with a WOPE, you may have several activities being implemented by several partners and all are at various stages in their life cycle. This might mean that one may not have had a chance to even start producing results whereas another might be nearing the end. It’s tricky to time it so that you are really capturing the effect of synergies between activities.
CH: Any departing advice you have for those looking to complete a WOPE?
PM: There is no one-size-fits-all approach to conducting a WOPE. Each one will require careful management and resources. Some may require intense data collection while others might require none at all. The WOPE should answer questions that aren’t being answered by other means (through activity monitoring or activity-level evaluations). And always, always the WOPE should ensure findings are used to inform programming and policy decisions.
The views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government.