> For the complete documentation index, see [llms.txt](https://urd.gitbook.io/compass/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://urd.gitbook.io/compass/evaluating/evaluating-a-projects-quality-and-accountability.md).
# Evaluating a project's quality and accountability
How do you use the Core Humanitarian Standard 2024 to evaluate a project?
This section outlines how to use the Core Humanitarian Standard (CHS) commitments alongside the OECD-DAC evaluation criteria to prioritise analysis areas and develop relevant evaluation questions and information needs.
It is intended for those commissioning evaluations to aid in designing the process and for evaluators to guide their preparation and execution.
***
### **Why using the Core Humanitarian Standard to evaluate a project?**
Evaluation supports quality, learning, and accountability by assessing what works and what doesn’t in complex, changing contexts. It enhances performance and fosters accountability to communities, staff, partners, and donors.
Evaluation differs from other steering processes as it is *the systematic and objective examination of humanitarian action, to determine the worth or significance of an activity, policy or programme, intended to draw lessons to improve policy and practice and enhance accountability (ALNAP, 2016)*.
#### **Evaluation criteria**
The OECD/DAC framework, 2019, is the primary evaluation tool in the humanitarian and development sectors. Initially created for development, it is based on six criteria, which ALNAP has expanded in its guidance on Evaluating Humanitarian Action (EHA) 2025.
**Similarities and differences in ALNAP's EHA criteria**
Relevance (OECD & ALNAP)
**Similar** definition, further clarified for EHA to assess the suitability of humanitarian action concerning the needs and priorities of crisis-affected individuals, with reduced emphasis on policy alignment (addressed under coherence).
Coverage and Inclusion (ALNAP)
**Specific to EHA** to evaluate the extent to which humanitarian action is needs-based and therefore impartial (in turn relating to the widely accepted humanitarian principle of impartiality), and provided without discrimination.
Effectiveness (OECD & ALNAP)
**Same** definition but further unpacked for EHA to emphasise outcomes.
Efficiency (OECD & ALNAP)
**Similar.** Looking at three dimensions of efficiency: economic efficiency, operational efficiency and timeliness.
Sustainability (OECD & ALNAP)
The OECD criterion assesses the likelihood that the net benefits of an intervention will persist over time. However, this approach is not always suitable for humanitarian action, which is often short-term in nature. Instead, the concept of interconnection examines how short-term humanitarian efforts consider medium- and long-term outcomes. It also looks at how these efforts connect with development and peacebuilding initiatives, taking a broader temporal perspective into account.
Coherence (OECD & ALNAP)
**Similar.** The definitions emphasise the importance of consistency and complementary actions in humanitarian efforts, focusing on avoiding duplication and ensuring added value. The European Humanitarian Aid (EHA) definition specifically addresses the coordination of humanitarian actions to achieve these goals, while the OECD refers to this concept as "external coherence."\
\
Both definitions highlight the need for alignment with international norms, standards, and broader policy frameworks. The EHA also considers how to manage the tensions that may arise between different policies and standards in practice.\
\
The OECD's concept of "internal coherence" involves creating synergies and connections between interventions within the same institution, placing significant importance on the humanitarian-development-peacebuilding (HDP) nexus. In contrast, the EHA views the HDP nexus as a matter of interconnection, promoting an external and temporal perspective on the relationships among humanitarian, development, and peacebuilding actors.
Impact (OECD & ALNAP)
**Similar.** Looking at the higher-level and transformative effects in different domains.
ALNAP's revised guidance on humanitarian evaluation in 2025 adds three priority themes to the OECD criteria, which offer another lens for assessing humanitarian action:
1. Putting affected individuals at the centre of engagement.
2. Locally led humanitarian action (localisation).
3. The environment and climate crisis.
The OECD criteria are widely recognised and facilitate meta-evaluation while highlighting weaknesses in humanitarian action. However, they have limitations:
* Thoughtful and contextualised application is often overlooked, and its relevance depends on the assessment's purpose.
* The term "criteria" can be interpreted variably, which can be both a strength and a weakness. Moreover, these criteria do not address power imbalances, as what matters to evaluators may differ from what is important to the affected communities.
* Top-down applications typically prioritise performance over participation and experience. Alternative frameworks may better focus on these aspects.
While the OECD criteria are prevalent, evaluating humanitarian actions should also consider other relevant frameworks, as they can provide valuable insights for shaping evaluation questions and processes.
#### **Using the Core Humanitarian Standard for evaluation**
The CHS is a widely recognised standard that fosters equitable relationships between communities and their supporters. It aims to address power imbalances and enhance accountability to people affected by crises. Evaluating with the CHS ensures a focus on both results and the methods used to achieve them.
The CHS can be beneficial because the commissioning agency has committed to it, offering several advantages:
* It promotes internal acceptance of evaluations by providing a shared language focused on the perspectives of people and communities.
* If the agency's MEAL framework is aligned with the CHS or uses COMPASS, project monitoring data can directly inform the evaluation process.
* A consistent evaluation framework enables cross-analysis, supports meta-analysis, and fosters organisational learning and improvement.
* It can also aid in CHS-related verification processes.
The use of the CHS for evaluation adds value by offering clear, people-centred commitments that simplify the development of evaluation questions and enhance 360° accountability. It highlights often-overlooked topics in DAC evaluations, such as participation, safe complaints, staff conduct, and the role of learning in adaptation, while also integrating critical areas like environment, gender, inclusion, and PSEAH. Additionally, the CHS emphasises the responsible use of information and technology, including data protection and the implications of AI tools, ensuring that evaluations consider risks, benefits, and relevance \[3].
Examples of usage of CHS in Evaluations
* Some organisations (World Vision International, Humanity & Inclusion) have developed their own evaluation framework aligned with the CHS and mixing criteria from the CHS and the OECD.
* The Disaster Emergency Committee (DEC) has used the CHS Standard as its main reference for real-time learning exercises.
* Reported use of the CHS Standard in evaluation and real-time learning
* Groupe URD's evaluations
***
### How to link the Core Humanitarian Standard commitments with the OECD-DAC criteria
Each OECD-DAC criterion is incorporated and reflected into one or more of the nine CHS commitmentsquality criteria, making its use for evaluation in line with best practices in the sector. The following table underlines linkages between the nine CHS quality criteria and the seven OECD-DAC criteria; this could help evaluators choose focus areas and design the evaluation. Many evaluations will combine OECD-DAC criteria with selected CHS commitments based on intended use and available resources.
#### Legend
{% columns %}
{% column %}
{% endcolumn %}
{% column %}
Perfect correlation
{% endcolumn %}
{% endcolumns %}
{% columns %}
{% column %}
{% endcolumn %}
{% column %}
Close correlation
{% endcolumn %}
{% endcolumns %}
{% columns %}
{% column %}
{% endcolumn %}
{% column %}
No correlation
{% endcolumn %}
{% endcolumns %}
***
### How can you use the CHS to evaluate a project?
The evaluation cycle includes four main steps:
{% tabs %}
{% tab title="Preliminary Phase" %}
#### Preliminary Phase
This stage guides the decision whether to proceed with an evaluation. It mainly involves a quick analysis of the needs, risks, and opportunities related to a potential evaluation to make a "well-founded" and rational decision.
The CHS can be utilised at this stage to quickly evaluate the situation and help determine if an evaluation is suitable and feasible.
**Important:** The evaluation should, whenever possible, involve affected individuals in its design, planning, execution, and utilisation. If an evaluation lacks well-formulated questions (from a people-centred viewpoint) or does not include diverse groups and the perspectives of affected individuals in the analysis of results, it risks reinforcing biases and limiting opportunities for learning and accountability. In cases where there is little or no participation from affected communities, evaluators should acknowledge this at the outset and clearly state it in the report as part of transparency about the potential limitations and biases of the evaluation.
{% endtab %}
{% tab title="Planning and preparing the evaluation" %}
#### Planning and preparing the evaluation
This phase results in the writing of the evaluation terms of reference and the designation of the team that will be responsible for conducting the evaluation. It focuses on the ways and means of the evaluation mission to be conducted (who, what, why, how).
The nine CHS quality criteria can be utilised to:
* Provide a quick overview and help prioritise criteria for detailed analysis during the evaluation.
* Guide the evaluation's objectives and stakeholder engagement, clarifying roles, areas of interest, and approaches for gathering information.
* Help refine evaluation questions to encompass both outcomes and accountability, providing examples of related information to collect.
* Offers key questions that aid in defining data collection protocols, including recommended methods, engagement strategies, and measures for confidentiality, informed consent, and data management.
**Important:** During this phase, evaluation questions are identified and evaluation criteria confirmed. Avoid starting with all the criteria and then selecting questions for each one, because of resource limitations, but also because an evaluation should focus on users’ needs\[1], prioritise a smaller number of criteria/commitments
***
\[1] ALNAP, Humanitarian evaluation guide, p. 115
{% endtab %}
{% tab title="Conducting the evaluation " %}
#### Conducting the evaluation
This phase concerns the implementation of the evaluation mission and results in the writing of the evaluation report. It consists of two interacting components: the collection and analysis of information on the one hand, and the synthesis and presentation of findings on the other.
The nine CHS quality criteria can be utilised to:
* Help develop data collection tools (e.g. complement/confirm questionnaires, interviews, etc.) that reflect people-centred commitments
* Encourage the actors to specifically focus on relationships between actors, both internal and external to the NGO
* Structure the analysis and the presentation of findings that reflect both results and experiences, with an attention to power dynamics and 360° accountability
* Assess data quality and responsibility, including the use of digital tools and AI-supported processes and their implications
* Assess (if applicable) the implications for the environment
{% endtab %}
{% tab title="Using the evaluation " %}
#### Using the evaluation
This phase concerns accountability and learning, the overall objective of the evaluation process. It consists of presenting the results, and possibly publishing, distributing and disseminating them (internally and/or externally). It is focused on planning how lessons can be taken into account and how their implementation can be monitored.
The nine CHS quality criteria can be utilised to:
* Structure the evaluation in a way that links recommendations to specific commitments and roles.
* Share findings in accessible formats (including to affected communities) explaining what could change and what cannot change, and why.
* Facilitate comparison across evaluations within an organisation or across partners, improving meta-analysis and organisational learning.
* Enhances transparency by putting results into perspective against recognised quality and accountability criteria from the CHS.
{% endtab %}
{% endtabs %}
{% hint style="info" icon="fire" %}
Why are the first stages of the evaluation process the most important for the quality of the evaluation (source: ALNAP (2016) Evaluation of Humanitarian Action Guide)
As outlined in the graphic below by ALNAP, the most important stages of the process for the overall quality of the evaluation are the first steps that concern the framing of the evaluation.
{% endhint %}
***
### **Checklists**
You can consult the tools developed with key evaluative questions by CHS commitment online by clicking the links in the right menu, or you can download the PDF file below, available in English, French, Spanish and an accessible version.
{% file src="/files/edzG2SwFiNxoOF9g2nQh" %}
{% file src="/files/EF1W4igjM6ISiOyL1gpa" %}
{% file src="/files/DSGcda8Uhrkhq0S9x7oY" %}
***
\[1] OECD-DAC – The Organisation for Economic Cooperation and Development’s Development Assistance Committee
\[2] For more information on adapting the OECD-DAC criteria check: Adapting the OECD Criteria for the Evaluation of Humanitarian Action, ALNAP, 2025 -
\[3] AI in the Humanitarian Sector – Essential briefings for humanitarian decision makers, ALNAP, 19 May 2025. -
