00:00:05 Introduction

Hello and welcome. Today we will cover a very broad topic within the hour that we have. We will start with the introduction regarding the role of data in Monitoring and Evaluation (M&E) and how this has evolved through the years. We will look at common challenges in data use and how we can address those challenges to build quality from the ground up. We will examine how an M&E system can lead to high-quality data collection and how we can bridge the gap between data and action. Finally, we will leave some time for a Q&A session.

00:01:02 The evolving role of data in M&E

The nature of our work in the development and humanitarian sector is emergent. It cannot be explained by the behavior of individual components but rather emerges from the interaction between them. It is nonlinear, meaning small changes in individual components may result in disproportionate effects on the global state of the system. It is also adaptive; different stakeholders and situations change behavior to adapt to changes in other components. We are called to organize and define the information we want to use to transform these complex systems into simpler systems we can use efficiently.

Data analysis allows professionals to identify patterns, uncover trends, and highlight gaps in project implementation. For example, in a health project, data analysis can reveal geographic areas with low vaccination uptake. It is also important for transparency. Stakeholders, donors, partners, and beneficiaries demand accountability. Data analysis plays a pivotal role in fostering trust by ensuring project outcomes are measurable and verifiable. By visualizing key metrics through dashboards and reports, organizations can communicate progress effectively.

One of the most critical roles of data analysis is its ability to guide real-time project adjustments. Projects are not static; unexpected challenges require adaptive management. Data analysis empowers decision-making based on evidence rather than assumptions. Furthermore, data analysis increasingly includes beneficiary feedback loops and participatory monitoring, using data not just about people, but with people. Finally, with more data being collected, there is increased awareness of data governance, privacy, and ethics, as well as a push for localization to build local data analysis capacity.

00:05:33 Building quality from the ground up

The starting point lies within the M&E system. The role of M&E in a project requires a well-thought-out plan for data analysis before the project even initiates. This system should outline the timeframe, characteristics of data, methods, data structure, templates, and how to carry out analysis and interpretation. This goes hand in hand with the Information Management System (IMS); one system leads to the design of the other.

You need to consider different levels of implementation—project, country, region, and global. Policies and procedures play a crucial role, dictating data collection, use, analysis, and storage. Capacity building is crucial across all levels, as is integration with existing systems. All these components lead to real-time data collection, transformation, and analysis.

To start, you need policies at the organizational level which dictate processes at the country level. You need a team structure across all levels with different skill sets. You need an approach to organizational capacity building. Then, project design begins with M&E planning documents. The objective is that these documents are translated into the first step of the IMS: the data model. You must tailor capacity building to the target audience and turn policies into specific procedures. Finally, as implementation starts, both the M&E and IMS systems must be maintained and monitored.

00:12:16 Common challenges and how to overcome them

We operate within a system involving people, processes, and tools. Common challenges often stem from these areas. In terms of people, there is often a lack of knowledge on how to establish M&E or IMS systems, or how to translate organizational policies to the field. We may find a lack of appropriate tools or tools that do not match our needs.

This translates to a lack of knowledge among field staff regarding data collection, leading to messy or non-compliant data capture. Ultimately, when we want to take action, we find ourselves unable to use the data because we don't know how to perform the analysis, or the process wasn't documented. Field staff may lack access to data, and this chaos creates a lack of motivation to use the system. If people stop using the system, it collapses.

00:14:25 Case study: Building a system in practice

Let's look at a practical example. We are working in a country that has become a primary destination for refugees and asylum seekers. Over the past six months, there has been an unprecedented influx of displaced populations. Our organization has global headquarters, regional offices, and works in over 20 countries. We are implementing a "Cash Plus" project in collaboration with the UN, focusing on enabling beneficiaries to meet basic needs via unconditional cash transfers and using information to navigate the host country.

In practice, we have regional staff, in-country staff, managers, and partner field data collectors. We need to consider the skills required at each level. Regional staff need to support country staff and understand the context. Country staff need to understand and implement organizational policies and create country-level resources. Partner staff need to understand and implement procedures and supervise their teams.

Capacity building is the area we can affect the most. We need a "Training of Trainers" approach. We train the leads who are responsible for the M&E and IMS systems. These leads then train the field staff and partners on topics like policy, M&E components, and specific data collection tools. Training should be tailored to the audience, broken down into small parts, include hands-on exercises, and avoid jargon.

In terms of process, we have organizational policies that translate into country-specific procedures. These include steps, roles, timing of M&E, data collection procedures, and quality assurance. Clear instructions are key to localization, allowing partners to update their internal policies.

00:20:49 Data quality assurance

Data quality focuses on validity, reliability, integrity, precision, and timeliness. Validity ensures data represents the intended result. Reliability ensures clear processes and definitions are applied consistently. Integrity ensures mechanisms are in place to reduce manipulation. We also need clean data and data that is sufficient and timely for decision-making.

The M&E system becomes the basis for the design of the IMS. Under the M&E system, we identify data needs, analysis, reports, and roles. The M&E plan is the core document. It acts as the basis for the IMS, where we identify data collection forms, unique identifiers, relationships, and reports. The data model is the key bridge between these two systems.

00:23:12 M&E plan and data model

An M&E plan starts with the results framework and objective statements. It details indicators, targets, baselines, methods, sampling, data collection tools, responsibility, frequency, and analysis types. This plan is the basis for the data model.

The first step of the data model is identifying user groups (who collects data) and defining their roles and access rights. Then, we identify the datasets based on the indicators (e.g., beneficiary registration, vulnerability assessments, Post-Distribution Monitoring). We connect the user groups to the specific data collection forms. Finally, we visualize the relationships between forms (e.g., connecting a PDM to a beneficiary registration via an ID) and define the required reports.

A data model is important because it identifies who needs access and why, how data is organized for storage and retrieval, how to integrate with other sources, and how to scale up. It provides a blueprint and a common language for stakeholders.

00:29:43 ICT4D tools and data quality assurance

When selecting tools, we need reliable data collection, data privacy, adaptability to changing contexts, and data access across all levels. Access is crucial for creating incentives; field staff need to see the data they collect. We also need reliable analysis and timeliness.

To ensure data quality in practice, we must avoid data redundancy. We need processes for data review and system monitoring. The tool should support validation, skip logic, and calculations. For example, if a beneficiary is registered as over 18, the system should verify this against their date of birth. We need to be able to review information and check for duplicates.

Using a tool like ActivityInfo as an example, we can enforce best practices. We can set up forms that automatically calculate age groups from dates of birth, reducing data entry errors. We can enforce specific formats for phone numbers. We can set up review processes where specific roles validate the information. The system should also allow for adaptability—enabling offline and online collection, and allowing forms to be modified or copied for new projects.

00:37:50 Bridging the gap between data and action

The key word here is learning. We learn to adjust strategies based on real-time evidence (adaptive management). We learn to scale up what works and avoid repeating mistakes. We build institutional memory through organizational learning. We engage local actors in reflection, and we demonstrate accountability to stakeholders.

To achieve this, we need to define our learning questions. We use four main components:

00:45:59 Key messages

To summarize, the M&E system and the Information Management System are interconnected; one drives the design of the other. The M&E plan is the key document of the M&E system and serves as the basis for the data model in the IMS. Capacity building is key to both systems across all levels. Data quality policies and procedures dictate a well-functioning system. Finally, thinking through this information and process saves time and allows you to adjust in a fast-changing world.

00:47:01 Q&A session

How should we adapt our M&E framework during a crisis or unexpected event like a natural disaster?

The starting point should be having instructions established at the organizational level regarding priorities during a crisis. In the first days or weeks, you go "light." You cannot do long surveys. You need light monitoring, qualitative data, and observations. You might rely on desk reviews. As the situation stabilizes, you can add more information. You need an adaptive system that allows you to start with a quick registration form and later expand it without losing the initial data.

How does evaluation enter here? Should we go beyond the logical framework?

Project evaluation often focuses on the upper levels of the results framework, checking how interventions contribute to strategic goals. You need a document expressing the evaluation modality and questions. However, evaluation can also examine processes not in the M&E plan, such as team structure efficiency. This requires a dedicated evaluation document, though it often reuses monitoring data.

Can you share how this process is modified in an emergency context when partner capacity is affected?

Gradually, you build organizational learning around emergencies. You use qualitative and secondary data primarily. If a partner lacks capacity, the organization must take a leading role and step in to create capacity and training as soon as things stabilize.

What is the difference between an M&E Plan and an M&E Strategy?

A strategy is the direction or priority objective you take (e.g., focusing on learning in a pilot project). The M&E plan is the specific document detailing how you will achieve that objective, including indicators, definitions, and calculations.

What is the difference between an M&E Framework and an M&E System?

The terms are often used interchangeably. Generally, the "Framework" or "System" is the umbrella that includes all policies, procedures, and documents. The M&E Plan is a specific component or subset of that system.

What does real-time monitoring mean exactly?

Traditional monitoring often relies on monthly or quarterly reports. Real-time monitoring means having a report with preset information that updates live as data is entered. It allows you to see the status of indicators at any given moment without waiting for a reporting period to end.

Which software is best for M&E systems?

This depends entirely on your needs. If you need pure survey collection, one tool might be best. If you need powerful inferential statistics, you need statistical software. If you need case management and analysis, a tool like ActivityInfo fits. There is no single "best" tool; it depends on what you need to achieve.