00:00:00 Introduction

Thank you so much, Faye, and thanks for all your work in organizing this webinar, but also for all the documentation and tutorials that you write for ActivityInfo. It is really a pleasure.

Today we are going to talk about the Agile methodology and M&E data collection systems. It is something I wanted to talk about because it actually comes from the field of software development, which is where I started my career even before I got involved in international assistance. I think it has a lot that applies to M&E, and I guess you think so too because we had almost 1,400 people sign up for this webinar. I hope I can distill it down into a good hour's worth of information.

We are going to talk about what Agile is and where it came from, then look at how we can apply the principles to monitoring and evaluation. We will take a quick look at a few Agile methodologies and processes that are in use, and finally, we will look at how ActivityInfo, the software that we work on, can help you make your M&E more agile.

00:01:22 What is agile?

Let's start by talking about what agile means and where it comes from. The word "agile" with a lowercase "a" simply means to be able to move quickly and easily, without pain or problems. The word "Agile" with a capital "A" dates back to 2001, about 20 years ago. There was a conference of leading software developers who got together because they were frustrated with the traditional process of building software—how slow, costly, and risk-prone it was. They came together to articulate what they thought would be a better way of developing software, and they called it the Agile way, bringing forth the Agile Manifesto.

This Agile Manifesto included four values that they articulated. They spoke about valuing individuals and interactions over processes and tools; valuing working software as a goal rather than having reams of paperwork or comprehensive documentation; valuing collaboration with stakeholders rather than negotiation; and responding to change rather than following a plan in lockstep.

If we look at these values, it is not to say that the items underneath—like processes, tools, or following a plan—aren't important or useful. However, they argued that it was more important, for example, to respond to change than to blindly follow a plan, or to value individuals more than processes and tools. These four values led them to articulate 12 principles. After this manifesto came out, it seems to have struck a nerve because over the last 20 years, people have found ways to apply these principles to fields ranging from leadership to human resources to project management. Certainly, with this webinar, we are setting out to do the same for monitoring and evaluation.

00:03:25 Why agile?

You might ask, why Agile? How is it different or better than other approaches? The Agile approach is most commonly compared with a Waterfall model. If we think about this in terms of M&E, you might have a stereotypical waterfall process with linear steps. You start with a logical framework, which everyone discusses, and that takes a while. Once you have the logical framework, you move on to designing an M&E system, creating a nice document for that. Then you go off and build the system, either internally or by hiring an outside consultant, which can take weeks or months. Once it is ready, you train your staff, start data collection, and then finally, you can start providing results to project managers, donors, and other stakeholders.

Some of the problems with this approach are that you don't really see results until the end of the system. If there are any mistakes or things omitted in the early steps—for example, if something is left out of the logical framework, or an assumption made during system design turns out not to be true, or you realize at the end that managers need different information—it can be very costly to go back and redo these steps. All the while, the system might not be working or providing what is needed.

How does the Agile process differ from this? To answer that, I want to look at the principles and values of Agile by focusing on three different themes that resonate with monitoring and evaluation: the idea of providing value to external stakeholders, responding to change, and the importance of organizing teams.

00:05:35 Providing value to stakeholders

Probably the biggest idea behind Agile is aligning an organization to provide value to external stakeholders. In the original Agile Manifesto, the value is working software. For developers, the whole point is to build software that works; that is how they provide value to customers. A 200-page requirement document doesn't help the customer at all.

I encourage you to think about how this applies to M&E. What is the value that we are providing to our stakeholders—donors, colleagues, project managers, nutritionists, and beneficiaries? Donor reporting is the most basic value; it helps your organization stay in business and continue to serve beneficiaries. If you can help your team make a donor report on time that is accurate and trustworthy, the organization will value your work.

Hopefully, M&E doesn't stop there. You might think about management—providing project status reports to project managers. Being able to show a manager that things are on track for certain activities but behind in a specific province helps them take early action. Impact analysis is another value: testing assumptions to see if outcomes are leading to the desired impact. This helps specialists adapt the program. Finally, validating or invalidating assumptions in your theory of change is a key role for the M&E team.

If we look at the Agile principles adapted for M&E, our highest priority ought to be to satisfy stakeholders through early and continuous delivery of valuable information. We shouldn't wait three years until the end of the project to bring forth a huge report. By that time, we have missed chances to provide value or adjust the course of the project. We should deliver this value frequently—every couple of weeks or months if possible. We should judge our progress by having a working system that produces this information.

Concretely, this means setting smaller goals but executing more frequently. You might start with a log frame that has 200 indicators, but maybe you can start by implementing a monthly project report with just three key indicators. If the project is struggling to get an M&E system in place, start small, pick the essential information, and communicate that quickly. One great idea from a user was doing proactive donor updates every two months with just a few slides of information. This provides feedback early, allowing you to adjust so you aren't surprised at the end of the project.

Another aspect of providing value relates to data quality. A vicious circle often arises where you don't have good data, so you wait to report until the quality improves. However, if you aren't providing value to stakeholders, they start ignoring M&E, which leads to even worse data quality. You can jumpstart a "virtuous circle" by finding the Minimum Viable Product (MVP)—the smallest thing you can do to provide value.

For example, when I was working in Eastern Congo supporting the non-food item cluster, it was tough to get information from NGOs. We had an incomplete map of interventions. We decided to just put the map we had up on the screen at monthly meetings, acknowledging it was incomplete. People immediately noticed gaps in areas like Penekusu and realized they needed to send their data to be represented. This sparked a change; everyone started sending data because the map provided value for targeting activities, and they didn't want to be left out. That jumpstarted a cycle where data quality improved.

00:16:25 Responding to change

The next theme is about change. In the private sector, customers always want the latest technology. In M&E, we are used to a rapid pace of change, whether in the humanitarian sphere where political contexts shift, or in development where we must find new approaches. Change complicates the Waterfall model.

There are two kinds of change we deal with. First is changes to our environment. For example, you might start working with returnees, but then conflict reignites, leading to new displacement and cholera outbreaks. If you take six months to build an M&E system, it won't work because you need to respond to changing objectives quickly. Even in development, you learn so much during a three-year project that the data you need will change.

The second type of change relates to uncertainty. In the beginning of a project, there are many things nobody knows. This is often referred to as the "Cone of Uncertainty." At the start, it is difficult to make a specific plan regarding data collection. As you move forward, work with beneficiaries, and run pilots, you weed out uncertainties. The Waterfall model views change as a threat—often formalized as "change requests," a bureaucratic process that discourages adaptation.

The Agile approach views the ability to respond to change as a strength. Instead of one big outcome at the end, you break the project into smaller pieces and smaller outcomes. This isn't just about milestones; it is about pieces that people can start to use. If you have to change things based on what you learn, it isn't as expensive because you don't have to go back and redo a massive system.

Applying the principles to M&E, we want to welcome changing requirements. If a stakeholder asks for health center utilization rates seasonally, instead of panicking because it wasn't in the original plan, we should say, "Great, we learned something about how to provide value." In the next iteration, we include that information. Delivering useful information frequently allows for more feedback and lowers risk.

00:21:25 The importance of teams

The third theme is the importance of organizing teams. One way to think about this is the difference between a bureaucratic team and an Agile team. In a bureaucratic team, a manager gives instructions to individuals who report back on progress. The database person builds the database, the survey person organizes the survey, but they aren't necessarily talking to each other.

An Agile team is autonomous and cross-functional. You put the M&E person, database specialist, nutritionist, and project manager together, and they talk to each other constantly. Instead of a 20-page requirements document, the database person talks daily to the nutritionist to ensure the workflow makes sense.

Many Agile principles focus on teamwork. Project managers, specialists, and M&E staff must work together daily. You want to build projects around motivated individuals, giving them the environment and trust they need. The most efficient method of conveying information is face-to-face conversation. Agile processes also promote sustainable work; rather than working like crazy for the last month before a donor report, you maintain a sustainable pace by continuously putting out improvements. Finally, the team should reflect regularly on how to become more effective and adjust behavior accordingly.

00:25:05 Agile methodologies

There are dozens of methodologies that have arisen to meet these values. Two you might have heard of are Scrum and Kanban. Kanban is a continuous process, often used for continuous software release. Scrum focuses on blocks of time called "sprints" where you focus on a set of goals.

For M&E, Scrum is a process that can really work. The process starts with everyone coming together to decide what is needed in the next month or two. You agree on goals—like a survey or donor report—and the cross-functional team plans those goals into a set of tasks (the sprint backlog). You have a fixed period where you focus only on those tasks, checking in every 24 hours. At the end of the sprint, you deliver a piece of value, such as a dashboard or analysis.

00:27:25 How ActivityInfo helps

Faye and I work for BeDataDriven, and our day job is making the ActivityInfo software. This software was actually inspired by the frustrations I faced working in M&E, specifically how long it took to build systems. ActivityInfo is a relational database designed for M&E with integrated mobile data collection. It is meant to be easy to use so you can iterate quickly without needing a developer or consultant.

You can determine what you want to collect easily. If you need a new indicator, you can add it quickly, and everyone using the mobile or web interface sees it instantly. You can link forms, visualize data with maps, do analysis, and share dashboards with stakeholders. It allows you to get set up quickly and respond to change. We also have a template library for various use cases and a self-paced training course to empower your team.

00:30:30 Q&A session

Sarah asks: Under the umbrella of Agile, there are two terms which I find confusing: incremental and iterative. Can you explain the difference?

I think those terms are mostly synonymous. However, a distinction might be that "incremental" could refer to milestones in a Waterfall project (like finishing a document), which are steps forward but don't provide usable value yet. "Iterative" implies that at each stage, you are putting something useful out into the world that people can use.

Precious Moyo asks: How do we set smaller goals and execute more frequently when most projects are donor-centric?

If a donor requires a report on 125 indicators at the end of the year, you can still break it down. You might decide to focus on 25 indicators in the first quarter. Some NGOs proactively go to the donor and say, "I know we report annually, but can we share what we have so far?" This allows you to break the work into smaller increments and get feedback early.

Al-Nafi asks: How often do we need to check on our Theory of Change?

As often as possible. Every day might not be feasible, but the Theory of Change is meant to be constantly reviewed. It would be amazing if you came up with a perfect theory at the start. Checking every quarter to look at the data and assumptions would be very helpful.

Ali asks: What if project managers are comfortable with just reporting numbers and not viable data that can be used to improve the project?

It is unfortunate if stakeholders only want the minimum for the annual report. However, this is part of the vicious/virtuous cycle. If you take the opportunity to provide some small, valuable information projects, you can demonstrate what M&E can do and stimulate demand. Stakeholders will start to pay attention.

Stephen asks: What are some cons to the Agile methodology?

Not all projects are amenable to Agile. If you are building a bridge, Agile is not a great idea because we know how to build bridges, and it is expensive to change. In civil engineering, you can plan step-by-step because there isn't much uncertainty. If a project has no uncertainty or cannot be broken down into smaller units of work (like a massive national survey that can't be split), Agile might not fit. However, in many M&E cases, there is much more uncertainty than in bridge building.

Pauline asks: Is it necessary to analyze the risks when responding to uncertainties?

Yes, it is very important to look at the Cone of Uncertainty and identify what you know and don't know. For example, if you need to collect data from a national health system that hasn't been built yet, you should identify that as a risk and avoid building that part of the system until you know more.

Karen asks: Is the Agile model a replication or amalgamation of learnings from several small pilot projects?

I think that is true. One way to think about Agile is as a series of pilots—starting small, staying small, and iterating. You might eventually figure everything out and be ready to scale up, which is a great outcome.

Moses asks: Is there an opportunity for more on the Scrum Master role?

We will share a link to a guide by Atlassian on the Scrum framework, which covers roles and ceremonies. It is a useful starting point.

Ali Enoch asks: How can we encourage such a model to be followed? Shouldn't this come from donors?

It would be great if donors mandated it, but we can play a role in jumpstarting it. Instead of waiting for the perfect environment, focus on a small aspect. Spend a month getting good quality data for just three indicators and share it. You will likely be surprised that people start to pay attention because you are providing value.

Collins asks: Is Agile a real-time data collection and reporting software?

Agile is a set of principles and values. It argues for faster, more regular feedback (weekly or monthly). ActivityInfo is a piece of software that helps you put that into place.

The Llama asks: Distinguish between the bureaucratic team and the Agile team—which is best?

I have a clear preference for Agile teams. I enjoy working where everyone has a seat at the table and communicates. I would argue Agile teams produce better value, but that is my experience.

Hemen asks: Are Waterfall and Agile mutually exclusive? Wouldn't it be better to blend these approaches?

There is truth to that. We are talking about principles, which are general ideas applied to specific situations. Even in a top-down process, you can infuse Agile principles like teamwork and constant interaction. You can apply the principles to any process.

Simon asks: Following an Agile approach, how would project budgeting change? In a bridge project, costs are identified and a provisional sum is set for uncertainty. How do you price uncertainty and budget accordingly in Agile?

This is a fantastic question. In project management, you have three variables: scope, budget, and time. You have to fix at least one or two. In Agile, because you often cannot nail down the scope (since you don't know the right activities or data needs yet), you instead fix the budget and time, but leave the scope flexible.

You might say, "We have a fixed budget and a fixed timeline for this iteration (e.g., two months), and we will revisit the scope at regular intervals." You fit the scope to the budget for each iteration, deciding what is most important to do right now. This is common in humanitarian rapid response programs where the budget is fixed, but activities are determined based on immediate needs. In development, you don't rip up the plan every quarter, but you leave room to adjust indicators and focus within a fixed budget.