00:00:00 Introduction

So now I would like to introduce you to our panelists: Ms. Falkenburg and Ms. Owuor. Naomi Falkenburg is a monitoring, evaluation, accountability, and learning specialist with a cross-cutting focus on inclusion. She works at DevLearn, a firm specialized in consulting and training for inclusive economic growth. Naomi has worked with development and humanitarian actors around the world to design, manage and learn from their interventions, and conduct insightful research and analyses. Her experience spans a range of sectors and themes, such as disability inclusion, education, employment, entrepreneurship, gender equality, migration, trade promotion, and women's and youth empowerment. Naomi is certified in project management for development and holds an MSc from the University of Oxford’s Department of International Development.

Mercy Owuor is a public health and development specialist with expertise in project monitoring and evaluation. She is interested in participatory M&E and social inclusion. She is currently based in Uganda with an organization called Strategic Initiative for Women in the Horn of Africa Network. Faith, who will be facilitating this session, is the marketing director of ActivityInfo. So, without further delay, welcome everyone, and the virtual stage is yours.

00:01:40 What is participatory M&E and why is it important?

Thank you so much, Jane, for the introduction, and thank you very much, Naomi and Mercy, for joining this session and for wanting to share your insights on this very important topic. Let's jump right into this interesting topic. Let's start with the basics: what is participatory monitoring and evaluation, and why should we care?

I'll take the 'what' part of that question, which may sound simple, but it's a very important one. We hear the word 'participation' being bandied around quite a lot in our sector in general, and within M&E as well, without it being super clear exactly what we mean by this. For me, it really all boils down to power. Participatory monitoring and evaluation is a way of approaching M&E that, firstly, recognizes that power relations between people tend to be unequal, and secondly, it actually tries to proactively change the balance of power. In the context of M&E, those who usually have the power are the researchers or the practitioners. The reason that they have that power is because traditionally we think of research, monitoring and evaluation as completely objective activities, and we tend to think of the products of those activities—so knowledge, data and evidence—as being completely neutral. This just means that the people that are researched are not usually involved because they're not the experts, they're not academically trained.

Participatory Monitoring and Evaluation is not one specific methodology or even a set of particular methods. It's actually a set of principles and practices for undertaking research, monitoring and evaluation that actually engages those stakeholders who are usually excluded from the process. Participation can take a lot of different forms depending on who you talk to. Basically, the form that participation takes is really linked to the aims of our M&E process itself, which you can think of as ranging from being functional to being transformative. When our M&E is functional, it means that we use it to find out something new, produce some new knowledge, maybe assess an intervention, but all just according to the questions and goals that are set by practitioners. Whereas towards the more transformative end of our aims, we use M&E and those findings to really affect action or change as defined and driven by the intended beneficiaries of our intervention.

We can really think of participation as being on this continuum. On one end, we are maybe informing or consulting with the community. This is quite a tokenistic exercise; it's participation in name only because the practitioner basically just holds all that power. Whereas on the other hand, you see more meaningful collaboration between practitioners and the communities, including shared decision-making, or even the empowerment of the community to actually lead that M&E process itself and to act on its results. This continuum is a very helpful way to think critically about power and what participatory monitoring and evaluation actually means.

Regarding why we should care, there are a few different types of arguments. One type of argument is the instrumental one. You can argue that participatory monitoring and evaluation is a way to improve our own M&E and, by extension, our own interventions. There are benefits to having participation in all stages of the M&E cycle, which primarily come from this combination of the theoretical and methodological expertise of practitioners with the real-world knowledge and experiences of the affected communities. Bringing those two things together can really improve our processes, tools, data, and findings. For example, if you involve people in the planning phase when setting up indicators, you'll probably end up with more relevant and meaningful indicators because you ensure that they really relate to the lives of the participants. Involving people from that very beginning also increases the motivation to actually participate in M&E processes later on and can give much more realistic expectations about the outcomes.

The other very important case is one that's based on equity, human rights, and ethical questions. Conventional approaches can often feel very extractive to the people who are being researched. They don't have a say in things that are very much relevant to them and their context. The approach that participatory M&E takes is one that says you don't need to be academically trained or an expert in order to contribute to knowledge production, and the people who are closest to an intervention probably know the most about it; they're experts on their own experience and on their own lives.

Participatory M&E is about elevating community voices. If you want to achieve success, you want to involve and hear from the community so that as they plan for the project, they are part and parcel of it. Their voices have been taken into account. It also addresses the aspect of inclusivity. With participatory M&E, you're guaranteed that you'll be able to hear from the different groups. For example, in household surveys, if the head of the household is the respondent, you are likely to miss the voice of the women in the house where economic issues are involved. It has the capability of bringing inclusivity into the design, the monitoring and evaluation, and also the knowledge generated from whatever project is being undertaken.

00:14:05 How does it look in practice?

How does participatory M&E look in practice? Could you tell us a little about methodology or give us some examples of tools we could use?

When you talk about methodologies, there are so many aspects to this. Participatory methodology started off in the '60s or '70s, especially with participatory appraisal and learning approaches. For example, engaging a group of people in an interview process is in itself participatory, but it is a process, not just getting in, getting the information, and getting out. Things like storytelling—asking community members to give you an account or tell you a story about their experience with the project—is participatory.

There are also examples like community or social mapping approaches. For instance, if you are doing a project on community sanitation and want to understand issues around water sources or latrines, taking a transect walk to map out the services and facilities jointly with the community is a participatory method. They understand why it is important and how we need to do this. Other methods include diagramming or PhotoVoice. You might ask community members to draw out or map out safe spaces regarding sexual and gender-based violence. Working together with the community to build that map on safe spaces is participatory, rather than just handing them a list. PhotoVoice involves asking community members to document events, perhaps through photos if they are not comfortable writing, sharing a small shot of an activity to describe what is going on.

The thing that makes a method or tool participatory is the process. There is really no one right way to do participatory M&E, just as there is no particular set of methods and tools that is right for every situation. You might combine several methods and tools over the course of a project. There are freely available toolkits and guidelines out there that give hundreds of examples.

My advice would be to consider a set of questions when you set out to choose the methods and tools. First, what are the ultimate aims of the M&E? If you have a more transformative aim, you will choose tools that put power in the hands of research participants. Second, what is the purpose of the specific M&E activity? Different methods will be appropriate depending on whether you are designing methodology, collecting data, or analyzing data. Third, who are the research participants and what are their needs and preferences? People communicate in different ways; visual, written, oral, or tangible modes of expression might be better depending on the context. Fourth, what resources do I actually have? Do I have the time, money, human resources, and support materials? Finally, what is the potential of a particular method or tool to do harm? You need to weigh the risks against the benefits.

Additionally, the methods and tools are only as useful as the facilitator—the skills and the attitude of the person facilitating. Just because a tool is designed for participation doesn't mean you can't use it cynically. Some of these methods require skill. For example, when doing Outcome Harvesting, it's not just about organizing a group and getting data. There are processes involved, including validation of that information. This requires capacity building for both the facilitator and the participants.

00:25:11 Challenges in adopting a participatory approach

What would you say was the biggest challenge or some of the biggest challenges you faced when trying to adopt a participatory approach to monitoring and evaluation?

I would say that the big one for me quite often is buy-in. As practitioners, we're very often dependent on other stakeholders with decision-making power, like donors, senior managers, or clients. You really need to be able to get these people on board in order not only to do participatory M&E, but to actually do it well. They are the gatekeepers to resources. One of the challenges in getting this buy-in is skepticism about the methodology. Participatory M&E embraces flexibility and diversity, which contrasts with more traditional, structured quantitative research that people often view as producing more valid findings.

The other challenge is that you are often up against resistance both structurally and personally. Participatory M&E is about questioning power relations, and it can be extremely uncomfortable to reflect on our own role in the power dynamic. These approaches require us to look critically at power and redistribute it, valuing local knowledge and capacities. That is a difficult shift to make for many people as individuals and as institutions.

In addition to that, the main challenge I have experienced is having it as part of the plan from the word go, so that it's not an afterthought. If it is a project you're working on with a donor, participatory methodology should be an approach selected from the start. We need to be clear about the methods and processes, and account for resources. Otherwise, you end up doing it for the sake of it, realizing you don't have enough time, or getting data you don't know what to do with. It needs to be part of the initial plan, with funds and capacity accounted for.

00:31:08 Addressing challenges and improving buy-in

Are there other possible solutions to this challenge? What about facilitation skills, advocacy, and communication?

What is required is trying to reflect on the value addition that this is going to bring. What value did it add to the usual survey? We need to think through that in the initial processes so that it builds as part of the evaluation methodology. We must plan with the community members because some methods require them to have necessary skills. For example, if you are planning an advocacy event, how do you track that? Advocacy is not easy to track, but you need to keep tabs on the processes. If it's about community members building their capacity to advocate for their rights, they need the right skills to "sniff" the changes happening so they can celebrate progress. We need to give them the right tools and resources so they are part and parcel of tracking the changes.

Regarding the challenge of buy-in, if we want to use participatory M&E, we need to advocate for it and persuade others that it is useful and legitimate. As M&E practitioners, we are actually in a position of power; we get to decide what counts as progress and impact. To solve the problem of buy-in, we need to educate ourselves and be able to make a clear case for it—whether instrumental, ethical, or methodological. We need to normalize the idea and present it as an option whenever possible.

This persuasion and advocacy also apply to the people we would like to participate in our research. Because participatory methods are time-intensive, we are asking people to invest a lot of time and energy. We need to communicate why they ought to do it, show that our intentions are genuine, and demonstrate the real benefits. This is especially important among groups who already experience survey fatigue. We need to communicate effectively to get buy-in from both decision-makers and participants.

00:38:33 Working with vulnerable and marginalized communities

How can we best use participatory monitoring and evaluation when working with marginalized and vulnerable communities?

This goes back to the question of how we adapt this to participants' needs and the situation they are living in. Deciding on M&E processes is fundamentally an ethical choice and one that should be conflict-sensitive and guided by the principle of 'do no harm.' Any type of research has the risk of unintended consequences, like reinforcing discrimination or putting people's safety in jeopardy. Participatory M&E is a very time-intensive process, and relationships are often long-lived. You need to think whether a participatory approach is appropriate in a certain context. For example, in a humanitarian crisis, a long, drawn-out process might not be appropriate when people need to focus on survival.

When working with particularly vulnerable people, like survivors of gender-based violence, you need to be sensitive to their trauma. You need to provide referral information to support services and safeguard their anonymity. Participatory approaches are often community-based, and there can be consequences for individuals. We also need to think about power relations within the community itself. We have a responsibility to prevent existing inequalities from being reproduced in our M&E processes. Methods could be dominated by powerful voices—political leaders, men over women, adults over youth. We need to design the process to include marginalized groups without compromising their safety.

Understanding the context and the group is crucial because there are biases or environmental issues that might prevent people from presenting information freely. In some instances, you need to build trust so they can open up. Being aware of hidden biases is key. Also, while research often requires impartiality, in cases like GBV, you cannot just extract a story and leave. You must offer opportunities for referral or counseling. If they request anonymity, we must respect that, but ultimately be able to offer a solution rather than just collecting information.

00:45:44 Ensuring participation in all phases of the M&E cycle

How do we ensure participation is happening in the different phases of the monitoring and evaluation cycle? How do we know who should be involved per phase?

Ensuring participation in each phase of the M&E cycle requires consistency. The approach should be integrated into all different phases, which means you need to plan for it. I know that's not always possible if an M&E system is already set up, but to the extent possible, we should apply these principles. Importantly, when you involve someone, you need to make sure that you act on any of the outcomes of that participation in the subsequent phase. If you don't carry that over and use it, it runs counter to the philosophy of a participatory approach.

Relationship building encapsulates the rest of the cycle. Building and maintaining culturally responsive relationships between practitioners and participants is so important. Good facilitation draws on that relationship of mutual trust. Regarding who should be involved, ideally, you would like people to be involved from the very beginning. This should be a collaborative, open process of dialogue. In the discovery or setup phase, you would usually do a stakeholder mapping to identify relevant parties and ensure you don't lump people together who have different experiences.

The process should be clear from the beginning. In most cases, evaluators collect information but don't give feedback. We need to build in reflective cycles throughout the process. As we work with them to monitor and collect information, we must share it back to help them in planning. It should be built into the entire process rather than just waiting for annual reflections. The beneficiaries should be part of the process to help them reflect on milestones and change course if needed.

00:53:15 Participatory M&E when working remotely

How do we ensure participatory monitoring and evaluation when working remotely?

It is a tricky one, but the same considerations apply. We must ensure that adopting digital or virtual tools doesn't produce barriers for certain people to participate. Using digital tools comes with challenges: people may not own devices, have internet, afford mobile data, or have the skills to use them. Privacy and security are also concerns. It is possible to do participatory M&E remotely using various digital tools, but we must be intentional and honest about what we can achieve and how inclusive we can be. You can try to work around barriers, for example, by subsidizing data costs. However, working remotely adds complexity. Personally, I rely on people who are physically on the ground to do the bulk of the work.

Being intentional and honest is key because it is complicated. Some tools, like SenseMaker, are available but require technical skills. I agree about identifying people on the ground and building their capacity, as well as the capacity of the beneficiaries themselves. For example, they can keep advocacy logbooks, journals, or diaries. You can create linkages for touching base and reflecting on the information they are collecting. If you are not able to be on the ground, you must invest in the people who are there.

00:57:55 Conclusion

Thank you both, Naomi and Mercy, for your participation today. You shared really interesting insights. I hope our participants enjoyed this session and learned something new. The recording of the session will be on our website soon, and we will also share it via email. ActivityInfo is here for you if you want to try the platform for simplifying your monitoring and evaluation activities. Thank you, everyone. Take care.