00:00:02 Introduction

Hello everyone and welcome to another webinar hosted by the ActivityInfo team. During today's session, "Making Information Management and Monitoring and Evaluation Work at the Global Level," we will be hearing from representatives of organizations ACDI/VOCA and IREX. There will be one presentation by each organization, and then we can have a discussion and reply to audience questions. ActivityInfo is an information management platform used for monitoring and evaluation, humanitarian coordination, and case management for over a decade by UN agencies, INGOs, and NGOs.

I would like to introduce you to our panelists. From ACDI/VOCA, we have Dr. Emmanuel Dormon, Managing Director and Head of the Monitoring, Evaluation, Reporting and Learning department; Dr. Jennifer Himmelstein, Director of Corporate Analysis, Monitoring, Evaluation, Reporting and Learning; and Mr. Robert Sackey, the Regional Systems and Technology Specialist. From IREX, we welcome Mr. Charles Guedenet, Senior Technical Advisor for Monitoring, Evaluation, and Learning, and Sharon Lang'at, Data Management and Learning Specialist. Finally, Alexander Bertram, Executive Director of BeDataDriven and founder of ActivityInfo, will be moderating the session.

00:02:50 ACDI/VOCA: Organization overview and M&E approach

Emmanuel Dormon: Thank you very much for inviting us to share our experience in M&E in general, and specifically how we use ActivityInfo to support our global M&E management system. ACDI/VOCA is proud to be celebrating our 60th anniversary this year. Founded in 1963, we have worked in over 146 countries. Our home office is based in Washington, D.C., and today we are in 20 countries working on about 26 different projects. Our focus is to deliver technical and management assistance, especially in the area of agribusiness. Lately, we are focused very much on market systems because we believe that the way to make our work sustainable is to make systemic changes in how agribusiness is done. We also focus on ensuring financial services are available to clients and participants, alongside community development and food security.

Our approach to M&E is very much one of accountability to donors, stakeholders, participants, and our organization. Most importantly, it is about learning how to do things better—learning from our failures, successes, challenges, and context to improve how we implement or design programs. This requires a reliable data system; we cannot learn from raw data. Our data management tool is focused on evidence-based decisions, ensuring high quality and security while allowing us to draw correlations between services received and outcomes like productivity and income. Internally, we use a relational database to track these linkages. Data is a management tool critical for understanding success or failure, checking project performance quality, and analyzing data across projects to identify what works in different contexts.

00:10:35 ACDI/VOCA: Rolling out ActivityInfo

Robert Sackey: At ACDI/VOCA, we have staff supporting specific projects in specific countries, including M&E and MIS staff. At the HQ level, we have teams supporting all projects across the globe. The rollout of ActivityInfo took about six months, involving a combination of needs assessments and gap analysis. We explored various platforms and settled on ActivityInfo. We brought on stakeholders to get feedback and conducted tests globally. We identified four pilot projects ranging from complicated to simple to see how the platform adapted. The feedback was exciting, so we scaled up, migrating existing projects from our old platform and starting new projects from scratch on ActivityInfo.

Our database design approach is driven by project needs. We start with a list of approved indicators for the donor. We hold brainstorming sessions to ensure a common understanding of requirements, data collection frequency, and reporting frequency. This helps us create a schema where we identify the necessary tables and relationships for every single indicator. We work with the MIS and MEL teams of the respective projects to empower them through training. We then input dummy data and use Power BI to create dashboards that automatically calculate indicators and refresh as data flows in.

Regarding database maintenance and support, we ensure sustainability through our brainstorming and development process. The project teams provide the first line of support, HQ provides the second line, and ActivityInfo provides the third line for complex issues. ActivityInfo has simplified data synchronization significantly. We used to juggle offline data collection tools like Kobo or ODK, downloading to Excel and doing bulk uploads. ActivityInfo’s mobile app allows for offline data collection, and the bulk upload feature is very efficient, allowing for partial uploads where good records are accepted and problematic ones are filtered for cleaning. The formulas and syntax are very close to Excel, making it easy for staff to adapt.

00:24:46 ACDI/VOCA: Project and corporate level M&E

Jennifer Himmelstein: At the project level, we build the database around donor-required indicators. Routine data collection is built into trackers and registries, while outcome data is typically built into survey tables connected to these registries. To enhance data quality, every project has a data management manual detailing calculations, sampling methodology, and verification documents. ActivityInfo’s relational database allows us to maintain unique records for individuals and organizations using primary keys, preventing duplication. Data must be auditable, so we upload verification documents directly into the system.

At the corporate level, we have a set of indicators aligned with the UN Sustainable Development Goals. We aggregate data from across our projects into corporate dashboards using ActivityInfo. We are working on automating this further. We also produce an organizational impact and learning report using data extracted via the R package for analysis in platforms like SPSS. We maintain a MERL team space with a document library for policies and guidance, ensuring staff have access to best practices.

We utilize data for learning and adaptive management, using the USAID CLA matrix. We have regional knowledge-sharing hubs where projects discuss issues and troubleshoot together, promoting connections across regions. We also have communities of practice for M&E and CLA with regular webinars and capacity building.

00:30:50 ACDI/VOCA: Summary and recommendations

Emmanuel Dormon: In summary, our MEL system is designed for accountability and learning, with data quality being critical. We chose ActivityInfo because it is user-friendly, supports online/offline mobile data collection, and provides data integrity and audit trails. We are working on effectively aggregating data from projects to the corporate level for analysis. Our pilot approach was crucial in avoiding mistakes during migration.

Robert Sackey: Our recommendation is to involve field teams from the start. Their ownership and feedback were critical during the pilot and rollout. While we faced challenges with reporting visuals, necessitating the use of Power BI, we see enhancements coming within ActivityInfo. The migration support from the ActivityInfo team was awesome; they were prompt in resolving issues.

Jennifer Himmelstein: The ability to include verification documents directly encourages partners to submit evidence with raw data. Traceability features, like tracking who entered data and when, are vital. The bulk upload feature helps identify issues while maintaining validation rules. The mobile data entry platform is a huge time-saver and ensures quality. The user tutorials and webinars provided by ActivityInfo save us from creating our own training materials. Finally, having standardized MEL policies and resources is important for any organization using an MIS platform.

00:42:10 IREX: Global MEL strategy and vision

Charles Guedenet: IREX is a global development and education organization working in over 100 countries with an annual portfolio of about $100 million. We work in cultivating leaders, empowering youth, strengthening institutions, and increasing access to quality education and information. Our portfolio is diverse, ranging from small projects to large $40 million programs, which means our M&E strategies and contexts vary significantly. This creates a challenge in building a global MEL system that accommodates diversity in programming, context, and learning needs.

In 2016, our strategy was to use Salesforce for standard corporate indicators. This worked for communicating reach but didn't meet learning needs at the program level. Around 2021, we launched our 2025 strategy and a new approach to Strategic Evaluation and Learning (SEL). We now collect program indicator data in ActivityInfo, requiring programs to report all donor-approved indicators. This allows different users—such as leadership, education practices, or specific approaches like "Learn to Discern"—to query the database for their learning agendas. At the corporate level, we map project indicators to our strategy outcomes to report on progress towards targets.

Our vision is to connect program-level results to global strategy and impact, make data accessible for cross-program learning, and identify how programs measure hard-to-measure outcomes. We aim to build a culture of learning and data-informed decision-making. We piloted ActivityInfo in late 2020 with a few programs and scaled up in 2021 and 2022, bringing all programs into the system to report indicator data. We also have programs using it for their full MEL data management. We are now institutionalizing the system, focusing on program adoption, data consolidation (moving internal quarterly reports into ActivityInfo), and providing support through onboarding packages and customer success plans.

01:04:00 IREX: Supporting programs and adoption

Sharon Lang'at: We have started actively rolling out ActivityInfo to programs over the last four months. Adoption is demand-driven; programs come to us with their needs. Usage ranges from low-level (pre- and post-test analysis, participant tracking) to high-level (full MEL indicator data collection, centralizing raw data, and analytics). Some programs use ActivityInfo for data collection and visualization, while others query data out to R or Power BI.

For new programs, we start with an orientation call, create a dummy database for them to interact with, and then proceed to database design where we work together to create the schema and forms. We pilot the tool, train them on database management practices, and provide ongoing support. For existing programs, we draft a data migration plan, design the database, migrate the data using the bulk import feature, and train the team.

Challenges include adoption resistance from programs used to other platforms. We manage this by involving users in the design process and allowing flexibility. Best practices include utilizing the ActivityInfo online resource library and providing continual training and webinars so programs can learn from each other.

01:13:00 IREX: Challenges and future innovations

Charles Guedenet: As we institutionalize the system, we face process-oriented challenges like managing users during staff turnover and archiving closed programs. We see a need for a help desk ticketing system as requests increase. We are also looking for workflow features within ActivityInfo, such as triggers for data review. Interoperability is another focus; while the API works well for Power BI and R, we want to streamline data entry across other platforms like Salesforce and SharePoint to avoid duplication. Finally, we are looking for ways to connect separate program databases within ActivityInfo to each other.

01:15:42 Discussion: Centralization vs. decentralization

Alex Bertram: It is interesting to observe the difference between the two organizations. ACDI/VOCA has a centralized process where the tool decision was made centrally, while IREX gives programs more flexibility. Emmanuel, is that accurate?

Emmanuel Dormon: Yes, it is a central decision and policy that every project uses the ActivityInfo platform (LEAP 2.0). This gives us control and visibility over data quality and reporting. We are responsible for what our projects report, so we want to ensure we know what is going out.

Charles Guedenet: At the global level, we do have a requirement that all programs enter donor-approved indicator data into ActivityInfo quarterly. This is centralized for corporate reporting. However, at the project level, it is less centralized and more demand-driven. Because of IREX's growth and the introduction of many new technologies, we found an incremental process worked better to manage change and reduce resistance. We aim for all programs to use it, but we will always allow flexibility for programs with specific needs to use specialized software if necessary.

01:23:48 Q&A: Corporate vs. project indicators

Alex Bertram: A question from the audience asks if the corporate indicator database is based on project indicators or if they are separate sets.

Charles Guedenet: In 2016, we had standard indicators everyone reported on. Now, our only standard indicators are around reach. For strategic outcomes, we map project-level indicators to those outcomes. We tag and code them, and then aggregate data where possible or use qualitative statements to report on our corporate strategy.

Jennifer Himmelstein: We used to have a set of required indicators. In the last few years, we changed our approach. We did a mapping exercise to identify the most common indicators across projects within thematic areas aligned with the SDGs. We prioritize these; if a project collects them, they are aggregated into the corporate indicators.

Emmanuel Dormon: Going forward, we are looking for room to have projects collect additional data needed for corporate reporting if it falls within the project's mandate. We try not to disrupt projects by asking for data unrelated to their work.

01:27:17 Discussion: Staffing and roles at headquarters and project levels

Alex Bertram: What roles are important at the headquarters level to support a successful global MEL strategy?

Robert Sackey: My role is to build the capacity of field teams so they own the platform and are responsible for their numbers. We provide support and troubleshooting to build their confidence.

Sharon Lang'at: My role is similar to Robert's. I support the MEL program teams ensuring they can own their work and train their teams, while I remain present on the back end for support.

Jennifer Himmelstein: I focus on collaboration, learning, and adaptation. Supporting a portfolio of projects keeps me grounded. Dividing labor between MIS and M&E staff is crucial; combining these roles is often overwhelming and leads to headquarters having to fill gaps.

Charles Guedenet: I have a big-picture view and interface with the strategic level. I work with Sharon to ensure she has support. We also recently added MEL Liaisons in practice areas to support data entry and learning.

Emmanuel Dormon: We have three core support areas: direct project support checking reports and survey quality; the database management team (Robert's team) setting up and maintaining databases; and the learning support (Jennifer's team) looking at project and cross-project learning.

01:39:35 Q&A: Cost-benefit analysis and policy engagement

Alex Bertram: We have a question about using cost-benefit analysis to measure investment and how to engage stakeholders like the government with evaluation results.

Jennifer Himmelstein: We don't often get requests for return on investment (ROI) assessments as they can be difficult. We do look at indicator data to see beneficiaries per dollar or leverage versus grant amount. Regarding policy, we track policy indicators. Sharing data with stakeholders is a best practice; we hold learning events and use social media to disseminate information to implementers and government bodies.

Charles Guedenet: We also don't see many requests for ROI. It is hard to monetize many benefits. Regarding advocacy, most of our programs work with partners, so the advocacy to local government is usually done by those local partners. We may hold learning events where government officials are invited, but direct advocacy is usually partner-led.