00:00:06 Introduction

Thank you for the gracious introduction, Faye. As you introduced, my name is Brendan O'Neill. I'm the commercial director at BeDataDriven. Really excited to see all of you here today for this webinar focusing on program data. We have a pretty packed agenda today. We're going to go through a few different topics. Initially, we're going to discuss some of the challenges with program data and managing it. Once we set the context and provide you with some information about program data to make sure we're all on the same page, we're going to discuss actual information management systems that will help you manage that program data and get insights out of it.

We'll pivot over to some of the architectures that we see for program management information systems. How does program data fit into the broader IT ecosystem at small to large organizations? And at the end, we'll be sure to leave some time for questions and answers. I encourage you to, as we're going through the presentation, if you have any questions, please add them to the Q&A rather than the chat. If we don't make it to them today, then we'll be sure to follow up afterwards.

00:01:47 Challenges with program data

First, I'd like to start with the discussion around what are some of the challenges with program data. Based on our experience working with social impact organizations, there's typically three pillars of information that needs to be managed effectively in order for a data-driven organization to be successful. There might be some variations in this depending on the organization you're working for, whether you're a local nonprofit or maybe a UN agency, but I think more or less this largely holds true.

We see fundraising as a key pillar where you really need a system to track your funding prospects and help manage relationships with them. This use case is really similar to sales workflows in the private sector, if not the same. This is why you see a lot of CRMs or customer relationship management systems being used pretty successfully to support this pillar of information management for fundraising operations.

The other pillar that we see is really around enterprise resource planning. You need a system like this to handle your day-to-day operations, to help manage finances or HR and procurement, for example. And there are a number of pretty well-established providers of ERP systems. They're really good at supporting predictable lines of business like those I just mentioned: HR, procurement, finances.

Program management systems are the focus of the webinar today. They're a little bit different and sometimes they're a bit challenging for IT organizations, but they're really fundamental to what we do. Our programs are what we do every day. They're how we drive impact. And the other systems don't really exist except to support our programming. But nonetheless, it's often the information management system that a lot of IT professionals struggle with. And maybe this is something you struggle with in communicating to your IT, or if you're an IT professional, you struggle in addressing the needs of your program teams. It's not really surprising because it's relatively easy to configure a CRM or an ERP to support fundraising or ERP-type workflows because those technologies specifically are designed to support those workflows. But managing program data is different. And when ERPs or CRMs are used to manage program data, it can create some challenges.

00:05:03 Defining program data

So before we go further into this, we just want to get everybody on the same page. I know a lot of you are coming from the M&E world, so this might be obvious. But what we see in a lot of organizations is that maybe not everybody is aware of all of the data that's actually being managed within an organization. So if we start at the foundation, then we're really talking about the project level. This is what we would call primary data or the raw data, the granular data. This is the data that documents who are the individuals that you're supporting or that you're uplifting. Could be also households, schools, clinics, facilities. What are the activities that you're undertaking? The organizations that I see in the chat span a wide range of activities from education to emergency aid to market systems. And of course, surveys to assess impact, assess needs, find satisfaction with the program.

This data serves obviously the monitoring and evaluation team. This is the input data for our monitoring and evaluation programs. But it's not just M&E; this is the data that supports our field teams. This is the data that supports our project managers, our program managers. It's really the foundation of our work that data exists. Even if it's on paper, even if it's in Excel sheets, it's somewhere in your organization. Now then, from this raw data at the project level, ideally, you're calculating your indicators. Those indicators are being calculated from the raw data. So then you get the value of the indicator that you're reporting to your donor, to your stakeholder. But it's this raw primary data that we're concerned with at the project level.

Now, if we look one level up, when you roll up to the organization or program level, then you start to hear people talking about, "Oh, I just need to collect indicator results. I just need to get the numbers from the project or from the field offices." And so you go from potentially hundreds of thousands of data points or millions of data points to maybe a few hundred numbers, a few thousand numbers, depending on how much disaggregation you're tracking. To what degree are indicators standardized across projects? What is the level of granularity? Some organizations, in terms of their central approach at headquarters, they may stop here with an information system. You may have an information system from your headquarters where every quarter you have to go in and key in your project updates or your indicator numbers.

But not all organizations are thinking about where those numbers come from. If I'm reporting to you the number of boys, girls, men, and women that we supported in education over the last quarter, do we have a system in place, a repeatable, reliable system for collecting, managing, and computing those numbers? So part of this presentation is really just an appeal to keep in mind those primary data systems as well as the indicator data.

I just want to mention there's often a third level for large organizations with hundreds or thousands of projects. Each of those projects may have their own sets of indicators that are driven by the sector that they're working in, or the donor, or the language. And often there's a need to roll those indicators or link those indicators to strategic indicators. Those are the kind of numbers that might end up in your annual report, but they're also key information for people writing new proposals, for grant writers, for fundraising to be able to demonstrate the kind of work that you're doing. So this whole landscape here is what we mean when we're talking about program data. And it's not just M&E, though it's often the responsibility of M&E to set up these systems. But it's information that really serves everybody working in the organization if we do our job right.

00:10:15 Why program data is different

My experience is that program data in the social sector is fundamentally different than the kind of data that's managed in CRMs or in ERPs. When you roll out an ERP across your organization, it's not always an easy experience. It can be a long process, but it can be centralized. You can define a process for invoicing, for human resources, for procurement that is simply enforced across your entire organization, no matter how many countries you're working in. And that can be centrally administered from headquarters.

Program data is really different, especially when you are approaching the humanitarian end of the spectrum. One day you could be rebuilding schools for returnees from a conflict or from a natural disaster. As the political situation changes, as we've seen in the last year to really devastating effect, you're shifting from long-term activities to food distribution to emergency life-saving activities. And so suddenly the data that you need changes overnight. And that makes it very difficult to use many of the tools from the private sector in our work.

The result is that program data is often not organized. It's maybe the last thing that an organization will tackle after they've established the fundraising, after they've gotten an ERP system in place. I was just at NetHope in Washington D.C. and I talked to a lot of great people doing great work, but I met a few IT leaders and they say, "Oh yeah, how are you managing program data? That's not my job. That's tough stuff. I just run the finance system." And there might be a division of labor there that makes sense, but often it's an area that in my view doesn't get as much attention as it should.

And what happens is that you see a couple of patterns. One is you might have a system in place for gathering indicators, but there's no attention being paid to where do those indicators come from. How are we coming up with those numbers? Do we have a system in place for tracking rights holders, beneficiaries, activities, and is that done consistently? Sometimes you'll see a data collection strategy without a data management strategy. So you've got people out there running surveys every month, but where does that information live? Is that accessible to people in the organization and is that data connected to your activities? Or does somebody write an email or a report and then it's gone?

Or you'll get a situation where you have bespoke systems in each country. So in one country, you've got a spreadsheet genius who's created this amazing system with Excel. In another country, somebody coming from maybe a more techie background builds a database. And in a third country, you've got a five-year program, so you hire a local company to build a software specifically for that program. It can work in some places and you can have some great local examples, but it's often not the case that it's harmonized across the organization, and that brings risks and costs with it.

One thing is just data quality. If you don't have a system in place of how do we get these indicators, you know the rule: garbage in, garbage out. The information that we spend so much time, energy, and cost collecting—how much money do we spend collecting this data that isn't accessible to other members of our organizations? And the third thing is the risk of it going wrong when there is not a central governance plan. If we don't have a plan for how we're managing this data in each and every one of our field offices, where does this live? You'll have spreadsheets with beneficiary personal information scattered on laptops and unmanaged phones or in a constellation of online web programs, Google Sheets, Microsoft 365, and 15 other random apps that might be good on their own, but not if they're being used by one person with a credit card. That's a real risk.

00:16:10 Poll results

Before we move into program management information systems and what we think is a potential solution, I wanted to jump into our poll. We're trying to capture a little bit of information that we can share back with you on how you're managing program data in your organization. We're also interested in understanding the different technologies that you're using to manage it as well as if you trust the information that's being managed there. Is it something that you feel confident or comfortable using for decisions or for reporting to your donors or other stakeholders?

There's actually a surprising number who don't really have an information system in place to manage program data. So hopefully some of the information we share might be helpful to you. But I'd say the leader, and that's probably born out from my experience, is that there's generally something in place, but how is it coordinated? What's the standardization? That's the thing, is that you could say, "Oh, we don't have one," or "That's not something that we do," but it's there. It's either shadow IT, or people are rolling their own. I've been there in my first job as a junior program officer. We didn't have anything like this in the field office that I was working. We threw this together, something to manage. I had to tend to 140 schools and clinics. You gotta build a system, and that's gonna be in Excel or I think I used Microsoft Access at that time. But it wasn't centrally governed. There was no one saying, "Hey, this is the best way to do it," or "This is the way we do it in our organization."

00:20:22 The solution: Program MIS

Let's pivot over to the solution to these challenges, which is really a program management information system. Based on our experience with the different organizations that we work with, there are a few requirements that a program MIS really needs to have in order to be implemented, adopted, and sustainable over time. The first is it needs to be easy to implement. Long IT projects are a challenge, long time to value, they're expensive. So there needs to be a pretty low threshold for the investment of time and resources to get going.

Also, they need to be easy to use. There are varying degrees of technical literacy both from the people that are managing the system as well as those that might be reporting into it. So it's really important for whatever solution you use to be easy to use and easy to manage so that it's not a single point of success for someone that's enthusiastic or a technical guru, but then when they leave there's no one there to really fill that void and take that system forward.

We mentioned the challenges especially in the humanitarian field where the landscape is really constantly evolving. The data management needs are constantly evolving. The system needs to be flexible in order to accommodate these different data models, these different data needs. And lastly, until we're fully connected with satellite Internet, we still need to have that offline mode where an individual can continue to work and do their job when that connection is either low or intermittent or completely disconnected.

Once you've got your program MIS, and maybe you've proven it in a limited context, implementing it institutionally across these different regions or thematic areas has a lot of value that it can bring to your organization. First, it improves trust with stakeholders. It also has the ability to lower HR overhead as well as IT overhead investments in people and technology. And I think one of the most important benefits is it leads to more responsive and efficient or effective programs themselves. It does this by maintaining better quality of data and achieving economies of scale in deploying or implementing across multiple contexts.

In addition to the features baked into the technologies, the actual components you need to implement are really these standardized processes with standard information flows on a common technology. And we understand that every context is different. In fact, we highlighted that in our introduction and that different thematic teams or countries are going to have their own information management needs. Still, there needs to be some core data that should be standardized institutionally. If you're really thinking strategically at a global level, then there is going to be some core needs for specific data types that you're collecting across the organization so that information can roll up.

As a technology platform, ActivityInfo helps achieve this by providing a no-code relational database. It is super easy to use and implement. And it's scalable and flexible in that you can implement unlimited databases for practically unlimited use cases on the same platform. And it's really one system that supports the whole program data management life cycle. Specifically, when we talk about what ActivityInfo does, it supports these core pillars of data collection, management, analysis, and reporting through the ActivityInfo web and mobile applications.

00:26:24 Case study: ACDI/VOCA

I just think we'll look first at a case study of an organization that I really admire and how we work with them to help them improve their management of program data. I want to first start with ACDI/VOCA. If you're not familiar, they work more in the development space. They're based in America but they work in I think 35 to 40 countries. They've been around a long time. And they work to improve market systems to expand opportunities for people worldwide. And so they have programs that can be quite diverse from programs supporting coffee farmers in the Philippines to smallholders in Ghana.

When we met them in 2022, they were quite far ahead of the curve of a lot of organizations. They had implemented an institutional standard built around a couple of different technologies. I think they were using SQL databases for each of their projects. So each project in each of their 40 projects had their own database that was connected to something called Access web databases, which have since been deprecated. But this didn't really provide a solution for data collection or data analysis. So then they kind of connected a bunch of different mobile data collection tools in different countries to bring this data together.

Some of the pain points, the reason that they came to us, is that just moving this data between all of these different tools was just exhausting. You do a survey of beneficiaries. You want to link the beneficiary to the other surveys that you did six months ago or a year ago or activities that you've delivered. So you use some beneficiary codes, but that relies on people using and typing these codes correctly. What they would have to do is dump this into an Excel spreadsheet. They would do a full day or two of checking against these codes. Go back to the data team to say, "Hey, you wrote Maria Fernandez here. Is this the same as Maria Sante Ferranda's because the code is missing or it's wrong?" It just created so much friction. So that even getting to an indicator value of how many distinct people did we train over the last year really took all of the effort. And so then you don't have space for learning or looking at surveys across different contexts to understand common drivers of change.

We were able to work together to roll out ActivityInfo to essentially replace their technology stack. Over the course of the pilot, it was six months. I think by 12 months they had rolled it out to most programs and by the two-year mark this year, I think all of their projects are running on an ActivityInfo database.

I just want to connect to the points that Brendan was making. First of all, it reduced the cost of staff time. Cleaning data and the complexity of linking all of these tools together was very costly. Also in engineering time, having staff on their team that could do this. Whereas in ActivityInfo, we provided in their case a fully managed and supported platform. The integration with the mobile app was so key for them because it meant that they didn't have to do this extra cleaning step. With ActivityInfo's mobile app, you've got a replica of the database on your phone. It's not just a data entry form, it's a replica. So when you do that survey, the mobile phone can immediately check that this is a person that's enrolled, that it's the right person with the right birth date, etc. And that frees up space to actually start looking at the data, which is the phase that they're in now, and learning from what their data tells them about their interventions.

00:33:00 ActivityInfo demo

Like I mentioned with ACDI/VOCA and many of our other customers, essentially for each of your projects, you're going to create your own database. You can do it from a template, but often programs are so different that you really have to start with thinking through what do I need to collect, what's my MEAL plan, what do my field teams need, and set it up to specification. ActivityInfo makes it easy to add that database. To do that, I'm just going to add a database here. I'll just call this the project MIS.

Let's add some structure here before we open up to the rest of the team. Just click on add form. The first thing we want to maybe do is track beneficiaries, rights holders, participants. I've got here a form builder. I can just simply add the things that I need. Let's add a text field for the full name. I'll make that a key field as well. Make sure that's unique. You can see the different data types that you can capture with ActivityInfo: everything from quantities and dates to calculated fields, geographic data, attachments, image media. Let's just add a date for the date of birth. I'm going to set some validation rules to make sure that we get good quality data from the start. Make sure that the date of birth is less than today.

Now I'm ready to start bringing data in. We'll look at the mobile app in a second where you would actually use this in the field. But often you've got data coming in from legacy systems or from a partner. Let's take a look. I've got here a Google sheet with some fake beneficiary data. I can just click on the import button, paste that data into my database. ActivityInfo is going to help me line up this data to my schema. All the other columns are automatically detected and they seem to be good. But there were two invalid records. I've got some data here where the sex was left blank and some required fields were missing. You can download those separately, send those back to your partner or correct them yourself.

Now it's also going to do some basic duplicate checking. The importer will do exact duplicate checking to make sure that that full name that I marked as a key, that we don't have two people with exactly the same name. But we still might have approximate duplicates. You might have people with similar names or the same name and a slightly different birth date. One of the exciting things that we've rolled out in the last couple of months is the duplicate checker. I can go up here to Tools and scan for duplicates. It's already found one group of duplicates. You can see here I've got two Dora Williamson with an extra O. So I can go ahead and either merge that record if I need to keep it, or if it's not linked to any other data, I can just go ahead and delete that extra. ActivityInfo is going to keep a record of all of the changes. If I make any edits or if I delete or merge anything, there's a full history of those changes and who made it and when.

Now let's build out our database a little bit more. We've got the beneficiary registration form, but so far it doesn't look much different than something you do with a data collection tool. What we're going to do now is we're going to start to link some of our activities with that data. Let me go ahead and add a training form to keep track of the trainings that we're doing. Maybe just the date and the theme of a training. And now I'm going to add some of the relational fields which is really ActivityInfo's superpower at the project level. It helps you connect data between forms.

I've got a list of trainings here. What I'm going to do now is add a subform which is essentially like a list within this because I've got my training and then I want to know the list of people who attended. I click on subform and now I've got a nested form, a list of participants where I'm going to add not the name or ID of a participant. I don't want people to click that in directly. I'm going to use a reference field. A reference field allows you to choose from another form in your database. We're going to link this to the beneficiary registration and save all of that. Now I've got two forms that are linked together. One is about my beneficiary with some baseline information and the second is on the trainings that were provided.

Let's go ahead and invite some field staff. I'm now going to go into the user management portion. You can configure this very granularly to choose exactly what roles and what operations each person is allowed to do. I'm just going to go ahead and invite somebody as a field worker. You can choose the language. We have automatic translation and manual translation built in so you can easily work across languages.

Now from my phone, the field worker gets the email, says hey, you've been invited to go to the database. That's going to download and install the app on your phone. So now I've got access, I've got this database in my pocket. I can download that on my phone just like you can in the web browser. The offline mode works in the web browser as well. And now I can start tracking these trainings that we're doing. As I mentioned before, this is not just a "fill out a form and send it." I've got actually a list of the trainings that have already been registered but I'm going to go ahead and add a new one today. I'm going to choose the district it takes place in, choose the theme, and now you see this subform. So now I can start adding participants and it's going to allow me to search within my previously registered participants for the right person.

ActivityInfo doesn't stop there. As I showed in the ACDI/VOCA case study, you're probably going to want to aggregate this data up to the program or global level so that you have a list of all of the projects in your organization that's fed by those project level data. You can do that manually, you can do that with automations. All of that feeds into a global dashboard for the organization where I can see the active projects. I can see 60% of my projects are on track. Based on the latest reports, you can see where the organization is most active and I've got highlighted for maybe our grants manager or our central team what are the two projects in my portfolio that are off track and maybe require attention.

00:45:30 Integration into the IT landscape

We don't have a ton of time left and I want to have some time for questions, but I do really quickly want to touch on how Program MIS and ActivityInfo fits in the broader IT infrastructure. A lot of our customers will actually use ActivityInfo for the complete data life cycle: data collection to report generation. But we understand that some organizations want or maybe they're required to use other tools not just for program data but also for other systems.

So that kind of brings us to another implementation pattern for ActivityInfo, which is ActivityInfo as a system of record for program data. In this architectural pattern, ActivityInfo is really that authoritative source for program information and then other tools are extending its capabilities, you know, business intelligence tools, GIS tools, or other scripting frameworks. So you might be using R or Python to do more advanced analysis or using the APIs to do system communications. These integrations, except for R and Python, they don't really require any development knowledge or scripting know-how. It's really easy to create a connection to your GIS, to your BI tools directly with ActivityInfo as the back end. So that as data is coming into the system, it's being updated and managed, you're getting live views in those other systems.

And the last part I wanted to highlight is really ActivityInfo as a source system for data warehouses. When you're managing your program data in ActivityInfo, it's already structured and BeDataDriven is maintaining robust, well-documented restful APIs and that makes it really easy to move data from ActivityInfo into a data warehouse and vice versa. This is powerful because then you can combine these authoritative sources that these other systems are supporting. And when you're combining the different institutional data, then you can gain better insights. So for example, maybe your program teams want access to financial information that's in another system. They can pull that into ActivityInfo and create new visualizations or reports that help them monitor, track the programs that they're implementing.

00:54:45 Q&A

We've got quite a few really good questions. I'm going to start maybe with some easy ones just about features. There were a number of questions about offline support. I want to underscore that ActivityInfo has excellent offline support. It's something that we invest a lot of time and energy in. The mobile app can work entirely offline. You synchronize a copy of the database. It's the only tool that I'm aware of that does a full replica of the database, at least what you have permission to access, on your phone. The same mechanism also works in the web browser. You can be cleaning data, updating, making changes. The generator goes down, the 4G is out, and you can keep working.

Quentin: The idea is to ensure consistency at organizational level with the main KPI of the organizations while allowing for flexibility at country level to create their database forms and manage them. How do you ensure that with ActivityInfo?

Alex: That's a really excellent question. The answer is, it depends. There are two main ways that we approach this: the top-down or the bottom-up. What I would call the top-down approach is built on processes essentially. Clearly defining and communicating your organization's key KPIs and ensuring that every database that's created has a report that responds to those KPIs. You can build and tag a report that has those indicators that can then feed the rest and your organizational dashboard can pull from those indicators across all of the projects.

I've also seen recently we worked with one of our partners on a kind of bottom-up approach. They simply weren't in a position to communicate beforehand because there were so many different types of activities. So they just accepted that diversity to start with and each program reported in ActivityInfo their indicators and they ended up with I think 3,000 indicators across 40 countries. They used that as a way to determine what should our key indicators be. They used partially our deduplicator function but also some external scripting to kind of merge these indicators, find some common patterns, and then link those 3,000 indicators to a key set of 20-some indicators.

Stephanie: The organization that I work for, WeForest, we do a lot of mapping activities. One of the anxieties that is felt about trying to move to one more consolidated system is that we will lose functionality around mapping and GIS. I was excited to see one of your slides earlier there was like a two-way arrow between the ESRI logo, the QGIS logo, and ActivityInfo. Just wondered if you could say more about that.

Brendan: I can speak most knowledgeably about the ESRI stack. If you go to an ActivityInfo form, you can actually get access to a JSON or a GeoJSON endpoint of the form. To be able to work with ActivityInfo data in QGIS or in ArcGIS, you need to basically copy that GeoJSON endpoint and paste it into QGIS. Specifically with ArcGIS Online, you use a tool called Data Pipelines. It's a very similar workflow. It's kind of an ETL (extract, transfer, load) with a user interface instead of writing your own script, where also you can drop that endpoint into a data pipeline and then work with that just as a normal geospatially enabled dataset within ArcGIS. I will say that ActivityInfo also has mapping capabilities within the system as well if you're generating ActivityInfo reports. But if you need to extend and do more advanced analysis or map production, then those tools work as well.

Moses: I'm looking at a humanitarian setting where normally there are a lot of changes, a lot of movements of the targeted beneficiaries due to a number of reasons. What ways can you use to manage that and keep linking the baseline registration data to the continuous activities you're implementing?

Alex: It comes down to data modeling. As people move between settlements, what information do we need about these individuals? Do we need personal information about them as they move around? Or is it enough to be in the right location? Or if we're interested in providing continuity of care, then making sure that we find a way to identify people even as they move around. Especially in cases where you have a state collapse, one option is to issue informal registrations. Our mobile app has a QR code scanner so you can essentially issue people cards that they can carry with them.

I think that's where ActivityInfo really shines on the mobile app. When you open the mobile app, you can review those baseline registration data and you can make changes. It's not a one-way system. Basically, your field team has a copy of the database in their pocket and can make those changes either directly or you can model it so that you have a new record of a contact point and then their person record is then updated automatically from the most recent contact that you've had with them. So in that sense, ActivityInfo does make it quite a lot easier to manage those kinds of dynamic situations.