00:00:00 Introduction

Welcome to today's webinar. We will briefly introduce the different types of shelter operations, which vary greatly depending on their specific objectives. We will explore the monitoring and evaluation frameworks for these operations and then transition from monitoring and evaluation theory to practical implementation. A core element of this transition is the design of an information management system. We will examine how to bridge the gap between theory and implementation, ultimately using a practical case study to demonstrate how an information management system can support the execution of these operations.

00:01:04 Setting the scene and audience poll

Before we begin, let's open a brief poll to get to know our participants better. The results show that the majority of attendees work in shelter operations, primarily in monitoring and evaluation roles. Many of you are actively seeking an integrated system or currently rely on manual, fragmented tools. While some are somewhat satisfied with existing systems, we hope to provide ideas on how to better support your work and provide you with more options.

00:02:53 The need for shelter

Let's start by discussing the fundamental need for shelter. Shelter is far more than just a roof and four walls; it provides a sense of safety, security, privacy, and belonging. People often consider shelter their most important asset and their highest living cost, making it a primary need that must be satisfied before other aspects of life, like working, can be addressed. Currently, global events such as disasters, wars, and rising inequality are increasing the number of displaced people in both urban and rural areas across developing and developed nations. According to 2020 UN figures, over 1.12 billion people live in slums or informal settlements, and an additional 300 million face absolute homelessness without any stable shelter. This includes populations affected by disasters, internally displaced persons, and those who have lost their accommodation within their country.

00:04:42 Defining shelter and settlement

It is useful to distinguish between a shelter and a settlement. A shelter is a basic physiological need, representing a specific physical structure or place that offers protection from the environment for an individual or a family. In contrast, a settlement is a socially and geographically defined area, such as a neighborhood, village, or camp, where people live. The primary difference lies in scale and scope. Think of a shelter as a single unit and a settlement as the broader system containing those units. A settlement approach targets a specific geographic area of high need and is frequently multi-sectoral, integrating water, sanitation, and hygiene (WASH), health, protection, and livelihoods. It engages the entire population, including both displaced people and the host community, and involves working with local stakeholders, governments, community leaders, and businesses.

00:06:05 Types of shelter operations

Shelter operations can generally be categorized into three different phases or scopes. The first is the emergency phase, where the objective is to provide life-saving protection from the elements and immediate security. Examples include distributing tents or repurposing public buildings into large collective centers following a disaster. The second is the transitional phase, which acts as a bridge. The goal here is to provide a habitable environment that supports a family's dignity and the gradual recovery of their daily routine. Projects in this phase might focus on providing rental support for a few months or offering prefabricated, flat-pack cabins. Finally, we have the permanent phase, aimed at providing a long-term, legally secure, and sustainable dwelling, such as fully repaired houses. Moving from the emergency phase to the permanent phase is often most successful when local or governmental authorities integrate policies like "Housing First" into their frameworks.

00:07:40 Real-world examples of shelter operations

Looking at real-world examples, a great case from the emergency phase is the self-recovery protocol implemented in the Philippines following Typhoon Haiyan. This project focused on remote, disadvantaged coastal areas with low local coping capacities. It quickly transitioned from providing emergency items to supporting permanent repairs. The interventions were area-based, integrating water, sanitation, and community infrastructure alongside individual support, and used various modalities to match the different recovery rates of households.

For the transitional phase, the Norwegian Refugee Council (NRC) implemented the Host Community Shelter Programme in Lebanon. This strategic intervention addressed the acute housing crisis while supporting the local economy and social stability. NRC used a market-based approach, providing financial assistance to landlords to complete or upgrade properties to a basic habitable condition. In exchange, the landlords provided rent-free housing to vulnerable families for 12 months.

For the permanent phase, there are many examples across European countries that integrate "Housing First" policies to guarantee long-term solutions. A notable project in Glasgow targeted individuals with complex needs, successfully mobilizing public housing and fostering strong partnerships with social clubs.

00:09:53 Monitoring and evaluation frameworks

When approaching the monitoring framework for these operations, we first look at development results, which include outcomes and impact. These articulate what the project hopes to achieve. Everything begins at the implementation phase, where activities are undertaken to transform inputs into outputs through technical, financial, advocacy, or partnership initiatives. Outputs are the products and services delivered upon completing these activities, achieved with the provided resources within a specified timeframe. Outcomes are the project's benefits, representing changes in institutional and behavioral capacities or development conditions that occur between completing outputs and achieving broader goals. Impact refers to the long-term effects on target populations produced by the intervention. These effects can be economic, social, cultural, institutional, or technological, and often involve changing conditions for citizens, city planners, or local authorities, with data always disaggregated by gender and age.

00:11:47 Defining and categorizing indicators

A key component of the monitoring and evaluation framework is tracking progress against established indicators. Indicators are variables used to assess progress towards expected outcomes, providing evidence of results and supporting effective planning, management, and reporting. They are typically quantitative, following SMART criteria, though qualitative indicators using SPICED criteria are also used. The choice of indicators depends on several factors: the type of shelter operation (emergency, transitional, or permanent), the focus area (such as shelter access, assistance, or damage assessment), and the specific stage of the project management cycle (planning and coordination, implementation, or evaluation).

00:13:15 Indicators by shelter type and focus area

The monitoring and evaluation approach differs based on the project phase. In an emergency, monitoring is acute, focusing on immediate survival, speed of delivery, and coverage, such as the percentage of the target population receiving basic shelter kits. In the transitional phase, the focus shifts to tracking how families move towards recovery and how settlements function, using indicators like the covered floor area per person, with the standard guideline being a minimum of 3.5 square meters per person. In the permanent phase, monitoring emphasizes long-term safety, building code compliance, and structural integrity, such as the percentage of shelters constructed according to local standards.

When categorizing by focus area, tracking access involves identifying the characteristics and needs of communities and households, like the percentage of target households living in adequate shelters that meet cluster standards. When focusing on assistance and provision, we monitor material or service delivery, such as the percentage of households receiving needed shelter support. For damage assessments, we monitor data about destroyed shelters and construction types to understand which shelter categories are most effective, tracking metrics like the percentage of houses damaged by a specific event. Additionally, shelter interventions are frequently combined with non-food item (NFI) assistance, requiring indicators like the percentage of the target population receiving NFI kits.

00:15:26 Indicators by project management cycle

The phase of the management cycle also dictates our indicators. During planning and coordination, we focus on baseline indicators to understand the context and the broad scope of a population's needs, which informs strategic planning. These are collected at the start of an operation, are time-sensitive, and often extend beyond just shelter-related information, such as the percentage of households indicating shelter as a priority need. During the implementation phase, we use process indicators to monitor what we are doing in the field. These are linked to operational stages and are monitored regularly to report on progress, activities, and inputs, such as the number of shelter toolkits provided. Finally, impact indicators measure the overall effect of the intervention. While not monitored as frequently, they must be measured against the initial baseline to assess changes, such as the percentage of beneficiary households satisfied or unsatisfied with their shelter solution.

00:17:29 The case management process in shelter operations

While shelter operations vary greatly, they share common case management steps across emergency, transitional, and permanent phases. The process typically begins with an assessment or mapping phase, such as a needs assessment or mapping local markets and landlords. Next is the registration and assessment of beneficiaries, usually at the household level to keep families together, gathering basic socio-economic information. Simultaneously, shelter monitoring takes place, which is particularly relevant in transitional and permanent phases. This involves monitoring the physical units, assessing construction needs, and securing landlord agreements. This dual focus on both the beneficiary and the physical property is a major difference between shelter operations and other types of case management, like protection. We then track the support provided to both landlords and beneficiaries, including financial assistance, NFIs, and lease agreements matching households to specific properties. Finally, the exit phase involves evaluating and officially handing over the project.

00:19:50 Moving from theory to implementation

To put this monitoring and evaluation framework and case management process into practice, a key component is the information management system. This system supports the entire data lifecycle, from production to consumption. Key components include collecting high-quality data, organizing it to maximize system reliability, processing the data for analysis, and ultimately using it to transform data into actionable information for decision-making across all organizational levels. An effective information management system must operationalize standard operating procedures (SOPs), handle data privacy and coordination, perform analysis, and disseminate results in a timely manner.

00:21:07 Case study: NRC shelter project in Lebanon

To illustrate information management system design, we will use a case study inspired by the Norwegian Refugee Council's Small Shelter Unit Habitation project in Lebanon. The main objective was to upgrade unfinished buildings to provide safe houses for refugees and vulnerable populations. The project involved legal agreements signed between landlords, beneficiaries, and implementing agencies. NRC financed the construction works undertaken by the property owners, and in return, the owners agreed to reduce the rent for the tenants. The project focused heavily on improving existing infrastructure to avoid legal complications with municipal authorities and prevent rent inflation, prioritizing quick repairs to roofs, walls, and WASH facilities before winter.

00:22:48 Case study monitoring and evaluation framework

The M&E framework for this case study operated on three levels. At the implementation level, they ensured technical compliance and quality assurance by having qualified site supervisors conduct 100% monitoring of construction sites at critical milestones, such as placing foundations. They used a Bill of Quantities approach to verify that materials and labor complied with technical and safety standards, validating every unit against minimum shelter and WASH standards before a family moved in. They also monitored legal formalization, with information counseling teams ensuring lease agreements were legally sound, protecting tenants from eviction and landlords from property issues.

At the operational results level, the project provided economic relief; the 12-month rent-free period allowed households to allocate scarce income toward other basic needs like food, healthcare, and education. It provided family assistance by housing thousands in safe conditions meeting international standards, and it created housing stock by turning unfinished units into habitable spaces.

Regarding long-term impact and outcomes, they measured the percentage increase in a family's ability to afford basic needs. They evaluated social cohesion by monitoring landlord satisfaction and property value increases resulting from the rehabilitation. Finally, they assessed the sustainability of tenure through follow-up surveys, finding that after the rent-free period, 40% of families remained in the units, with 60% of those successfully transitioning to formal paying tenants.

00:25:18 Designing the information management system

When adopting an information management system to support this framework, there are several main steps. First is the decision phase: determining why the system is needed and who will lead the initiative. Second, and most crucial, is collaboration: defining how teams will work together and assigning responsibilities. Third is data organization, or creating the data model. Finally, there is the launch and adoption phase, which involves building the necessary capacity to maintain the system. It is essential to choose a primary platform that meets your needs and is adaptable to change.

00:26:14 Decision and coordination

During the decision point, you must consider who the involved stakeholders are, what decisions need to be made collaboratively, what information is required for those decisions, and who needs to be consulted. Coordination is crucial for implementing any decision. You must establish the process and timeline, assign responsibilities for each step, and identify risks and mitigation measures.

Effective coordination involves defining specific roles, which often align with governance structures. These include a governance council that takes the lead, and teams handling data protection policies, data collection, and data cleaning. Data owners decide what data is needed and how the system is designed, while data custodians (often IT teams) handle the technical aspects. Proper coordination ensures unified reporting for both internal use and donors.

During implementation, there are two primary roles: data producers, who collect the data in the field, and data consumers, such as supervisors, who need timely data to make decisions. The system must have separate permissions for creating, updating, and deleting records, utilizing role-based access control and flexible permission levels to restrict access by forms, records, or specific fields.

00:28:37 Building the data model

The data model answers critical questions about what data is needed to inform decision-making and how it will be used. It clarifies data needs, how data is grouped into different datasets, the relationships between those datasets, and the data collection and review process. It also defines how data will be collected, who gets access, and whether the system needs to send collection links.

In our case study's data model, we have beneficiaries associated with several subsequent forms, such as vulnerability assessments, post-distribution monitoring surveys, referrals, and legal agreements. Because this is a shelter project, we also have forms associated with the shelter units themselves, tracking the type of construction needed, the Bill of Quantities, and matching specific beneficiaries to specific housing units.

00:30:07 Coordination and access control in practice

To facilitate coordination, we define how different roles interact with the system. For example, field officers act primarily as data producers, while supervisors act as data consumers. Field officers might have access to view, add, and edit records in the housing forms. However, for beneficiary forms, they may only need basic information to do their jobs; they cannot add records or view all sensitive information. Furthermore, they only have access to the specific records they are assigned to work on. Conversely, a supervisor has access to view, edit, and delete records across all forms, but only for the cases assigned to their specific team.

00:31:08 Exploring the system via live demonstration

Let's explore how this looks in ActivityInfo. The data collection forms are organized into different folders for easy navigation, such as case management forms and beneficiary forms. Within the system, you can view a beneficiary and easily see all their associated forms in a single table. This allows you to quickly check if a person has a completed vulnerability assessment or a post-distribution monitoring survey, and navigate to different forms depending on the current step of the case management process.

Simultaneously, we can monitor the housing units. The system connects which household is assigned to which unit. For a specific unit, you can view the assessments, the required construction, the quantities and rates of materials, and the total budget dedicated to those works. You can also view the lease agreements and see exactly which individuals are living in that unit. The flexibility of the data model allows us to track both beneficiaries and housing units and the relationships between them.

Shelter projects also heavily rely on visual monitoring, which frequently involves taking photos to assess damage or construction quality. The system allows you to easily store and preview these pictures alongside the qualitative assessment questions to understand exactly what is happening within a specific unit.

Looking at coordination from the perspective of a field officer, they can see the housing unit records assigned to them and edit them to correct mistakes, such as updating the construction site status. However, on the beneficiary registration side, their access is restricted. They can only see basic necessary information, like a first and last name or a phone number, while sensitive fields remain hidden. This targeted access facilitates coordination and protects data privacy in complex field realities.

00:36:40 Data use and reporting

The next component is data use. We must determine our data points so they can be effectively utilized. Data use occurs at different levels. A supervisor might need a dedicated view that pulls specific data points from various forms to monitor cases and organize fieldwork, such as seeing the number of referrals for a specific person alongside basic assessment details. Alternatively, project managers need access to aggregated reports to monitor project results, track status, and report to donors. The system can generate reports showing indicators from post-distribution monitoring surveys, such as safety and satisfaction disaggregated by gender, alongside monthly registration totals and qualitative conclusions.

00:38:46 Launch, adoption, and capacity building

The final phase is launch and adoption, where capacity building is crucial. When introducing a new information management system, you will encounter forces for change, such as the desire for a common system and time saved in collection and analysis. However, you will also face resistance due to past negative experiences, lack of resources, or limited time and staff.

To strengthen the forces for change, you need a strong team of allies and champions to create an internal user community. You must invest in capacity building with a clear plan, and visibly celebrate and share successes. To weaken resistance, you need a solid communication plan, multiple support channels for receiving and providing feedback, and clear definitions of who is responsible for what within the system design.

00:40:24 Training and system documentation

Capacity building is built around three components: manuals, training sessions, and system monitoring. When planning training, you must determine the necessary resources and support structures. Before a training session, create an outline or draft schedule identifying the objectives for each session, followed by a detailed facilitation guide mapping out supporting documents. During the session, tailor the content to align with learning needs. Afterward, collect feedback, evaluate the session, and recognize that training is an ongoing process requiring regular refreshers on specific or advanced topics.

A dedicated manual is also vital for coordination and adoption. It should specify the target audience, outline main sections (like a system overview, data collection instructions, and how to seek support), and provide specific details, such as explaining the data model, how to log in, and data protection tips.

00:42:34 Key takeaways

To summarize our key takeaways: Shelter is more than a physical roof; it is often a person's most important asset and highest living cost. Therefore, a robust monitoring and evaluation framework is particularly important for shelter case management systems, and this framework must adapt depending on whether the focus is emergency, transitional, or permanent shelter. Moving from theory to implementation requires building a strong information management system. Key steps include codifying collaboration terms, organizing the data model, and executing a capacity-building approach for launch and adoption. Ultimately, a successful system relies on the people using it, making dedicated training essential for every project and organization.

00:43:38 Q&A session

We will now move to the Q&A session.

The first question is: How do you bridge the gap between qualitative case management and the quantitative evidence and scale-centered questions? Qualitative case management includes caseworker notes and field observations that cannot be strictly quantified but are necessary to explain quantitative data. It is advisable to keep this qualitative data within the same information management system. However, a large part of the case management process can be quantified, such as demographic information, vulnerability assessments, and consumption scores using single-selection choices and scoring systems. Whether data is quantitative or qualitative, keeping it in a unified system allows you to better monitor data protection and access.

The second question is: When we develop indicators, is it advisable to develop indicators at the activity level? It is rarely advisable to develop indicators at the activity level. You need to be realistic about the number of indicators you can track; a rule of thumb is two or three indicators per results framework statement to avoid overwhelming your analysis. If you do track activity-level indicators, like the number of items distributed or registrations completed, keep them simple and ensure they directly feed into the next level, which is the output. For example, if you track the number of NFIs distributed, your output level should track how many people actually received those items.

The third question is: Is this approach recommended to all individual projects? The general monitoring and evaluation framework approach—moving from inputs and activities to outputs, outcomes, and impact—is applicable to all projects. However, the framework must always be tailored to the specific project's context, focus, and available resources, including budget, time, and personnel. In an emergency project, you might focus heavily on lower-level monitoring because speed and coverage are critical. In a permanent shelter project, long-term impact and evidence of sustainability are much more crucial. The core levels remain the same, but the focus shifts based on the project's reality.

The final question is: What goals or coordination models have worked effectively in improving information sharing and continuity of care between government and non-governmental shelter systems? Coordination between governmental and non-governmental systems requires a higher level of collaboration due to differing policies and politics. The core data governance roles and data protection principles I mentioned still apply. Depending on the situation, you might use a single shared information management system, or, if a government agency has strict rules preventing this, you may need two separate systems. In the latter case, you must find a way for the systems to integrate or provide cross-access to the respective teams. The universal rules of information management governance remain the same; it is simply the implementation method that differs.