Developing Monitoring Questions

The development and implementation of the monitoring plan requires sufficient human and financial resources, as well as information management systems. It is recommended that you assess available resources for the coordination of activities, data collection, quality management, analysis and dissemination of information before commencing of any monitoring activities. However, given typical resource constraints, it is generally wise to take advantage of readily available resources for M&E such as indicator guides, M&E materials and communication tools rather than developing new ones to implement the project monitoring plan.

One of the essential steps in developing a monitoring plan is to translate research objectives into variables that can be readily and objectively measured. These should be defined prior to the commencement of the project implementation and comprise a blend of those that focus both on processes and outcomes. They should be based on the research question and objectives of the project and their rationale should be based on the logic model and information needs of decision-makers. The indicators should be relevant, accurate, feasible, distinctive, useful, and consistent with international/national standards. Selection of suitable indicators is iterative and participatory, and should involve relevant stakeholders. It is helpful to develop an indicator matrix, summarizing the indicators in the monitoring plan. Table 8 describes, data sources for the indicators at various levels of the logic model.

Target setting is a critical part of M&E planning. In order to determine variance (the percentage of target reached), it is necessary to not only measure the indicator but pre-determine a target for that indicator. Targets should be set in consultation with all stakeholders so that everyone understands what the project has committed to achieve. By setting targets, you will have a concrete measure by which to judge whether the project is progressing as expected or whether it may be essential to adjust the implementation or timeframe. Targets should be realistic, but they should also be challenging enough to encourage staff and stakeholders to think about the potential achievements within the project life cycle. Factors for consideration when setting targets include: Baseline levels; past trends; expert opinions; research findings; what has been achieved elsewhere; client expectations; the capacity and logistics to achieve targets. The targets set at the time of protocol development – which may have been based on secondary data information – may be refined after baseline values are collected. Furthermore, the targets may continue to change during the implementation, due to external influences beyond the researchers’ control. In all cases, any modifications to targets should be communicated to stakeholders and any changes made should follow proper procedures and approval.

In order to make evidence-based decisions, decision-makers require information from various sources. Despite many potential sources of data for monitoring, these may not be sufficiently comprehensive or appropriate to inform an IR project, particularly given contextual considerations. The data may also be collected from several different levels within the health system, depending upon the specific objectives of the project. Data existing sources and data collection tools might include: Service statistics; administrative or programme records; geographical information systems; facility assessments; qualitative interviews; observations; and questionnaires/surveys.

In general terms, the monitoring of the implementation process should be relatively quick and simple, with larger and more costly data collection reserved for measuring outcomes or impact. The power of qualitative data should also not be overlooked. In establishing the primary targets’ perceptions and experiences from the project, success stories, key lessons and experiences from stakeholders, photographs can be valuable complements to facts and figures, filling data gaps and providing insight and understanding into the statistics. Generally, monitoring the process may require using different data sources other than what is often used for monitoring project outcomes (See Table 8).

The frequency of data collection should be sufficient to support management decisions, but not so frequent to over-burden team members. Furthermore, the same data sources should be used to measure indicators throughout the monitoring cycle.

Examples of data collection for project monitoring purposes include:

• Review of routinely collected data (e.g. HMIS) (example number of malaria treatments among children under the age of five).
• New data collected to monitor the project implementation (e.g. interviews with health workers involved in a project to provide counselling to mothers with sick children under the age of 5).
• Review of data collected specifically for the IR project (e.g. focus group discussions with traditional healers in malaria treatment and referred to health centres for children under the age of 5).

A monitoring system should be able to link data collection, its analysis and usage. It must also systematically and reliably store, manage and access the M&E data. Thus, the monitoring plan should have a detailed data analysis component indicating how the results will be analysed and presented. This procedure requires critical review of the resources for data analysis and storage. For effective decision-making, data management should be timely, secure, and in a format that is practical and user-friendly.