Selecting the research methods for your IR project

Implementation research can employ both quantitative and qualitative methods. However, researchers need to be aware that each approach has its own strengths and limitations. Table 4 summarizes the strengths and limitations of quantitative and qualitative methods. In general, the strengths of one method can be seen as the weakness of the other. Therefore, combining both quantitative and qualitative methods can improve the strength of an IR project.

Quantitative and qualitative studies have fundamentally different criteria to assess the rigor of the study due to the paradigm used and the nature of the methods. The criteria are analogous but not interchangeable. Each has its own appropriate and no less rigorous standards. There are four analogous criteria that are comparable to assess the quality of both quantitative and qualitative studies i.e. truth-value, applicability, consistency and neutrality.⁹

The quality of study depends on how effective the researcher is in measuring the concept being studied. In a quantitative method, this means the extent to which the measurement reaches the concept it intends to measure. Validity assumes correct operational measures for the concepts being studied. The validity of the study can be improved by ensuring that there are no selection and measurement biases (through the use of standardized instruments and procedures).

Credibility is the corresponding criterion of validity in qualitative research. It focuses on whether the investigator has achieved in seeing the truth according to the informants’ eyes and in understanding of the context in which the research is conducted. Credibility can be accomplished by triangulation of informants, data collection methods or analysis; prolonged engagement with people; continual observation in the field; the utilization of peer researchers; researcher reflexivity; and participant checks, validation or co-analysis.

The strength of qualitative research lies in validity (closeness to the truth). Good qualitative research, using a selection of data collection methods, should touch the core of what is going on rather than just skimming the surface.¹⁰

Applicability refers to how we can apply the research findings to a wider population, beyond that under study. In a quantitative study, it is known as the external validity or generalizability of the findings. Generalizability is a goal of quantitative studies. It is accomplished by selecting large enough random samples to minimize the probability of error and to be able to statistically represent the population from which the samples are drawn.

Transferability is the qualitative analogue to the concept of generalizability. Transferability is the degree to which the results of a study can be applied to other contexts and settings or other groups. It also means the level to which the audience will be able to generalize the study findings into his or her own context. The audience that requires to transfer the findings into different situations are responsible to assess the transferability of the study, rather than the researchers of the original study. Transferability can be achieved when the researcher gives adequate information about researchers’ backgrounds, any prior knowledge and possible bias, as well as the research context, processes, members and researcher-participant connections so that the reader can decide to what extent the findings of the study is transferable to their own contexts.

Consistency refers to whether the study conclusions would be similar if replicated with the same subject matter or in a similar context at a different time. In a quantitative study consistency refers to the reliability of measurement. When we measure variables under study, all measurements involve some degree of error. When the amount of error is low, the reliability of the measurement is high.

Consistency is defined as dependability in qualitative research. Dependability refers to the way researchers ensure that the study is conducted consistently across time, researchers and analysis techniques and that the procedures of the study are explicit and repeatable. This can be achieved by an audit trail, which is a process of keeping a detailed chronology and description of the research activities, including: an explanation of the choices and justification for the different research designs, data collection and analysis, emerging themes, and an analytic memo.

In summary, ‘reliability’ in a quantitative study is the repeatability and independence of findings from the specific researchers generating those findings. While in qualitative research, reliability implies that given the data collected, the results are dependable and consistent.

Neutrality implies that the researcher maintains objectivity, minimizing any possible bias due to the researcher’s values or interests. In a quantitative study, objectivity can be achieved by avoiding selection bias (by randomization) and measurement bias (by standardized instruments, procedures and masking participants’ status during measurement). In a qualitative study, however, these same strategies would be counterproductive. To be able to capture reality as accurately as possible according to participants’ perspectives and experiences, the researcher needs to be inseparable from the study participants. Furthermore, the researcher can act as an instrument during data collection. As a result, the researcher cannot be fully objective. Objectivity (or conformability) is therefore a way of knowing that the researchers have maintained the distinction between their own personal values and those of the study participants. The readers should be able to see that the integrity of the study findings is based on the data, and not the researchers’ beliefs or biases. Conformity can be achieved through the use of a reflexive diary.