Preparing for implementation research

Engaging relevant stakeholders

Implementation research is a needs-driven process and research questions must reflect real-life challenges and needs related to the implementation of a particular intervention. As such, it is important that relevant stakeholders who are or will be connected to the digital intervention are identified early in the IR process in order to benefit from their insights, experiences and knowledge. This will enhance your team's understanding of the implementation challenges and their potential causes and help you to develop research questions that can adequately address these issues.

A stakeholder analysis may be a useful exercise to help properly identify potential stakeholders who should be consulted and involved in the development of your IR study (Box 1).

Understanding the implementation challenge

This section is about identifying and understanding the key challenges that constrain the implementation of your digital technology intervention, which will be further explored in your IR study. The series of steps that follow should be led by the core IR team in collaboration with the identified stakeholders.

Reviewing the intervention purpose and implementation goal

The first step in identifying current or potential implementation challenges is to clarify the overall goal of the intervention and its implementation: What is the purpose of the digital intervention? What are the implementation goals you are trying to achieve? Consider whether the digital technology is intended to add impact to an existing, proven intervention (e.g. digital adherence technologies to support TB adherence), or is it designed to independently trigger a health outcome (e.g. do connected diagnostic platforms improve the quality and timeliness of services provided by TB care providers?). It is important that all stakeholders and relevant parties are in agreement about the overall goals and have the same expectations of the study. With the goals clearly defined, it will be easier to identify the current or potential challenges. Here are some examples:

Appraising current implementation within the local context

Once the team has clarified the overall goals of the intervention and the implementation, this should be compared to the current implementation status or process in order to identify potential challenges or gaps and their causes. For teams who are already implementing a digital intervention, you may already have an idea about where and why you are facing challenges with implementation, however it is important that this review is done in a systematic manner with the involvement of relevant stakeholders. For teams who are considering implementing a new digital technology intervention, this step will help you consider potential challenges you may face when it comes to implementing the new technology, and to identify the best strategy for implementation.

The RE-AIM framework

Adopting a systematic approach to thinking about implementation can be aided by the use of a theoretical framework. While there are many different frameworks available, in this toolkit we have chosen to use the RE-AIM framework² – one of the most widely adopted frameworks – to evaluate potential implementation challenges to digital interventions.

RE-AIM stands for Reach, Effectiveness, Adoption, Implementation and Maintenance and was originally developed to evaluate the potential or actual impact of public health interventions. It does so by assessing traditional health outcomes (i.e., effectiveness) alongside important implementation outcomes that provide context for an intervention. Table 2 presents a definition of each domain and how it applies to digital interventions; read through the table and reflect on how this relates to the digital intervention being implemented or considered.

Assessing your current or planned implementation in light of the five RE-AIM domains can help pinpoint specific challenges that need to be addressed, and these challenges may become the basis of your IR study. The following case studies illustrate how the applying the RE-AIM framework can assist in the identification of potential implementation challenges and areas for further investigation.

When considering potential implementation challenges, it is imperative to consider how local and contextual factors may influence your ability to meet your implementation goals, and the impact this has on the overall effectiveness of your digital technology intervention. Such factors play an important role in how a digital technology intervention is rolled out, used and accessed, maintained, scaled up and integrated into policies and practices.

IR takes place in 'real-world' settings and attempts to support better understanding of contextual factors neglected by or unexamined in more traditional forms of clinical research. These settings are influenced by factors such as the physical, socioeconomic, geographical and cultural environments, and access to health systems. For digital health, these factors can include the maturity of the existing in-country information and communication technology (ICT) infrastructure, technological literacy, attitudes and policies related to the use of digital health interventions, workforce capacity, and the alignment or appropriateness of digital technology within the existing digital ecosystem.

The local context also includes varying understanding of the need for behaviour change and transition to new practices, which often accompanies the introduction of innovative tools and new approaches, such as changing from an entrenched paper-based records system to an electronic recording system. A digital intervention will succeed only if it is taken up by users, adds value to existing TB services, and facilitates the desired outcome. As such, implementers must be aware of stakeholder motivations and interests, as well as potential barriers and resistance to the disruption of the status quo that may affect the fidelity of deployment, understanding that this may temper the possible benefits of new digital health interventions.

Digital technology solutions are often regarded as complex interventions due to the various components required to create change. It is not enough to simply introduce a new digital intervention without any regard for the broader context in which you are working. Understanding how or why an intervention may be successfully implemented requires an understanding of the contextual factors that may shape, influence, or create potential implementation challenges.

Some potential factors that may shape or influence the implementation of a digital intervention are shown in Fig. 3.

Reviewing the existing literature

Once you have a better understanding of some of the implementation challenges and potential contributing factors as they relate to your specific context and digital intervention, the next step is to review the literature to assist in the development and articulation of a specific research question that can be addressed by an IR study. For example, the literature may provide evidence in support of a particular type of digital intervention, may further describe or explain the magnitude of a specific challenge, contextualise known implementation challenges, and/or identify solutions previously or currently implemented by others. The literature review should look at digital health interventions with evidence of feasibility and usability in their intended setting, as well as established efficacy under study conditions. Special consideration should be given to interventions deployed in settings that closely resemble those that you are interested in. The following are some possible resources for literature:

Peer-reviewed, published literature

• PubMed is a free, full-text online search engine for biomedical and life science literature. Guidance on how to perform a comprehensive literature search using PubMed can be found here.
• Web of Science and Scopus are both subscription-based databases for peer-reviewed literature in the social and health sciences.
• Google scholar is a search engine for scholarly literature across all disciplines. Google scholar can provide access to peer-reviewed and grey and/or unpublished literature and provides greater access to full text articles compared to other subscription-based databases.

Grey literature

'Grey literature' refers to information that has not been peer-reviewed. This includes:

• Programmatic and technical documents that provide current information on the state of the TB epidemic and response in your country, such as national strategic plans for TB control, government or WHO-developed estimates for TB prevalence and incidence, national TB programme (NTP) data on case detection, TB treatment outcomes and/or patient sociodemographic characteristics, or epidemiological and programmatic review reports. This data may also highlight differences or variation in terms of TB epidemiology, coverage of services, patient characteristics, etc. among different sub-populations or regions. Experiences from pilot studies may only be reported in internal documents and could hold valuable lessons that point to relevant problems.
• Routine NTP monitoring data can provide a good indication of the overall performance of an intervention or programme and be a useful way to identify implementation constraints, or areas of poor performance.
• Conference archives may also be a good source of information to learn about innovative uses of digital technology for TB, such as The Union Conference.

Building an IR team

By this stage, you should have:

• identified the key implementation challenges;
• reflected on the local context and identified the various contextual factors that may be influencing or contributing to the implementation challenges;
• identified and consulted with relevant stakeholders to better understand the implementation challenges, develop some hypotheses about potential causes and contributing factors and gather additional information, and;
• reviewed the existing literature to learn more about implementation challenge including selecting some strategies to address it or understand it better.

Turning this information into a sound research objective that can be investigated by an IR study requires a suitably skilled research team. Building a research team is an iterative process. While you may already have a small core research team assembled, your team may need to expand as your IR study develops and additional needs and capacities become clear. For example, depending on your research objective and chosen study design, you may require qualitative researchers, health economics, statisticians etc., on your team.

In general, research teams are made up of the following people:

• Principal investigator/lead researcher.
• Co-investigators/researchers.
• Research coordinator/project manager.
• Data manager.
• Data analyst.
• Research assistants/data collectors.

Illustrative job descriptions for these key roles are available in the Annex. These can be adapted to suit your specific needs and can be used to help with recruitment of additional research team members if needed. Any new recruits will need to be accounted for in the budget, as discussed further in Module 5.

In addition to the research team, you may wish to involve some key stakeholders to provide oversight or represent the communities you will be working with (refer back to your stakeholder analysis). While the research team will be in the driving seat for the research, the other stakeholders can act as a steering committee and help provide guidance, help with decision-making and act as a conduit to other stakeholders relevant to your study.

Fig 4. provides some examples of individuals who could provide oversight to your research project. Perhaps you have already identified some of these individuals through the stakeholder analysis discussed earlier in this module?

Developing an IR proposal

The remainder of this toolkit is focused on turning your implementation challenge into a feasible research objective that will be developed into an IR study. By completing the following modules, you will be guided through the development of an IR proposal that can be used to support fundraising efforts by your team. Fig. 5 shows the various proposal components that will be addressed in this toolkit. We have developed a Generic proposal template that can be used by your team to develop the study proposal.

As prospective donors or funders may have specific requirements or timelines that will need to be addressed in the IR proposal, it is preferable that IR team members identify potential funding opportunities at the beginning of the IR process. Donors may also have specific requirements about the layout or contents of an IR proposal. In such cases, it is recommended that you follow the donor requirements instead of using the generic template included in this toolkit.

The team should also explore opportunities for domestic funding provided by government agencies or national NGOs that can be used to support the IR project. There are also various external funders who may provide support for research projects such as multilateral organizations, bilateral donors and NGOs, private foundations and trusts.