Delays in treatment initiation among people with TB is a key challenge in Brazil and this may be driven by a limited recognition of TB symptoms leading to delays in diagnosis by health care workers. This example is based on a study conducted to evaluate a new eLearning course designed to improve the capacity of nurses to identify, test and treat TB.
To evaluate the effectiveness and acceptability of the distance learning course and to assess barriers and facilitators to implementation by answering the following questions:
- How did the level of TB knowledge and TB among nurses following the completion of the course compare to their pre-course knowledge?
- What was the level of satisfaction with the course among nurses and what factors were associated with reported satisfaction?
There were approximately 48 740 enrolments during the three-year implementation period, representing about 50% of the overall target. The majority of enrolments were among women (80%), and those aged between 21 and 30 years (43%).
Areas for further investigation: What factors meant that women were more likely than men to enrol for the eLearning course? What additional strategies are required to enhance reach among men? What additional strategies are required to enhance reach among those aged over 30 years old? What are the differences in characteristics, sociodemographic profile and other important variables among those that did/not enrol?
Post-course evaluations showed that more than 80% of users agreed or strongly agreed that the course content was useful in their daily work and that they felt more equipped for their work activities. The majority of users (93%) rated that the overall quality of the course – such as the appropriateness of course context to their expectations, course design, navigability in the course environment, layout of course environment, media resources available etc. – as "good" or "very good". Nearly 93% of users reported that they are "likely" or "highly likely" to recommend the module on Clinical management of people with TB/HIV to others.
Uptake of the eLearning platform was observed in 60% of Brazil's 5570 municipalities. The UNA-SUS online platform was the main dissemination strategy for the eLearning platform; as a result, 41% of all users were from primary health facilities of UNA-SUS.
Areas for further investigation: What are the differences between settings that did and did not adopt the eLearning platform? What strategies are needed to enhance adoption among health care workers outside of the UNA-SUS health facility system?
32% of enrolled participants completed the eLearning course. A higher proportion of nurses and physicians did not complete the course when compared with other professions. Browsing difficulties were reported by 7% and technical difficulties with computer, mobile phone, tablet or internet access was reported by 8%. The average cost per participant for the eLearning course was approximately US$5 per course completion, compared to US$ 45 for traditional, face-to-face trainings.
Areas for further investigation: What proportion of target settings and intended beneficiaries continue to deliver and use the eLearning platform after the initial roll out? How do costs and infrastructure requirements (i.e. task-shifting, time commitments) associated with continued use of the eLearning platform vary over time (troubleshooting, maintenance, licensing, upgrades, additional features/developer time)? What components of the technology are sustained, discontinued or modified after initial roll out – and why?
This study used a quasi-experimental, before and after approach to evaluate the effectiveness of the eLearning platform on the participant's learning level. In addition, a cross-sectional study was conducted at the completion of the eLearning course to evaluate the level of participant satisfaction.
This study explored effectiveness, which was assessed through the comparison of learners' baseline knowledge (pre-test) and knowledge upon completion of the learning modules (post-test). User satisfaction was also assessed, defined as the reported level of satisfaction with various components of the course.
Nurses actively involved in inpatient or outpatient care of patients in Porto Alegre, Brazil were invited to complete the eLearning course and participate in the study.
This study was conducted in the city of Porto Alegre, which is a metropolitan area with 31 municipalities and approximately 3.7 million inhabitants.
knowledge about TB and clinical practice. The course comprised four modules of about 30 minutes each: TB concepts and epidemiology; development of TB; detection of TB cases; and TB transmission and biosafety basics. The modules were in the form of written text on slide show presentation.
The course could be taken at any time during a four-week period following registration. A course coordinator was available by email to address any questions about the course and a certificate was provided to each participant upon completion of the course.
- The course was developed within the Moodle platform (an acronym for Modular Object-Oriented Dynamic Learning Environment), a free learning–support software designed to support activities mediated by information and communication technologies by allowing the integration of multiple mediums, languages and resources to present information in an organized way, to develop interactions between people and objects of knowledge.
Data collection, management and analysis
Sampling and recruitment
Nurses meeting the eligibility criteria were purposively sampled. The sample size was calculated based on an anticipated difference between pre- and post-test scores of 12.1% (based on the findings a similar evaluation of an infectious disease-focused eLearning course), an estimated standard deviation of 2.5, a beta error of 0.20 set to 95% confidence intervals, resulting in a final sample size of 51.
Data collection methods
Pre- and post-tests that evaluated user level of knowledge were embedded at the beginning and end of each of the four modules and self-completed by the user. The user was able to score a possible 15 points for both the pre- and post-tests.
To assess user satisfaction, an electronic, structured questionnaire was embedded into the course and completed by users at the end of the fourth module. The questionnaire used Likert-type scale (totally dissatisfied, dissatisfied, satisfied, and totally satisfied) to assess course design, methodology, content, assessments, interaction with course coordination, and use of distance learning method.
Data management and analysis
The Wilcoxon test was used to assess differences between individual pre- and post-test results. A value of p < 0.05 was taken to indicate a statistically significant difference in the level of user knowledge.
Descriptive statistics were used to present data on user satisfaction. All analysis was conducted with SPSS.