Differentiating Virtual Education to Optimize Outcomes

Abstract: Prehospital medicine/patient care is a newer profession still being defined and, in many ways, redefining its role in today’s healthcare system. Prehospital medical/EMS education is growing and developing into a more robust and crucial component linked to high-quality patient care. Much of the current continuing education (CE) content tends to flow from an auditory/lecture delivery methodology. It is well known in the prehospital industry that EMS CE classes have become known as a “boring” or “tedious” task that needs to be completed to check the box of the various accrediting agencies. To date, little thought given to brain neurochemistry, introductory educational strategies, or foundational learning style variation between students.

This document summarizes an initial analysis of learning effectiveness comparing the results of a new and improved learning approach developed by Axene Continuing Education, LLC with the results of the representative current approach in the market.  This analysis has been completed by clinical and actuarial consulting experts from Axene Health Partners, LLC of which Axene Continuing Education, LLC is a wholly owned subsidiary.

Hypothesis: Comprehension and retention of medical content will improve with an engaging content delivery methodology and attention to student learning styles.

Methodology: Medical content to be delivered for this study will be selected from a content area few firefighters/paramedics/EMTs are familiar with.  If, for example, cardiac medication content was selected; there would be many students who could answer assessment questions correctly without participating in the content delivery.  The topic of toxins and poisons was selected for its applicable content and ease of retrieval of related less familiar content.

Arm #1 of this study will deliver content in an auditory style with visual slides of content. There will be a few pictures for reference but nothing in addition to the baseline “most common” teaching method used today.  For Arm #1 we delivered content using the “old” or “current” method of delivery over three days with students from Station 1.

Arm #2 of this study will deliver content in a more engaging podcast style with regular use of visual aids, comedy, acting, and performance as well as illustrations using a light board and demonstrations on stage by the presenter.  For Arm #2 we delivered content using the “new” method[1] to a second similarly sized group of students from Station 2.

The initial training focusing on comprehension of the material was completed for both groups in February 2023.  A follow-up was completed seven months later which focused on retention of the material previously presented.  In addition, we established a control group of students who had not participated in either of the two classes.  This control group demonstrates potential scores for students perhaps guessing for the correct answer.

All groups took an identical 10-question test covering the common content presented by the two different approaches.

Results:  A comparison of comprehension from Arm #1 and Arm #2 is shown in Table 1.  The average score for Arm #1 was 7.04 out of a maximum of 10.  The average score for Arm #2 was 9.49 out of a maximum of 10.

Table 1. Comprehension Results

The Arm #2 method resulted in higher scores.  The average scores from Arm #2 were about 95% compared to 70% using the Arm #1 methodology, an average 35% improvement.  The similar results by day, each with separate students, showed a range of 20% to 50% higher scores for the Arm #2 methodology demonstrating substantially higher scores with the Arm #2 learning approach.  The standard error for the mean for this analysis was +/- 0.17, a negligible amount.

The results for the knowledge retention part of this analysis are presented in Table 2.  The average score for the Arm #1 students, seven months after taking the course, was 4.42 out of a maximum of 10.  Similarly, the average score for Arm #2 students was 6.12 out of the maximum of 10.

Table 2: Retention Results

Not only did the Arm #2 methodology have 35% greater comprehension scores, as shown earlier, but it also had 38% higher retention scores seven months after initial instruction.

The Control Group, those not taking either of the courses, achieved an average score of 3.00 or 30% correct answers, slightly greater than expected if purely relying upon the chance to guess the right answer.

Alternative comparison approaches show even higher effectiveness than demonstrated above:

1. Excess over control group results:
   1. Arm #1 excess:
      1. Comprehension: 7.04 – 3.00 = 4.04
      2. Retention: 4.42 – 3.00 = 1.42
   2. Arm #2 excess:
      1. Comprehension: 9.49 – 3.00 = 6.49
      2. Retention: 6.12 – 3.00 = 3.12
   3. Comparison (Arm #2 net improvement)
      1. Comprehension: (6.49 – 4.04)/4.04 = 61% higher
      2. Retention: (3.12 – 1.42)/1.42 = 120% higher
      3. Arm #2 is substantially more effective on both comprehension and retention.

2. Retention Effectiveness (ratio of retention score to initial score):
   1. Arm #1 ratio:
      1. Total score: 4.42 / 7.04 = 63%
      2. Excess over control: 1.42 / 4.04 = 35%
   2. Arm #2 ratio:
      1. Total score: 6.12/ 9.49 = 64%
      2. Excess over control: 3.12 / 6.49 = 48%
   3. Both approaches have comparable retention effectiveness based on total score, however, Arm #2 has a much higher comprehension of the material resulting in higher scores for the student seven months after initial instruction.
   4. Results based upon excess over the control group show higher retention effectiveness for Arm #2 (48% vs. 35%), a 37% improvement.

Discussion:  The above results suggest that engaging the learner with a more stimulating content delivery methodology increased both comprehension rates and retention of this knowledge, a not surprising result. The retention result was the most impressive finding in this study as the Arm #2 students scored 120% higher than Arm #1 students.

Conclusion: The delivery method of EMS continuing education content appears to have a dramatic impact on both learning and retention. This pilot study confirmed our hypothesis and demonstrated greater retention than expected.  Future studies connecting EMS content delivery with patient care and patient outcomes would help stakeholders see the return on their investment in improving their existing EMS education system.  Axene Continuing Education, LLC, and their creative approach to educating this population is achieving very positive results.

For more information regarding this analysis, please contact Axene Continuing Education, LLC at ace@axenece.com or (469) 923-2836.

About Axene Health Partners, LLC (AHP)

AHP is an independent actuarial and clinical healthcare consulting firm, established in 2003 and wholly owned and operated by its Partners.  AHP provides healthcare consulting services to a wide spectrum of healthcare companies including health plans, health insurance companies, healthcare providers, brokers, employee benefit plan sponsors, healthcare innovators, and various governments and regulatory agencies.  AHP has offices throughout the United States with its corporate headquarters located in Texas. For more information, please see www.axenehp.com.

About Axene Continuing Education, LLC (ACE)

ACE is a wholly owned subsidiary of AHP based in Texas committed to providing high-quality, practical educational experiences that support the learner in their commitment to provide the best possible healthcare to their patients.  Initially, these efforts have been focused on firefighters, EMS, and EMT professionals. For more information, please see www.axenece.com.

Endnotes

[1] The “new” method uses the teaching methodology and approach adopted by Axene Continuing Education, LLC (ACE) in its curriculum (See www.axenece.com).