Semester of Graduation

Spring 2020

Degree

Master of Science in Industrial Engineering (MSIE)

Department

Mechanical and Industrial Engineering

Document Type

Thesis

Abstract

Emergency Medical Services (EMS) require staff to recertify periodically, which involves a significant amount of continuing education. Great effort is given to designing and delivering a continuing education program, but it is also important to know if the training had all of the intended impacts. Nationally, there is no evidence-based system in place for training evaluation at EMS. This research proposes a framework being used at East Baton Rouge Parish Emergency Medical Services (EBRP-EMS) to evaluate a course, generate reports to indicate the performance of staff, and identify areas of improvement.

The Kirkpatrick Model, which is a 4-level evaluation, is used in this research. The evaluation at multiple levels enables this model to create a holistic picture of the impact of a training, incorporating different types of data as opposed to simply looking at post-training feedback. The National Competency Course (NCC) is selected for the pilot study for this research. Reaction (Level 1) of trainees is measured using questionnaires that ask trainees to rate the quality of course content and instruction. The level of learning (Level 2) is evaluated using written post-tests which ask questions related to the course content. Evaluation of behavior (Level 3) and results (Level 4) are measured through Key Performance Indicators (KPIs) identified by stakeholders. These KPIs are indicative of critical behavior of Paramedics in the field and the targeted outcomes of the organization. An example of a KPI is “was End Tidal Carbon dioxide measured for cases of endotracheal intubation?”. This is recorded for individual Paramedics at Level 3. However, at Level 4, the same KPI is measured for the entire organization. The measurements are done using the data system in place for patient care reporting.

Data was collected for Paramedics that took the National Competency training and Clinical Guidelines exams during 2019. A stakeholder report is created to display the results found from this research. Descriptive statistics and Wilcoxon Signed-Rank tests are utilized for Levels 1(n=41, 38, 41, and 17), and 2(n=35 and 58). Test of proportions is utilized for Level 3 and 4. At Level 3, 25 KPIs are measured and tracked for 19 Paramedics, who took the NCC class in February 2019, over a period of 32 months. Additionally, at Level 4, 15 KPIs are measured and tracked for the entire agency over the same time period. Results from the statistical analysis indicate areas of improvement at each level.

The successful application of this framework has generated avenues of improvement for the training program at EBRP-EMS and has also created a link between the content taught in the NCC class to patient care in the field and the overall higher-level goals of the organization. Validation of this framework provides an evidence-based pathway for its use to other courses and potential for its national adoption for all Emergency Medical Services continuing education evaluations.

Committee Chair

Nahmens, Isabelina

DOI

10.31390/gradschool_theses.5131

Download

Share

COinS