TheraSim - The Science - Dynamic immersion simulation for healthcare professionals

IMSH 2010 ABSTRACT


An Expert Systems-based Virtual Patient Simulation System for Assessing and Mentoring Clinician Decision Making: Acceptance, Reach and Outcomes
David D. Hadden, TheraSim, Inc., Durham, NC.

Introduction: Traditional clinical training methods are expensive and non-standardized, remove clinicians from the practice setting, and impact is difficult to measure. We report on user performance with an interactive web-based simulation and data analysis program in which practitioners have managed hundreds of virtual patients with vast arrays of medical conditions.

Methods: Using an interactive virtual medical records interface, clinicians receive electronic mentoring and testing (dual mode) by reviewing histories, ordering tests, making diagnoses among hundreds of choices, and choosing treatments from more than 1,000 medications and other therapies. The simulations can allow or hide in-session diagnostic and therapeutic information which is produced by an expert system-based artificial intelligence engine (A.I.). The A.I. provides guidelines and evidence-based feedback on the appropriateness of choices. Finally, the simulation shows an explanation of reasonable choices for the case, a mini-review of the general topic, and the user’s errors, warnings and deviations from guideline, evidence and expert consensus-driven recommendations.  All choices are recorded for analysis. This paper summarizes 5-year results from 422 cases from 81 CME programs involving 41 medical conditions and appearing in a variety of internet and hospital venues.

Results:
Usage—122,990 registered users representing >200 countries have attempted 402,508 sessions with a completion rate of 49. Of the approximately 5 million page views, the average user viewed 71 pages (31 pages/session) and spent 18 minutes/case. Errors--The average score was 60 points (of 100), and 66% scored <80 points. Fewer errors occurred with testing, while there was an increased number of therapy-related errors made by users failing to diagnose appropriately. Outcomes—Using an analysis of a neurology program involving 1,946 users and 2,642 sessions, all clinical guidance was turned off for >100 sessions in each of 3 patient simulation cases. Success in making a difficult diagnosis increased from 12% to 36% with guidance operative, an incorrect diagnosis was avoided in 74% with guidance vs. 48% without, and appropriate treatment was more likely with guidance turned on: 74% vs. 44%, 67% vs. 52%, and 42% vs. 23%. User satisfaction remains positive for most users, including average scores of 4.2 of 5.0 using various questionnaires.  In 5 HIV training simulation deployments during 2006-2007 in 3 African countries using WHO guidelines and involving 2,780 pre-/post-test simulations, 241 clinicians passed 71% of pre-tests. After clinical feedback was activated, scores increased by 35 points, resulting in a final pass rate of 93% (p<0.001 vs. pre-). Similar improvements were noted in 3 separate programs at 80 hospital sites in 2008 and 2009, the latter utilizing a competency-based model, which eliminated the need for formal post-testing.

Discussion/Conclusions: Expert systems-based virtual patient simulation shows promise as a mechanism for assessing practitioner skill, detecting skill gaps and for electronic mentoring.  These systems have the ability to extend the patient simulation process into chronic and infectious disease states, an area that has been primarily overlooked by mannequin-based simulators.

IAS 2009
Feb 26, 2009

Title: Addressing Advanced ART Learning Needs in Ethiopia through Virtual Clinical Case Simulations

Authors: Marion MCNABB, MPH 1; Meg DOHERTY, MD PhD,1,2; David HADDEN 3; Wondwossen AMOGNE, MD1,4, Berhanu GUDETTA, MD 1; John G. BARTLETT, MD 2

Affiliations:

Location of Program: Ethiopia
Key Words: e-learning; computer based virtual learning; continuous medical education

Implementation Area: HIV/AIDS Care and Treatment - Training

Abstract:

Background
Continuing education opportunities for Ethiopian HIV/AIDS clinicians are not widespread despite noted advanced HIV learning and national guideline updates needs. Johns Hopkins University (JHU) and private sector partner TheraSim, Inc. collaborated to implement self-directed learning opportunities using ART clinical case simulations installed on computers in ART sites in four regions of Ethiopia.

Analysis Design and Methods
TheraSim and JHU developed a &lsquo;Basic HIV for Physicians&rsquo; training program with 20 pre/post test clinical case simulations addressing 35 competencies. TheraSim was implemented in 45 ART sites from Dec 2006 to May 2008 and evaluated June 2008. TheraSim data was collected through site visits, electronically compiled and clinical metrics analysis reports prepared. Clinical metrics were reviewed and urgent learning needs identified.

Results
JHU provided Ethiopia based clinical leadership and TheraSim offered simulation expertise throughout the partnership. From Dec 06-May 08, 289 users metrics (WHO guidelines HIV physician track) revealed gaps in: awareness of new national HIV guidelines/treatment options, initiating 2nd line therapy, and treatment failure. Program responses included: adapted HIV physician track for Ethiopia, developed &lsquo;Advanced HIV&rsquo; track (cases address 2nd line therapy

4th International Conference of Evidence-Based Health Care Teachers & Developers, Taormina, Italy, October 2007.

Implementing E-Mentoring Training and Clinical Performance Assessments Using Evidence Based Best Practices to Improve Quality and Safety of Patient Care Through Clinical Simulations As Part of a Critical Appraisal Skills Program (CASP)

Abstract:

Background: Adoption of the practices and processes inherent in the outcomes of applying clinical, evidence based health care has always been a recognized problem. The collection, aggregation and timely dissemination of information which directly effects diagnoses and clinical treatment has traditionally been dependent on clinicians&prime; access to journals and on-line resources - many of which are neither comprehensive nor timely. Simulation - historically confined to hands-on procedures on dummies - has now expanded into demonstrating real-time best practices in diagnostics and treatment through computerized case modeling and testing algorithms.

AIMS: Implementing an on-line clinical teaching tool that is configured to update a best practices "rules engine" (e.g. new contraindications, new laboratory tests, latest FDA warnings, latest clinical trial citations) across a spectrum of diseases and conditions has recently been proven to be effective in quantifying the behavior of clinicians and healthcare teams in the classroom and in the field. The aim is: reach more, teach more, save more.

Methods: Simulation of the clinical experience begins as a virtual "patient" presents the clinician with his or her "case" - an actual patient case that has been translated into a clinical, computerized encounter that tests the "student(s)" as they move through the logical process of assessment, ordering and interpreting labs, diagnosing, and prescribing meds - all of which is displayed in an intuitive EMR that includes clinical images - along with real-time mentoring feedback. As each clinician moves through cases specifically selected to improve skills in certain critical areas of expertise, the simulator records their behavior and scores performance based on weighted metrics. Deficiencies are presented along with specific references to current evidence-based best practices.

Results and Conclusions: Over 30,000 clinicians in 122 countries have demonstrably improved their skills by incorporating evidence based health care into their quality of care "curriculum" using computerized clinical case simulations and clinical performance management reporting.

Author Block: Coleman, C.A.(1); Bartlett, J.G.(2); Hadden, D.D.(3);
1) SAS Institute, Evidence Based Health Care Initiative, www.sas.com
2) Johns Hopkins School of Public Health/ Epidemiology, www.hopkins-aids.edu
3) CEO and Founder, TheraSim, www.therasim.com

A Novel Clinical Performance Management System: HIV, HBV, HCV and Adolescent Vaccines

Abstract:

Background : Traditional clinical training methods are expensive, take physicians out of the practice setting, and the impact is difficult to measure. We report on physician performance using an interactive computer-based simulation and data analysis program for practitioners to manage virtual HIV, HBV, HCV, and adolescent vaccination patients with various infectious and metabolic abnormalities.

Methods : Using an interactive virtual medical records interface, clinicians can review histories, order tests, make diagnoses, and start treatments for 76 patient simulation modules (3-20 cases/program) targeting >100 competencies in nine web-based and two African (CD-based) program sites. The simulation provides expert system, guidelines-based feedback on the appropriateness of choices, including a summary of medical errors, warnings, and deviations from guidelines at completion. Electronic mentoring occurs at the point of care.

Results: Usage: 4,669 clinicians representing 115 countries completed 9,893 sessions, averaging 27 pages in 18 minutes/session and an average score of 71 (42% failure). Errors: Users failed to: order required viral and metabolic tests in 25%, make secondary diagnoses in 40%, treat viral illness appropriately in 32%, manage co-morbidities (herpes, candida, DM and lipids) in 41%, use PCP prophylaxis in 21%, and order appropriate vaccines in 86% of sessions. In vaccine, lipid and arthritis programs, 44% of clinicians failed to order HIV screening. Outcomes: In two African training trials using WHO guidelines (1,319 pre-test/post-test modules), 32% of 164 users failed initially, but after activating clinical feedback, average scores increased by 34 points and failures declined to 4%.

Conclusions : These simulations show significant discordance between guidelines and clinical choices in therapy selection, preservation, and change. This tool can augment global HIV, HBV, HCV and vaccine mentoring and training, track performance of individuals and groups, target deficiencies, and provide a framework for certification of competencies.

Subject Categories (Complete): HIV and AIDS 2: Epidemiology/Prevention
Keyword (Complete): EDUCATION ; HIV CARE ; OUTCOMES

Biography : After graduating from the University of Virginia Medical School, George Washington University Hospital and Washington VA Hospital, Dr. Blevins earned board certification in internal medicine and infectious diseases and Clinical Assistant Professor of Medicine (UVA/Roanoke). As chief of infectious diseases and full-time practitioner for over 20 years at Lewis-Gale Clinic, a 100-physician multi-specialty group, he became chairman of medicine and President of the Board of Directors. Research interests have included hypertension, pathogen resistance, tick-borne diseases, sepsis, antibiotic safety, and infection control. He has served as advisor to or chairman of multiple committees and regularly lectured on antibiotics, infectious diseases topics and HIV/AIDS. Now residing in Durham, North Carolina, Dr. Blevins is involved in clinical, educational and advisory efforts on behalf of patients, clinicians and industry.

Author Block : D. Blevins¹, D. Hadden¹, J. G. Bartlett²;
¹TheraSim, Durham, NC,²John Hopkins Univ., Baltimore, MD.

NCHICA Annual Meeting, September 2007.
Collaborative Models to Improve Quality Care

Abstract : Novel artificial intelligence-based software systems are becoming available to improve the quality of patient care by measuring clinician behavior and targeting interventions to improve performance and reduce medical errors. When combined with business intelligence and statistical applications, these systems have demonstrated an ability to:

• Measure clinical performance and adherence to guidelines and evidence-based medicine
• Predict medical error rates and their financial impact
• Improve clinical performance through the implementation of preventative measures

The presenter will review collaborative models whereby expertise and best practices can be captured, tested in a competitive research environment, and applied to patient care through both simulation and clinical decision support software systems. Also discussed will be the quality and economic impact of artificial intelligence-based healthcare training, monitoring and management systems for healthcare providers.

Session Objectives :

• Describe the effectiveness of clinical performance management tools in influencing future clinical behavior in diagnoses and treatment
• Discuss strategies for managing healthcare provider risk and optimizing reimbursements for healthcare delivery organizations without compromising patient quality
• List ten steps in building and implementing clinical performance management systems

Author Block :

Charles Coleman, PhD1, D. Hadden 2 ;
¹ SAS Institute, Cary, NC, ² TheraSim, Durham, NC

44th Annual IDSA Meeting, Toronto, Canada, October 2006, Poster #911

DOUGLAS BLEVINS, MD1 , DAVID HADDEN, 01 , JOHN G. BARTLETT, MD2 ; 1 TheraSim, Research Triangle Park, NC, 2 Johns Hopkins, Baltimore, MD.

Background : Crucial to addressing the global HIV epidemic is a lack of trained clinicians. Instructors are costly and often unavailable for updates; present systems lack the capacity for student-specific assessment of improvement and take clinicians away from the work setting. We report on performance using an interactive web/CD-based simulated patient program that allows practitioners to manage virtual HIV patients.

Methods : Using histories, images, and labs that allowed practitioners to order tests, make diagnoses and start treatments, 75 patient simulations targeting HIV competencies were created. Advice and warnings were offered during the training simulation but could be turned off for testing. Included in the software we republished articles and guidelines (IDSA, CDC, DHHS, WHO, etc.) plus faculty over sight and comments that served to automatically gauge the appropriateness of choices and to provide links and feedback at each stage. Scores, an educational summary and a break down of warnings were provided at completion. Alternative cases for failed competencies allowed opportunities to document improvement. Compliance with standards for diagnosis, treatment and prophylaxis were recorded.
Results : An average case needed 21 minutes to complete; 3,559 users (36% USA) from 105 countries (22 African nations) completed 5,764 sessions. Average scores were similar from one region of the world to another. Treatment error rates were: hyperlipidemia (66%), HIV/HBV (57%), diabetes mellitus (48%), resistant HIV (10%), herpes infections (46%), candida (62%) and PCP prophylaxis when indicated (45%). Using country-specific standards of care, data from 8 formal training sites in Africa documented improvement in multiple specific individual skills.
Conclusions : These simulations show significant discordance between guidelines and clinical choices. Using this tool it is possible to augment global HIV training conveniently and inexpensively, to track improvement, and to provide a framework for certification of competencies.

Third Annual President's Emergency Plan for AIDS Relief (PEPFAR) meeting, June 2006, Durban, South Africa. Poster #248

DOUGLAS BLEVINS, MD1 , DAVID HADDEN, 01 , CINDY FIRNHABER, MD2, JOHN G. BARTLETT, MD3 ; 1 TheraSim, Research Triangle Park, NC, 2 Right to Care and University of the Witwatersrand, South Africa, 3 Johns Hopkins, Baltimore, MD.

Background : The lack of trained clinicians is a critical barrier to addressing the global AIDS epidemic. Instructors led courses alone are expensive, are not readily available for updates, provide passive learning without real-time detailed assessment of knowledge gained, and take clinicians out of the treatment setting. We report on clinical performance criteria and user satisfaction using an interactive web/CD-based simulated patient program that allows practitioners to manage virtual HIV patients.
Methods : Forty patient simulations reflecting important clinical competencies were created to allow clinicians to manage simulated HIV patients. Participants were provided histories, images, and labs as requested. They could diagnose, order tests and start treatments. Clinical feedback on users' therapy choices in the form of advice and warnings were offered during the simulation. Scores, an educational summary and a breakdown of warnings were provided at completion. Published guidelines (ISDA, CDC, DHHS, WHO, South African and others) were used to automatically assess the appropriateness of choices and to provide links and feedback. User satisfaction and compliance with national and international standards for diagnosis, treatment and prophylaxis were recorded.
Results : An average case required 20 minutes to complete; 2,779 users (36% USA) from 105 countries (22 African nations) completed 4,331 sessions. 51% failed to treat opportunistic infections, 45% failed to start PCP prophylaxis when indicated, 3% of clinicians failed to use HAART when indicated, and 44% failed to manage hyperlipidemia appropriately. Ineffective regimes based on resistance testing were ordered 9% of the time. Surveys revealed 97% satisfaction with the activity, and 98% considered simulation to be a valuable addition to clinical training. Additional data from 10 sites throughout Africa showing skills improvement over time will also be presented.
Conclusions : These internet- and CD- based clinical simulations reveal significant discordance between published guidelines and current clinical choices. Using this tool it is possible to augment global HIV training globally at low cost with high clinician acceptance, to track improvement over time, and to help provide a framework for certification of crucial competencies.

43rd Annual IDSA Meeting, San Francisco, October 2005. Abstract #753.

DOUGLAS BLEVINS, MD1 , DAVID HADDEN, 01 , BREWER, TIMOTHY MD MPH2; 1 TheraSim, Research Triangle Park, NC, 2 Mount Auburn Hosp., Cambridge, MA.

Background : The lack of trained health care workers (HCW) is a critical barrier to expanding antiretroviral therapy to address the global HIV pandemic. Available training methods such as instructor led courses are too expensive to expand to the scale needed, provide a passive learning experience, and/or do not provide real-time detailed assessment of knowledge gained. We report on performance and user satisfaction criteria using an interactive web-based electronic simulated patient program that allows practitioners to manage virtual HIV patients.
Methods : Six virtual patient histories were created using interactive educational software and actual patient data to allow clinicians to work through the management of simulated HIV patients. Participants were provided histories, photographs, and laboratory tests as requested, and were able to diagnose new diseases, order tests and treatments, and devise care plans. Advice and warnings were offered during the simulation, and an educational summary, including scores and a breakdown of any warnings, was provided at completion. Published treatment guidelines from IDSA, CDC, DHHS, WHO and others were used to assess the appropriateness of management choices and provide feedback. Time spent and user satisfaction with this training tool was recorded.
Results : The average case required 15 minutes to complete. Of the 744 clinicians from 34 countries completing 1,410 sessions, 71% completed 2 or more cases, and 92% finished any case started after completing their first one. While Caribbean participants demonstrated the highest average scores (most scored >90%), scores and time spent otherwise were virtually identical between USA and international participants. The survey revealed that 97% were satisfied or very satisfied with the activity and that 98% considered simulation to be a valuable addition to clinical educational training.
Conclusions : Using an internet-based clinical simulation program, it is possible to provide HIV training globally at low cost and with high clinician interest and acceptance.