geoscience.net logo
+ Resolve Article
+ Follow Us
Follow on FacebookFollow on Facebook
Follow on TwitterFollow on Twitter

+ Translate
+ Subscribe to Site Feed
GeoScience Most Shared ContentMost Shared Content

Integrating usability testing and think-aloud protocol analysis with "near-live" clinical simulations in evaluating clinical decision support


, : Integrating usability testing and think-aloud protocol analysis with "near-live" clinical simulations in evaluating clinical decision support. International Journal of Medical Informatics 81(11): 761-772

Usability evaluations can improve the usability and workflow integration of clinical decision support (CDS). Traditional usability testing using scripted scenarios with think-aloud protocol analysis provide a useful but incomplete assessment of how new CDS tools interact with users and clinical workflow. "Near-live" clinical simulations are a newer usability evaluation tool that more closely mimics clinical workflow and that allows for a complementary evaluation of CDS usability as well as impact on workflow. This study employed two phases of testing a new CDS tool that embedded clinical prediction rules (an evidence-based medicine tool) into primary care workflow within a commercial electronic health record. Phase I applied usability testing involving "think-aloud" protocol analysis of 8 primary care providers encountering several scripted clinical scenarios. Phase II used "near-live" clinical simulations of 8 providers interacting with video clips of standardized trained patient actors enacting the clinical scenario. In both phases, all sessions were audiotaped and had screen-capture software activated for onscreen recordings. Transcripts were coded using qualitative analysis methods. In Phase I, the impact of the CDS on navigation and workflow were associated with the largest volume of negative comments (accounting for over 90% of user raised issues) while the overall usability and the content of the CDS were associated with the most positive comments. However, usability had a positive-to-negative comment ratio of only 0.93 reflecting mixed perceptions about the usability of the CDS. In Phase II, the duration of encounters with simulated patients was approximately 12 min with 71% of the clinical prediction rules being activated after half of the visit had already elapsed. Upon activation, providers accepted the CDS tool pathway 82% of times offered and completed all of its elements in 53% of all simulation cases. Only 12.2% of encounter time was spent using the CDS tool. Two predominant clinical workflows, accounting for 75% of all cases simulations, were identified that characterized the sequence of provider interactions with the CDS. These workflows demonstrated a significant variation in temporal sequence of potential activation of the CDS. This study successfully combined "think-aloud" protocol analysis with "near-live" clinical simulations in a usability evaluation of a new primary care CDS tool. Each phase of the study provided complementary observations on problems with the new onscreen tool and was used to refine both its usability and workflow integration. Synergistic use of "think-aloud" protocol analysis and "near-live" clinical simulations provide a robust assessment of how CDS tools would interact in live clinical environments and allows for enhanced early redesign to augment clinician utilization. The findings suggest the importance of using complementary testing methods before releasing CDS for live use.

US$19.90

PMID: 22456088

DOI: 10.1016/j.ijmedinf.2012.02.009


Other references

L.L.ng; Sadofsky Seth J.; Bebout Gray E., 2003: Carbon and nitrogen input fluxes in subducting sediments at the Izu-Bonin and Central America convergent margins. Eos, Transactions, American Geophysical Union 84(46, Suppl

Zhang, H.; Ye, X-Jian.; Li, J-Shun., 2010: Preparation and biocompatibility evaluation of apatite/wollastonite-derived porous bioactive glass ceramic scaffolds. An apatite/wollastonite-derived (A/W) porous glass ceramic scaffold with highly interconnected pores was successfully fabricated by adding a plastic porosifier. The morphology, porosity and mechanical strength were characterized. The results showe...

Balkwill, F.R.; Griffin, D.B.; Band, H.A.; Beverley, P.C., 1983: Immune human lymphocytes produce an acid-labile alpha-interferon. We have described in this paper a novel human interferon (IFN) with antigenic and cross-species reactivity of alpha-IFN and physicochemical properties of gamma-IFN. This IFN is produced by normal peripheral blood mononuclear cells during an immune...

Wilson, Mark V.H.Reviewer, 1997: Devonian fishes and plants of Miguasha, Quebec, Canada (book review). Copeia(1): 4-6

Wang, D.; Zhong, Y.; Luo, X.; Wu, S.; Xiao, R.; Bao, W.; Yang, W.; Yan, H.; Yao, P.; Liu, L., 2011: Pu-erh black tea supplementation decreases quinocetone-induced ROS generation and oxidative DNA damage in Balb/c mice. Quinocetone (3-methyl-2-quinoxalinbenzenevinylketo-1,4-dioxide, QCT), a new feed antibacterial agent of quinoxaline-1,4-dioxides family, has been used as an animal growth promoter. However, few data about its potential toxicity in vivo were availa...

Eckert, J., 1967: In vitro development of invasive larvae of nematodes to parasitic stages. Zeitschrift für Parasitenkunde 29(3): 242-274

Banerjee, J.; Nema, V.; Dhas, Y.; Mishra, N., 2017: Role of MicroRNAs in Type 2 Diabetes and Associated Vascular Complications. Type 2 diabetes mellitus (T2DM) has become a major health threat worldwide. MicroRNAs (miRNAs) are a group of non-coding RNAs known to regulate various biological processes including the pathogenesis of T2DM. Recent studies have pointed out that s...

Bender, D.A.; Sheik-Bahae, M., 2010: Complementary ultrashort laser pulse characterization using MOSAIC and SHG FROG. A new (to our knowledge) method for generating the modified spectrum autointerferometric correlation (MOSAIC) trace from the second-harmonic generation frequency-resolved optical gating (SHG FROG) dataset is shown. Examples are presented illustrat...

Lasansky, A., 1974: Synaptic actions on retinal photoreceptors: structural aspects. Federation Proceedings 33(4): 1069-1073

Arey, B.J.; Stevis, P.E.; Deecher, D.C.; Shen, E.S.; Frail, D.E.; Negro-Vilar, A.; Lopez, F.J., 1997: Induction of promiscuous G protein coupling of the follicle-stimulating hormone (FSH) receptor: a novel mechanism for transducing pleiotropic actions of FSH isoforms. Under physiological conditions, FSH is secreted into the circulation as a complex mixture of several isoforms that vary in the degree of glycosylation. Although it is well established that the glycosylation of FSH is important for the serum half-l...