Page 133 - 2018_10-Haematologica-web

Impact of the eGVHD App on GvHD assessment
Introduction
Graft-versus-host disease (GvHD) refers to the reaction of the transplanted immune system against the recipient’s tis- sues. This pleiotropic disease affects up to half of patients after allogeneic hematopoietic stem cell transplantation (HCT) and can damage any organ system to various degrees. It is by far the most debilitating complication of HCT, considering its major impact on morbidity and mor- tality.1
Yet because of the lack of widely available GvHD bio- markers, the assessment of the presence and severity of GvHD still relies mainly on the clinical evaluation of multi- ple organs according to a relatively complex algorithm. Moreover, the recommendations underlying this evaluation are plethoric and sometimes even contradictory, potentially leading to confusion in the HCT community.1 In fact, it has been repeatedly shown that many HCT professionals have problems implementing GvHD assessment correctly, as demonstrated by a low observed accuracy in GvHD assess- ment2-5 and a slow uptake of the most up-to-date guidelines.5-7
The eGVHD App is an electronic tool that we developed in collaboration with the European Group for Blood and Marrow Transplantation (EBMT) Transplantation Complications Working Party and the National Institutes of Health (NIH) to assist healthcare professionals with their GvHD assessment.4 This tool is a web application, available on mobile devices and desktop computers (see www.uzleu- ven.be/egvhd for a complete list of the App’s characteristics). It allows intuitive and user-friendly access to the most recent international consensus guidelines and assists the user by automatically executing the required algorithm to calculate the severity of GvHD, once the relevant clinical characteristics have been entered.
Pilot testing was promising, suggesting improved GvHD assessment and good usability.4,5 Therefore, the primary aim of the present study was to compare the accuracy of the severity score of validated GvHD case-vignettes per- formed by healthcare professionals using the “eGVHD App” (“APP” group) with standard practice (“No APP” group). Secondary aims were to understand the characteris- tics that might affect the difference in accuracy between both groups and to compare the inter-observer variability in GvHD scoring results, as well as the time needed to per- form the GvHD evaluation of the full test package in both groups. We also assessed current practice patterns in GvHD assessment for all participants and post-test user satisfac- tion and experience in the “APP” group, to allow the tool’s usability to be further improved. To evaluate the generaliz- ability of the tool, we tested the eGVHD App in a variety of settings and with a wide range of healthcare practitioners with different professional backgrounds.
We hypothesized that the eGVHD App would improve GvHD assessment by improving the accuracy of GvHD severity scoring by healthcare professionals and reducing inter-rater variability in scoring results, without increasing the time required to assess GvHD.
Methods
Design
This study used a hybrid design (Figure 1). The first part of the study consisted of a 2-group multicenter randomized controlled
trial assigning healthcare professionals 1:1 to an intervention group (“APP”) or a control group (“No APP”) to evaluate the accuracy of GvHD assessment. The second part of the study was observation- al and described current practice patterns in GvHD assessment (“Survey 1”) and usability aspects linked to the use of the App (“Survey 2”).
Sample and setting
All Belgian hospitals performing allogeneic HCT were invited to participate (Online Supplementary Table S1) to optimize sample size and generalizability. Centers were selected on their willingness to organize a GvHD workshop on their own premises within the allocated timeframe (from January to April 2017). Healthcare pro- fessionals employed or studying at each participating hospital were recruited by convenience sampling. They were included pro- vided they attended the workshop (see Online Supplementary Methods for workshop details) and could recall having performed at least one GvHD evaluation in the past 12 months.
Information concerning data collection points, randomization procedure and blinding are available in the Online Supplementary Methods.
Outcome measures
The primary aim was to assess the difference in accuracy for GvHD severity scoring between the “APP” and “No APP” groups. (See Online Supplementary Methods for the planned sub-analyses.)
Variables and measurements
Demographics and practice patterns in GvHD assessment: a self- report questionnaire (“Survey 1”) captured participant characteris- tics (Table 1) as well as practice pattern in GvHD assessment and pre-test technology access and acceptance data (Table 2) at base- line.
Accuracy of GvHD assessment: participants were required to diag- nose and score a package of 10 randomly ordered GvHD clinical vignettes based on real-life clinical cases (see Online Supplementary Methods and Online Supplementary Table S2) according to the most up-to-date international guidelines.1 Four acute GvHD (aGvHD) vignettes covered the two types of aGvHD diagnosis (‘classic aGvHD’ and ‘late aGvHD’, two vignettes each) and the four aGvHD overall severity stages (I-IV, one vignette per stage), according to the Mount Sinai Acute GvHD International Consortium (MAGIC) criteria.8 Six chronic GvHD (cGvHD) vignettes covered the two cGvHD diagnoses (‘overlap cGvHD’ and ‘classic cGvHD’, two and four vignettes, respectively) and the three severity grades of the National Institutes of Health (NIH) 2014 criteria9 (two vignettes per severity level, i.e. mild, moderate and severe). Answers were given by participants using a multiple choice form offering the following mutually exclusive options for diagnosis (‘classic aGvHD’, ‘late aGvHD’, ‘overlap cGvHD’ or ‘classic cGvHD’) and scoring (‘grade I’, ‘grade II’, ‘grade III’, ‘grade IV’, ‘Mild’, ‘Moderate’ or ‘Severe’), respectively.
The individual answer of each participant was compared to the gold standard (see Online Supplementary Methods) and scored as ‘correct’ (if the answer corresponded exactly to the expert evalua- tion) or ‘incorrect’ (for any other answer, including missing answers) for diagnosis and severity scoring, respectively (Online Supplementary Table S3). The total number of correctly evaluated vignettes for the whole GvHD test package was also recorded per individual (score ranging from 0 to 10 correct answers), for diagno- sis and scoring separately. The time needed to complete the full GvHD test package was recorded for each participant individually by study staff.
Control group: participants randomized to standard practice (“No APP” control group) were allowed to use any of their usual meth-
haematologica | 2018; 103(10)
1699