Page 137 - 2018_10-Haematologica-web

Impact of the eGVHD App on GvHD assessment
continued from the previous page
Desktop
Other
Not answered
Median importance of the availability of the app in my native language°
Median reported level of likelihood
of using the app°
32 (42%) 0 (0%) 2 (3%) 4
(IQR 5; range: 1-10)$$
8
(IQR 3; range: 1-10)$$$$$
10 (27%) 1 (0%) 0 (0%) 4
(IQR 6; range: 1-10)
7.5
(IQR 3; range: 1-10)$
22 (55%) 0 (0%) 2 (5%) 4
(IQR 5; range: 1-10)$$
8
(IQR 4; range: 1-10)$$$$
n: number; IQR: Interquartile Range; NA: Not applicable. *Several answers were possible. °Reported on a Likert scale of 1 (lowest) to 10 (highest).The number of $ symbols used indicates the number of missing participants.
timation of the cGvHD grade (overestimation n=34, 15%; underestimation n=20, 9%; missing/other n=4, 2%) with- out any misclassification, whereas the “No APP” group tended to evaluate cGvHD severity erroneously according to the aGvHD criteria (n=62, 25%), without bias for sever- ity (overestimation n=36, 14%; underestimation n=36, 15%; missing/other n=7, 3%).
Consequently, inter-observer agreement of the severity
score was higher in the “APP” group compared to standard
practice: the probability that 2 HCT professionals agreed on
the GvHD score equaled 0.73 and 0.56 in the “App” and
In this study, participants in the control group were allowed to use any method of their choice to support their GvHD assessment, except for using the eGVHD App. Yet GvHD assessment results in the “APP” group, were striking- ly better. We believe that the superior performance of the App users could be due to a number of factors. First, App users were provided with the most up-to-date guidelines,1 without having to look them up actively. Second, similar to using comprehensive paper data collection forms, they were encouraged to work in a systematic fashion: they had to evaluate every possible aspect of acute or chronic GvHD (to avoid overlooking less intuitive aspects of the disease) in order to select the appropriate scoring system and come to the correct severity evaluation result. Finally, the digital interface also offered users a number of advantages such as the presence of pictures and definitions to support recogni- tion of GvHD-related features, the use of ‘skip-logic’ princi- ples (which allows healthcare professionals to avoid wast- ing time on filling in information with no direct impact on diagnosis or severity scoring), the automatic computation of the resulting score, and the option of generating a report.
We have to acknowledge that this superior performance was achieved at the cost of a significant increase in the time needed to score clinical vignettes, with an excess of approx- imately 24 minutes to score the ten clinical vignettes com- pared to using standard methods. This was partially due to the fact that “APP” users needed to get used to a tool they had never worked with before. Yet healthcare professionals remained open to the use of eHealth technology, both before and after actually using the App. The eGVHD App showed excellent usability, as no major technical issues were noted and user feedback was widely positive, suggest- ing a potential for optimal dissemination and uptake in the HCT community. Furthermore, in the event where the App-computed scores would be directly transferred into the electronic health record (eHR), the additional time spent inputting data into the App would be rewarded with poten- tially less time charting, and more accurate data collection. However, this integration also presupposes a number of basic pre-requisites, which still need to be developed: data cleaning methods to ensure the quality of data entry, the possibility of crosstalk between the eGVHD App and the different eHR systems, the reliability, privacy and safety of data transfer, and the option of identifying the individual who performed the data input.
Consistent with prior literature, our practice pattern sur- vey showed the lack of consensus in the HCT community as to which set of international recommendations should be used to assess GvHD, and confirmed numerous barriers to their successful dissemination and implementation.5-7
“No APP” group, respectively. The chance-corrected agree-
ment was significantly higher in the “APP” group (κ = 0.46, BP
95%CI: 0.23-0.68) compared to the “No APP” group (κ
=0.12, 95%CI: 0.03-0.21) (P=0.003).
The time needed to complete the total test package was
significantly higher in the “APP” group compared to the standard practice group, with a mean time of 48.84 minutes to complete all ten clinical vignettes in the “APP” group ver- sus 25.27 minutes in the “No APP” group (P<0.001) (Table 3).
Post-test user satisfaction and experience
No major technical issues were identified. Both “per- ceived usefulness” and “system usability” were considered to be good, as shown in Online Supplementary Table S4. Users reported being likely to use the eGVHD App in their daily practice and did not experience any issues with using the App in English. Spontaneously reported positive aspects of the eGVHD App were its clarity, ease of use, and its sys- tematic approach. Users suggested some potential improve- ments, such as decreasing its time-consuming components, reducing the number of evaluated items, and clarifying some specific terms in more detail.
Discussion
Several groups have recently advocated the use of elec- tronic tools to improve GvHD assessment, albeit without providing formal proof of their efficacy.1,4,12-14 In this rigorous multi-center randomized trial, we unequivocally demon- strate that the accuracy of GvHD assessment of clinical vignettes by healthcare professionals is significantly higher when using the eGVHD App compared to standard prac- tice. This effect was seen for both acute and chronic GvHD, across all severity levels (except for aGvHD grade I) and all degrees of experience and professional backgrounds, with- out any evidence for center effect.
BP
haematologica | 2018; 103(10)
1703