Leveraging Today's Gaming For Better Health Reported Outcomes Tomorrow - ICDVRAT 2018

October 16, 2018 Bill Byrom

12th International Conference on Disability Virtual Reality and Associated Technologies, In collaboration with Interactive Technologies and Games, University of Nottingham, UK, 4th – 6th September 2018.

Serious games have become an active area of research and development in the areas of education and health.  In healthcare, much activity has focused on the use of immersive gaming technologies such as virtual reality and motion-based gaming platforms (e.g. Microsoft Xbox) to develop applications to provide an engaging and enhanced approach to deliver rehabilitation exercise regimens [1]. However, there are also examples of the use of games to provide a means to measure and report health outcome data to enable measurement of disease status and changes due to treatment.  For example, the Project:EVO video game (Akili Interactive Labs, Boston, MA) has been used to evaluate multitasking performance in Alzheimer’s and Attention Deficit Hyperactivity Disorder patients [2]

This three-day conference hosted by the University of Nottingham, UK, brought together researchers from across the world to review work developing applications using immersive and mobile technologies as treatment options, or as a means of objectively measuring and tracking the effects of other treatment interventions.

In collaboration with researchers in the Medical Design Research Group and Nottingham Trent University, CRF Bracket presented a paper on methodologies to assess meaningful change associated with technology interventions or technologies used to develop new clinical endpoints to measure intervention effects.  This is an essential component of clinical endpoint development and in demonstrating the benefits of technology interventions to support health technology assessments.

Meaningful change

“Meaningful change” represents the smallest difference in an endpoint measure that would be perceived by patients as beneficial.  Meaningful change of the same clinical endpoint is likely to be different for different patient populations.  For example, the meaningful change in the number of steps walked per day is likely to be lower in less active patient populations compared to more active groups.  While the importance of statistical significance in demonstrating the effects of an intervention is unquestioned, it is also important to recognise that effect sizes detected through statistical tests may be of insufficient magnitude to be considered relevant to the patient. 

This meaningful change may be represented by the minimal important difference (MID), also called the minimally clinically important difference (MCID), or by the minimal individual change that distinguishes a responder (an individual exhibiting a meaningful improvement) from a non-responder.  The MID/MCID represents the minimum change in group means considered clinically relevant; whereas the individual responder definition represents the magnitude of individual change considered clinically relevant. 


While there are a number of methodologies to measure the MCID or define a responder, such as consensus panels, distributional methods, and anchor-based approaches, the most widely recommended approach is to use anchor-based methods.  These approaches relate the changes observed in the new clinical endpoint under development to changes observed in other endpoint measures for which meaningful change is understood (anchor measures).  This approach has been well described in the literature.  One example using mobile sensor technology was reported by Motl et al [3] who estimated the MCID for the total daily number of steps walked by patients suffering from multiple sclerosis (MS).  They compared changes in daily steps to changes in two patient-reported outcome measures: the Multiple Sclerosis Walking Scale (MSWS-12, a 12-item PRO measure assessing the impact of MS on walking-related activities) and the Patient-Determined Disease Steps (PDDS) scale.  Ten-point changes in the MSWS and a single point change in the PDDS were considered clinically meaningful, and corresponded to changes of 642 and 915 steps per day in the sample of MS patients studied – leading to the MCID estimate of 779 steps/day.

As we seek to develop immersive and mobile technologies as treatments of the future, or as engaging ways to measure health outcomes, measuring meaningful change as a component of the development and validation study programme is an important, yet often overlooked, component.



[1] Byrom B, Breedon P, Siena L et al. (2016).  Enhancing the Measurement of Clinical Outcomes Using Microsoft Kinect.  IEEE.

[2] Wright JM, Regele OB, Kourtis LC, et al., (2017), Evolution of the digital biomarker ecosystem. Digit. Med., 3, pp. 154-163.

[3] Motl, RW, Pilutti, LA, Learmonth, YC, et al., (2013), Clinical Importance of Steps Taken per Day among Persons with Multiple Sclerosis, PLoS ONE, 8, 9, pp. e73247

No Previous Articles

Game Technology Opens New Path to Clinical Endpoints - DIA Podcast
Game Technology Opens New Path to Clinical Endpoints - DIA Podcast

Dr. Bill Byrom, VP of Product Strategy & Innovation at CRF Health discusses the future of electronic patien...