March and April are traditionally two months that for me involve an unusual amount of business travel. This year has been no exception and in the past 6 weeks I have visited Istanbul, Paris, Vienna, Dallas, Washington, Philadelphia, Cape Town and at the time of writing I am in Tokyo with just Sydney and Melbourne to visit before returning home for Easter. The work has been varied and interesting, including advisory boards, user group presentations and a good number of investigator meetings. What is different about this year is that the indications under discussion have included depression, schizophrenia, ADHD, Parkinson’s disease, Alzheimer’s disease and a variety of other diseases, including hypertension, diabetes and fibromyalgia.
Of course the theme of these discussions and training sessions has been the measurement of cognitive change. What has been remarkable is that the same issues of, i) which cognitive domains (e.g. memory, attention, executive function, etc.) are impaired, ii) dealing with practice effects, iii) issues of rater training, etc. have occurred in the context of each indication. What is also remarkable is that the same or at least similar candidate tests have been proposed for use across all these indications. Sometimes the very same test has been proposed, so tests of executive function such as the Controlled Oral Word Association Test are near ubiquitously employed. Sometimes the specific test to be used varies. For example, episodic verbal memory (psychologists’ fancy name for remembering a list of words) is measured using the Hopkins Verbal Learning Test in patients with schizophrenia, the ADAS-cog Word Recall test in patients with Alzheimer’s disease and the Rey Auditory Verbal Learning Test in patients with depression. However, at root these tests are all measuring the same cognitive construct using a word list learning paradigm. Word list learning is also a component of many of the multi-domain composite tests such as the MMSE, MoCA and SCOPA-cog that get used to measure cognition in a variety of indications. Often the measures selected are traditional measures, so-called ‘paper-and pencil’ tests, borrowed from clinical psychology. More often these days these measures are augmented or replaced by computerized measures from commercial systems such as the CDR System, CogState, CANTAB, Cogtest and CNS Vital Signs. Experts typically have their own views on which is the best test of each cognitive domain, though are typically agreed on what the fundamental cognitive domains are. Characterising the domains of interest was helpfully provided by the Measurement and Treatment Research to Improve Cognition in Schizophrenia (MATRICS) initiative. The seven domains listed are typically impaired in patients suffering from Cognitive Impairment Associated with Schizophrenia (CIAS). A recent review from Milan et al has described how many of these domains are compromised in a number of psychiatric indications. Impaired performance in these same domains is also often seen in dementia and other neurological diseases.
The MATRICS group chose to borrow tests from clinical psychology to cover the specific domains of function. This approach whilst pragmatic as a legacy inherits some of the traditional challenges of measuring cognition. An alternative approach would have been to develop brand new tests, specifically designed to address the measurement of cognitive change and ideally computer-based. Computer assessment can, by delivering stimuli at the prescribed time and capturing data, lighten the burden of test administrators. Computer assessment allows also for the rapid transfer of trial data, which lends itself neatly to prompt inspection and audit, ideally automatically. These new measures could be designed to more obviously reflect the tasks actually encountered in everyday life. Back in the 1970s Ulric Neisser begged psychologists to develop measures of cognition that had what he called ecological validity i.e. tests that index cognition using paradigms encountered in real life. Developing a brand new set of tests could have addressed up front the issues faced when using measures in different linguistic and cultural settings. We could also employ sophisticated psychometric approaches such as Item Response Theory to develop tests that can be titrated to the abilities of the patients we test. These approaches don’t preclude the use of paper and pencil testing, but would benefit from computerised administration, as with computers only the items appropriate to the current patient would be presented. This would save time and, potentially, confusion. We could develop a selection of brief, easy to administer, sensitive ‘best of class’ measures. The challenge always is that there is precious little money to be made in developing new tests. Consequently we often borrow measures that were designed as simple, fairly rudimentary bedside measures of cognition and press them into service as efficacy measures in clinical drug trials.
Initiatives directed at improving our cognition assessment measures are often specific to individual indications, such as was the case with the MATRICS program. Having spent time in different places, with different groups, considering the same issues for different indications, I’m wondering whether we should pool our skills, experience and resources to develop a cognition assessment that would serve all our needs. A true, international consensus would lend credibility and authority to such an enterprise. The support of stakeholders such as academic and clinical experts, regulatory bodies, commercial test vendors and other interested groups would be important in helping to get this done.
Academic and clinical research groups are a good source of creative ideas and innovative solutions. However, the process of integrating computer based assessments into a format acceptable for clinical trial use is an expensive and exacting process. Perhaps one way to meet current need would be for partnerships between research groups and commercial entities, which could operate in collaboration to mutual benefit. An assessment system capable of being used in a variety of CNS indications could accommodate the needs of many stakeholders. Regulators could include reference to a system in notes for guidance, sponsors could employ standard versions in their trials and vendors could be licensed the rights to distribute and support the resulting measures. Ultimately patients, caregivers and society as a whole would benefit from knowing that the drugs for which marketing approval was being sought had been shown to be truly efficacious.
John Harrison, PhD
Scientific Consultant, CRF Health