ePRO Basics: 6 Key Success Factors

December 16, 2011 Jane Carter

Jane CarterWhile providing site training at an Investigator Meeting recently, I came across some interesting comments from sites that were not familiar with using eDiaries or tablets to capture PRO or ClinRO data.  One study coordinator was concerned about using eDiaries with certain populations, and was wondering if we were going to provide paper checklists or backup questionnaires on paper for subjects who were not comfortable using eDiaries.  This comment brought me back to the realization that while I have been working with ePRO for over six years, there are some people who still are not familiar with the technology or the risks inherent in mixing modalities of PRO collection.  I thought I would speak to some of these items, in hopes of providing a basic understanding of ePRO and some items to consider.

When choosing to collect Patient Reported Outcomes for a clinical trial, particularly for a primary or secondary endpoint, it is important to ensure the quality of the data being collected.  Most of us have heard of or experienced firsthand the “parking lot syndrome”, where patients fill in dosing diary cards or other PRO questionnaires in the parking lot immediately before going in for an office visit.  This is one of the main reasons sponsors choose ePRO over paper – ePRO technology allows for entry “windows” to ensure the data is being captured in a timely manner, and is not subject to the failings of memory recall.  I’m lucky I can remember what I had for breakfast yesterday, let alone how I felt or at what time I took medicine!

While some people are still concerned about technology and dealing with certain patient populations – in particular, the elderly – our statistics have shown that eDiary compliance by elderly subjects is extremely high (>95%).  Technology can be scary to people, but I have found that there are six key factors that improve the likelihood of success:

Success Factor 1 – Simple eDiary Design.  I follow the KISS model when designing ePRO collection tools – Keep It Short and Simple.  If the eDiary is too complicated, with strict entry windows and hard stops built into the eDiary, then workarounds need to be built and processes changed if a subject deviates from what is expected.  While I design to help the subject follow the protocol, I try not to make the design so rigid that the programming increases frustration and reduces compliance.

Success Factor 2 - Logical Edit Checks and Information Messages.  Inclusion of logical edit checks and information messages improves the quality of the data collected and provides guidance for the subject.  For example, if a subject is filling in a multi-question PRO and one response prompts for additional questions, then I build in the branching logic to ask the appropriate questions on a “Yes” answer, and to skip those questions on a “No” answer.  Additionally, I build in popup messages that remind the subject if he/she answers a question that is outside an expected range:  for example, if a subject enters an amount of medication that is outside the expected range for the protocol, then the eDiary would remind the subject to call the doctor.

Success Factor 3 - Reminder Alarms.  Reminder alarms can improve compliance and can provide “peace of mind” for subjects.  If a subject is filling in multiple questionnaires daily and a separate weekly questionnaire at a different time of day, I would build in an alarm to remind the subject to fill in the daily questionnaires, and I would build in a separate alarm that would sound to remind the subject to fill in the weekly questionnaire.

Success Factor 4 – Training Material.  The fourth success factor encompasses a wide area of items under the umbrella of Training.  Providing detailed, hands-on training to the monitors and site personnel using a “Train the Trainer” approach helps facilitate the training subjects will receive at site.  Additional support in the form of a detailed Site Manual helps reinforce the hands-on training and provides extensive troubleshooting and FAQs to ease site personnel with both their use and subjects’ use of the eDiary.  Having a Training Module built into the eDiary allows the subject to practice using the device and to become familiar with the device before leaving the office.  Providing easy to read, step-by-step guides for the subject to read and follow when using the eDiary helps ensure a comfortable transition from on-site support with the study coordinator to self-use at home.  While these guides are most used by older subjects who are less tech-savvy, they are very helpful for all subjects.  The guides should be easy to follow, include pictures of both the device and the screens to walk the subjects through everything they need to know about the device, and have a list of steps that they need to follow at any given time point.  Finally, access to helpdesk support for both sites and subjects is a key “safety net” to reinforce training and to get sites and subjects “back on track”.

Success Factor 5 – One Method of PRO Collection.  Sites often want “back up” methods of collecting PRO, and want to have paper questionnaires on hand in case a subject is not comfortable using ePRO.  I am now seeing protocols have an inclusion criterion that states that the subject must be willing to use an eDiary.  I recommend this be included in all protocols.  If a study provides alternatives like paper, then we not only deal with two points of collection and the multiple processes that need to be built to ensure consistency, but we also send a mixed message to the subjects and undermine their confidence in ePRO

Success Factor 6 – Site Personnel.  Whenever I am at an investigator meeting and am training sites on how to use the ePRO devices, I always encourage site personnel to be “cheerleaders” or “champions” of ePRO.  I remind them that If they act frustrated with a device or appear to treat the ePRO collection tool as just one more piece of equipment that they need to deal with, this attitude is transferred to the subjects.  I have seen a higher incidence of helpdesk calls, a lower rate of compliance, and a larger number of issues at sites where the site personnel appear unwilling or unmotivated during that initial training. 

Conversely, I have spoken with study coordinators who are high enrollers with high compliance rates, and I have discovered that universally those study coordinators make it a point of knowing everything they can about the devices, are engaged and ask lots of questions during the training, read all of the study materials including the guides given to subjects, and motivate their subjects by saying things like, “Your data is so important to this study, that the sponsor spent all this money getting these cool devices to collect your data.  You (the subject) are key to the success of this study!”

In conclusion, while we can provide all of the technology, design and support for the study, we must never underestimate the human factor, and how we present to and train our subjects on the use of ePRO.  It is a team effort!  The sponsor and ePRO provider can design and deliver quality ePRO to the sites, but we need the help of the doctors, nurses, monitors and helpdesk to provide seamless support and to be champions of ePRO in order to ensure subjects are successful.


ePRO Basics – Partnering with an ePRO Provider – What to Consider

ePRO Basics – Design Considerations for ePRO

ePRO Basics – Optimal Training Methods When Using ePRO

ePRO Basics – Monitoring ePRO Data: Does ePRO Data Get “Cleaned”?


Best Regards,

Jane Carter
Program Manager, CRF Health

The Future of GSM – Is 2G Wireless Technology Being Phased Out?
The Future of GSM – Is 2G Wireless Technology Being Phased Out?

I note that the discussion regarding the future of 2G networks has...

Interview with Diane Wild - VP ePRO & Translations - Oxford Outcomes
Interview with Diane Wild - VP ePRO & Translations - Oxford Outcomes

Diane Wild is a colleague that I have the privilege of working...