Clinical Trials have traditionally collected data from patients by using the most ubiquitous technology of all: the pen and paper. While the use of electronic patient reported outcomes (ePROs) is becoming widespread, and ePROs are seemingly well accepted by regulators and the industry as a whole, there still seems to be an implicit sense of paper being the gold-standard when it comes to questionnaire design. Consequently questionnaires continue to be written with the unique properties, and limitations, of pen and paper in mind.
Paul O’Donohoe and Paul Margerison argue that it is time for the industry to move beyond this mind-set. Not only do the electronic media match the capabilities of paper, but there are capabilities unique to electronic media which allow data capture to be engineered in ways that are not possible using pen and paper.
Embracing the new media actually changes the type and quality of data that can be gathered, while improving the quality of data captured under existing paradigms. The new media offer a chance to think creatively. New research territory is opening up, and it falls to the disciplines of study design and instrument design to explore it. By treating electronic media as the primary platform, and looking afresh at how data capture is devised and executed, the industry will likely see trials harvesting more complex data, more reliably, and faster than was ever possible before.
Good day to everyone joining us, and welcome to today’s Xtalks webinar. Today’s talk is entitled Electronic by Default—Why Paper is No Longer the Gold Standard of Data Capture in Clinical Trials. My name is Andrew Jordan, and I’ll be your Xtalks host for today.
Today’s webinar will run for approximately 60 minutes, and this presentation includes a Q&A session with our speakers. This webinar is designed to be interactive, so please feel free to submit questions and comments for our speakers throughout the presentation, using the question chat box, and we’ll try to attend to questions during the Q&A session. The chat box is located in the control panel on the righthand side of your screen. And if you require any assistance, please contact me at any time by sending a message using that chat panel.
And at this time, all participants are in listen-only mode, and the presentation slides will advance automatically for you. Please note that this event will be recorded and made available for future download.
And at this point, I would like to thank CRF Health, who helped develop the content for this presentation. CRF Health is a leading provider of electronic clinical outcome assessment solutions for global clinical trials. With the experience on over 450 trials across more than 70 countries in over 100 languages, CRF Health’s TrialMax eCOA solution consistently demonstrates the industry’s high data accuracy, patient and site compliance, and patient retention. CRF Health’s TrialMax eCOA solution improves patient engagement by fitting within the lives of patients and seamlessly integrating into clinical sites to maximize protocol compliance. Their eCOA solutions include patient-reported outcomes (PROs), observer reported outcomes (ObsROs), and clinician or rater reported outcomes.
And now I would like to introduce our first speaker for today’s event. Paul O’Donohoe is Director of Health Outcomes at CRF Health and is based in their London office. He is responsible for developing the company’s internal scientific expertise and supporting the scientific consulting being offered to clients. Previously, Paul worked as a research psychologist at a child and adolescent mental health clinic based in Dublin, Ireland. He moved into the health consulting field with United BioSource Corporation, where we worked across the health outcomes, health economics, and health data capture groups.
Our second speaker for today is Paul Margerison. Paul speaks to us from the field of product design. Paul has worked on the design of products across many sectors, from news to retailing and telecoms. Before joining CRF Health, he was Head of Digital User Experience for the Education and Arts organization, the British Council. Based in CRF Health’s London office, Paul champions the needs of the end user, whether that person is a patient, a caregiver, or an investigator. He is there to remind us that diaries end up in the hands of ordinary people who strongly desire them to be friendly, confidence inspiring, and easy to use.
And now, without further ado, I would like to hand the mic over to Paul O’Donohoe. Paul, you can begin your presentation when ready.
Thank you kindly, Andrew. And very warm welcome to everyone who’s joined us on the line today, for this CRF webinar with myself—Paul O’Donohoe—and my colleague Paul Margerison, here in a rather uncertainly gray November evening in London.
Today we want to talk to you about a topic that is quite dear to both of our hearts, specifically about placing electronic data capture in the position of the default mode people think of when it comes to clinical trials. And not just electronic data capture as it exists now, but in a form that completely breaks free from the current so-called gold standard methodology created when paper was our only option.
So despite my obvious bias, I think it is probably relatively uncontroversial to suggest that electronic data capture is now really quite well accepted within clinical trials, both within the industry as a whole, but equally important by the regulators. And I think that’s largely been driven by the undeniable fact that you simply get better data with electronic systems, whether that’s less missing data, no out-of-range data, significantly higher compliance, all those good things. And despite the perceived up-front costs of electronic systems—and this is a topic I’ll touch on again in a moment—I think many recognize that’s a worthwhile price to pay for a high-quality data set with all the benefits for analysis and the ultimate demonstration of the effectiveness of the new treatment that this brings.
This move towards electronic has also been driven from the patient side, which I think is important to remember, where there is a real desire for a low-burden solution that fits seamlessly into their lives. And I think this is something we lose sight of in the worries over various regulations, which are obviously important, or this frame of mind we sometimes get into around trying to keep questionnaires looking like the original paper version. Consistently, we within the vendor side, but also from our clients, hear that patients of all ages are expressing a preference towards using these electronic devices for providing data rather than reams of paper.
And I think it’s important to say we’re not really here to talk about the benefits of paper versus electronic administration, because it is the topic that’s really been done to death already. Rather, what we want to do is to challenge a far more insidious effect of paper, an effect which places unnecessary and invisible boundaries on the way we go about capturing data from patients. Practically every year for the last 30 years has been the year technology was going to revolutionize healthcare, and 2015 is really shaping up to be no different with this huge interest we have in wearables and bring-your-own-device methodology. But I think we can all admit the revolution has yet to be televised, never mind streamed directly to your smartphone. And of course this is largely because really the future is mundane. By the time it arrives we’re already bored of it and have set ourselves on the next shiny, the next revolution around the corner. And the revolution particularly for healthcare and technology has kind of already come and gone, and will continue to come and go as technology develops. Yet we in the clinical trials spaces still seem to be striving to take full advantage of the technology we literally have to hand. The argument that myself and Paul will make is that the historical shadow of paper is playing a large role in stifling the potential we have available to us right now, and which will only continue to grow in the coming years.
You might say it’s kind of arrogant for us to dismiss paper so lightly as one of the most influential technologies ever invented, I think up there with the flushing toilet, it surely has a lot still going for it, something which we really can’t deny. People highlight familiarity with paper as being a benefit for patients who are already burdened by their disease and the unquestionable rigors of being part of a clinical trial. But I think as people were familiar with cassette tapes, LPs, wax cylinders and I’ll let you age yourself with the appropriate reference, I think time is the simplest cure for this issue as Moore’s Law continues to ensure computers become both more readily available and more dispersed into ever increase mundane and minute aspects of our lives. Free text entry is also something cited as something that handheld electronic data capture systems struggle with a bit, and this is a point I would concede. Paper often does prove more adept at dealing with allowing patients to jot down a quick note. However, the actual utility of such data has to be questioned, and again, related to the first point, as technology becomes more widespread, you see generations defaulting to free data entry using their handheld or tablet-based electronic devices. The perceived expense of eCOA as I have already mentioned, eCOA compared to paper, is probably the most serious challenge raised, and consistently raised. But not only do I believe that this is something that is widely miscalculated due to the hidden costs associated with paper—as I said, I think previously this argument ignored the price one is willing to pay for good data. That said, it’s an argument I've often heard before that paper can be printed at “no cost”--in inverted commas, because it is obviously not the case but is perceived as such—thus making questionnaires readily available wherever you have a computer and a printer, whether that be at site or at the patient's home. But I would in fact argue that its very strength is in fact a horrible weakness, meaning you lose complete control of when and how questionnaires are being produced for your study. But as I said this isn’t really a presentation about the pros and cons of the various media.
What we really want to do is open some new vistas for you to consider when you are developing future trials and questionnaires. And this is really the question we asked ourselves when we sat down to initially develop this presentation. How would we build questionnaires now if we never had paperpaper. If we were starting from scratch with the technology we have available today. Now, a rather obvious caveat that I should say at this stage is that we’re acting very much in the devil advocate's role here. We don’t propose that any or all of what we're going to present here will be implemented tomorrow in your pivotal Phase III study. But rather we’re just trying to get you, the key decision makers and thinkers in this field, to perhaps have a moment of quiet reflection to see if you're caught in what I’d like to term this paper frame of mind. Are you really thinking about what we can actually achieve with the computers that permeate our lives. Because it's only really when we begin to envision these possibilities that we can even start to change the system to accommodate. And so, without further ado, I will hand you over to my colleague, Paul, to get us started.
Thank you, Paul O’Donohoe. Starting from scratch then, I should like to begin by raising some points about a physical
object itself, that’s to say the lump of plastic or glass. Usually, it’s a phone or a table, but we don’t know what it would be in a few years’ time. It could be a watch or a pair of glasses or even a jacket. But let’s imagine it’s a phone or a tablet. A tablet is quite large, and it does give a lot of space to lay out questions in a neat and easily digested way. But mobile phones, on the other hand, don’t. Their screen real estate is quite small, with restrictions that you never get on paper. So my first point here in championing electronic media, ironically, is about this very disadvantage. Because a disadvantage it is and it comes with a benefit. We’re forced to place one question per page. One question that gets your full and undivided attention. No other questions around, influencing answer or distracting you from thinking about this one. And people with some conditions are particularly prone t being sidetracked and distracted, to jumping ahead before completing the job at hand. So one question per page, and you can’t do that. The result then, more credible answers.
Electronic devices are innately private things and paper is not. If you have a week’s worth of questionnaires filled out on paper, you could be nervous about your family or even your home help from seeing them. If you're logged out, then no one else can get in. If a person who keeps a traditional handwritten diary can be said to have an intimate relationship with the diary, it’s like a close friend, a friend with whom they share all of their heart’s innermost feelings. Well at CRF Health, we refer to our electronic questionnaires as eDiaries. And yes, they also do promote intimacy. The subject matter of the questionnaire is often intensely personal, and intimacy can help us get more candid answers from people.
The physical object itself—usually a phone or a tablet—is more durable than you might think. Okay, they don’t withstand much dropping, but you can safely use them in a damp place. Using paper in a bathroom is just awkward. You need a hard surface, and you need a dry surface, and your hands need to be dry. So in practice, you need to wait until you’ve finished in the bathroom before reporting data about what happened in the bathroom. Darkness is another environmental factor. You could fill in an eDiary in the dark under the covers if necessary. You could wake up in the night, fill in your diary, not really emerge from sleep, and then go back to sleep again. And as a result, you can design an eDiary for filling in in challenging places, rather than waiting until the patient is back into the light, back into the dry, and remembering what happened.
On the subject of questionnaires in challenging places, well a mobile device is by definition something designed to take anywhere. The hardware industry has invested heavily in making mobile devices that are not only slender and portable, but they’re also objects that you wish to carry around with you. And it’s been so successful that some people will naturally not leave their house without a phone in their pocket, they feel unequipped without it. So our industry then should try to benefit directly from that product development work. And as a designer of a study, you should be able to expect to get information anywhere, wherever the patient may be. If it’s an event-driven diary, then you can collect data almost as soon as it happens and not rely on the patient’s recall.
Let’s think for a second about the way the data gets into the system. If your study is using paper, then the physical input tool is a pen. But the main input tool for a touchscreen device is the finger.
In many ways, the finger used for recording data goes back to caveman times making marks on the walls with our hands. So the gestures that you use in tapping your answers onto an electronic device are atavistic, they’re unlearnt. And the buttons on a touchscreen device are finger sized, and the sensitive area around the button can be even larger than the button itself. And again, the manufacturing industry has put a great deal of research into the button size and shape. And our industry really ought to be directly benefitting from that. What about the illustration you see here, knuckles, somebody inputting the device data with their knuckle. And as an input tool, it’s the natural result of people whose movements are compromised, for example after a stroke, or a sufferer of rheumatoid arthritis. And what I like about the knuckle as an input device is that it’s not actually second-best, it’s just as good as a finger. And then you input your data on an electronic device, and you get feedback, feedback in the form of a change of color, or sometimes a shake or perhaps a sound. Obviously, paper can’t do that. That feedback is very very useful. It’s a confirmation that you’ve chosen what you think you’ve chosen, and it gives a user—a patient—satisfaction, the feeling of closure, the feeling of a small job that has been well done and is finished. And you get that feeling after every action.
When we’re talking about phones and tablets, we are really talking about screen-based visual design. Screen-based visual design is a field in its own right. It’s all about flexibility of presentation. And there has for the past five years or more, there has been an overriding theme in the interface design world that is that content should be separated from presentation, content separated from the way it’s presented. Content can adapt itself according to the device that it’s being shown on. So you can display it on many different types of screen, but its meaning remains the same. And similarly, content can adapt itself, mold itself to particular needs of a user or patient, needs which may be unique to that patient, that one person. If you were to speak to any screen-based designer in the commercial world, then they would pretty well all be well-versed in the rules of web accessibility. They would be very concerned to make user interfaces that are equally accessible to everybody, whatever their sensory or physical impediments. And there are many many techniques and tricks to make that possible. But some of the simplest though are just allowing the patient or user to choose their own font size, choose the size which is comfortable for reading. If a person is partially sighted, that font size, their font size of choice, could be very very large indeed. But still it is possible to design an interface that works if the font sizes are huge. And another simple technique is to allow the patients to set their own colors and contrasts. Color of the text, color of the background. Some patients see light text on a dark background better than dark text on a light background. And all sorts of color and contrast combinations are suited to all sorts of conditions. Some of them are quite surprising. Well, with a screen-based medium and electronic medium, it’s very easy to allow a patient to set their own color combinations.
Okay, so Paul has discussed the physicality of the medium as well as the flexibility that’s kind of built into the interface, and the relationship that this can foster with the patient. I would really like to build on the fact that the software being run on these devices allows us to engage with the patient with unprecedented intimacy.
Nowthis can be as simple as modifying what the patient sees based on some key information about themselves at a really basic level. You can just use the patient’s name throughout the questionnaire. Although I suggest avoiding this kind of faux intimacy that you might see in places like Starbucks, but it can still be a nice way of engaging with the patient in a really simplistic fashion. Slightly more complicated than this would be additional information related to the patient’s visit schedule or treatment regimen being displayed along with the actual questions we want patients to complete, making the whole system less passive and more useful for patients. And as one could keep all personal details localized on the device itself, there’s not fear of sites and study teams seeing information that they shouldn’t. And of course, as Paul already alluded to, the whole display can be customized this way, allowing patients to dress up, if you will, the interface in a way that pleases them, making it more enjoyable experience to interact with.
Even more powerful than this simple window dressing, however, is the ability to give something back to the patients in the form of their own data. An electronic device can obviously easily display various data trends, whether very simplistic summary of how a patient’s responses have changed over time, or even more complicated breakdown of trends and potential disease management considerations. This information is really a powerful tool for the patient to truly understand their disease and the treatment they’re receiving and increases the engagement they’ll feel with the study and the data capture system. They’re getting something back from all the work and information they’re putting in, that’s something really important to remember. And ultimately, we really need to remember that the data actually belongs to the patient, a point that members of the general public are becoming increasingly savvy to, as we see various ethical issues relating to social media and internet traffic being debated in mainstream media. I think it’s an issue that’s going to be raised more and more.
A more powerful force again than this personalization and data trends piece, is the ability to provide meaningful connections to the broader patient or participant network via the actual data capture platform itself, whether that’s a patient on a smartphone, tablet, computer, etc. Suffering from a disease and taking part in a clinical trial can be a very scary and lonely thing, and we see an increasing number of patients turning to the internet and social networks to build connection and share experiences and information. People are really juts looking for reassurance and empowerment, all things which we can provide directly through the data capture system. Not only do these connections help patients feel safe and supported and understood and thus more engaged with the study, it also has the potential of providing a rich source of qualitative data in the form of symptom or AE reports being shared publicly, or so-called social listening. Of course this whole topic immediately frees clinical teams outreach to the dangers of blinding and how to deal with AE reporting etc. But it’s going to happen no matter what one does. In fact it already is happening. That genie has long fled, so we should look to make it not only benefit the patients, but also benefit to us on the study teams and use it as a powerful source of additional information. We’re seeing how treatments are really performing in the real world.
So not only can the flexible nature of the interface as we discussed already prove to be a way of reducing burdens on patients by addressing some of their needs, whether that’s issues with vision, etc. as well as making the general interface just more pleasing and hence, one would hope, more engaging. We also have the potential to present questionnaires in ways beyond the simple visuals of text, the way we’ve been doing it for decades via paper.
A typical smartphone obviously has a multitude of ways that it can interact with the user, obviously through visuals. But this does not have to be simply text. We can present pictures and even videos. A And in turn, patients can take pictures or videos of their surroundings or their symptoms, as well as providing auditory responses to questions. And the patient’s voice really can provide a wholly untapped source of information around strength levels and anxiety. The ubiquitous touchscreen that Paul already referenced, which we find surrounding us every day, also allows us an additional form of interaction for patients, whether it’s drawing information on a picture, copying shapes as a form of visual, spatial, or cognitive assessment, or simply addressing their issues with fine motor skills, as Paul already touched on.
We can also merge all of these modes of administration to build a narrative presentation, or an element of gamification to really build a truly interactive and engaging user experience, which enfolds over a period of time, adapting to the patient’s situation, increasing enjoyment and increasing engagement with the patient.
Let’s think about the linear structure of a questionnaire, sequence that usually goes beginning, middle. end.
Paper can do logic in questionnaires quite well. With good graphic design you can have branches in a linear flow that don’t confuse the patient. But it’s not a good idea to have sub-branches, that’s to say branches that split themselves into two. Electronic media though, allow branches to split into as many sub-branches as you want, dozens of them, hundreds or even thousands of them, without adding complexity to the patient. So imagine one of your branches uses a drop-down menu of hard-coded answers to navigate to the next level of logic, you can have 20 or 30 new possible routes off a single question. As authors of questionnaires, the logic can be just as complex as you want it to be, and the patient is hardly aware of any branching at all, it feels linear. Computer adapted testing is a form of complex logic, it’s a way for you to maximize the precision of a diagnosis by asking questions based on what is already known about the patient from previous questions. Answers to questions modify subsequent questions.
Studies can be long, and over the course of a study, a patient’s circumstances may change. And you may wish to inquire about that change. A new direction in a patient’s life may be interesting to your research. Sometimes a patient may require more care as the study progresses, and in fact become unable to continue with PRO, patient reported outcomes. But that patient need not be lost. They can be switched to observer reported outcomes just at the right time for them. In the illustration on this slide here, you see a happier event than that. A patient who begins a study isolated and rather lonely, reporting on his or her daily feelings, and then some time later this patient finds a partner and the feelings of well-being change dramatically. So at this point you may wish to expose a patient to modified or completely new set of questions.
Adaptive studies are the same but on a macro study-wide scale. Studies themselves can be dynamic, adapting in near real time because data is transmitted to the study center in near real time. And you might be able to spot a trend, a trend you’ve not expected. That could be a swell of individuals answering in a similar way that you didn’t expect. Therefore you may wish to adapt the study to change the line of questioning in response and explore more deeply the new landscape that’s opening up. And you can’t do it with paper, not without recalling the paper that has been printed already.
There are some things that electronic media allow you to measure that we wouldn’t be able to measure reliably in any other way. Medical questionnaires, patient reported outcomes, capture a patient's own feelings about their health conditions. They are and always have been combined with more objective metrics about the body chemistry and measurements of the body. So combining those two things is not a new idea. But the scale on which they can be done now is new. Any electronic device that captures data about the patient can be integrated with the results of an electronic questionnaire. At CRF Health for example, we integrate data from blood glucometers, as the diabetic patient answers questions about their symptoms, about their food intakes, and the questions are supplemented with readings that are passed from the glucometer directly into the data set. And now electronic measuring devices are becoming ubiquitous. Patients, that’s to say ordinary people, are more and more used to wearing them of their own choice. Activity monitors are both medical aids and fashion items. And the data from them is ready for harvesting for clinical trials. As before, we can and should be benefitting from the hard work of the hardware industry to develop richer results.
Moving one step further from the idea of integrating other forms of data about the patient’s body, we can think about integrating data about the patient’s environment, which may also have an effect on patient’s health. Ambient noise, light, temperature, distractions like communication, perhaps in the future pollen count as well, they can all be measured continuously and can be accurately compared with patient
reported data, by which I mean the spikes in any of the data signals can be timed exactly and compared with any of the patient’s events to see whether they do or don’t coincide.
And moving on one step further from the idea of integrating environmental data, we can start to think about how to use data generated by the world in general, so-called big data. And here, the possibilities really are genuinely new and unexplored and could yield some truly unexpected insights. One way of describing the use of big data in a simple way is the act of looking at disparate apparently unconnected data sets and finding that when you do in large enough numbers, some patterns and connections emerge. So in this drawing, for example, you have a crowd’s opinion of cheese is changing according to the weather conditions, a connection you would not have predicted.
I’d like to recommend to you a TED Talk by the author by the author Kenneth Cukier, who can describe this much better than I can. He has numerous examples of findings and predictions that you may not expect from disparate data sets. For example, there is work going on in the automotive industry to place hundreds of sensors into a car seat, revealing that every person, every driver, has a unique signature posture. And they’re doing so in order to develop security devices. But a by-product of this will be that when all car seats have those sensors in them, we can analyze changes in posture and given enough data we’ll be able to identify the patterns in change of posture that happen, say, five seconds before a road accident occurs. A highly creative and original way of using data to make connections between different measurable phenomena. And I’m sure that we can find some equally original insights for our clinical trials by bringing in data from other physical sources.
So, to almost bring together a lot of the things we’ve presented here, is a concept we’d like to present on the system being more than a passive depository of patient data, which we kind of feel like is how it’s viewed at the moment. Really, the system can also administer care and it can be brought to kind of an ultimate realization with simplistic elements of AI, which would allow the system to become a key part of the patient’s care strategy while still obviously capturing the data needed to explore the impact of any treatment. Acting as an extension of the patient’s care team, on-system resources or links to external information can educate and support. More importantly, and more intimately, the system can act as a real-time adaptive responsive monitor of the patient’s wellbeing, recognizing trigger data or events that might indicate a forthcoming worsening of symptoms or an adverse event, alerting the patient or care team to try and minimize any potential impact. This really lifts the data capture system beyond the confines of a simple clinical trial and integrates it fully into a patient’s life, making it a key element of their treatment with all of the benefits to engagement and treatment quality that would bring.
So we’ll stop there with our whirlwind tour. We could go on for hours describing new ideas and new possibilities, because there are so many new options that open up with electronic media, and you could probably add to those too. We’ve looked at quite a few characteristics and properties of the electronic media in the context of gathering data about the patient, whether it’s submitted by the patient or submitted by a different device. Some of the properties are things that data could do, but that electronic media can do spectacularly better, so much better that the two are not really comparable. But other possibilities are genuinely new, and as such, it’s down to us, down to you if you’re involved in study design, to think about what the new possibilities really are, because we don’t really know yet.
When a new medium is invented it brings with it possibilities that have not been predicted by its inventors. This phenomenon happens every time there’s a new medium. On your screen now, you can see the actress Lillian Gish in a shot from D.W. Griffith’s film, The Birth of a Nation, that was in 1915. The very earliest filmmakers had no concept of the close-up shot. They just place the camera in a fixed position in front of the group of actors. And then someone thought of zooming in onto one of the actors to see details of their emotions, their reactions, and thus a truly cinematic experience was invented, never seen before. And D.W. Griffith was one of those visionaries. The earliest film audiences didn’t know what to make of seeing a huge face like this without legs or a torso. And in fact some of them found it horrific. But audiences quickly got used to the idea of the close-up shot, and now we can hardly imagine cinema without it. But if filmmakers had to make it possible to reverse engineer all of their films back to stage plays, then we would never have had this invention at all. So this is the place where we are with electronic media in the context of clinical trials. We’re at the moment where we could discover or invent the equivalent of the close-up shot.
So after all of that, I imagine you might have more questions than answers, which really was kind of the intention to be honest. Or in fact you might just be completely dismissive of the whole thing. But I contend that this is the paper frame of mind talking, convincing us that such things are madness, that the regulators will never allow it, what about issues of equivalence, the lawyers will never agree to it, potentially more importantly, just that it generally can't be done. And to be honest, you could be right. But I honestly don’t think so. I think that this is coming. I think versions of it already exist, and versions of everything we’ve discussed and more besides, things that we haven’t and can’t even envision from our current techno-historical perspective. I think all of these things are coming. EDC systems used to be a fantasy; now you’d be mad not to use one. BYOD was science fiction for clinical trials not that long ago, but I would argue that now a lot of us are just waiting for the publication of the evidence we know is coming, showing that there is little difference in data between devices. Patients are truly not a limiting factor. A huge number of them have been providing reams of rich, multi-modal qualitative and quantitative data multiple times a day to services like Facebook and Twitter. As we’ve made clear again and again, the grinding inevitability of time and progress makes these ideas increasingly achievable, accessible, and at the end of the day, mundane. The message really we want to leave you with is simply this: study teams, questionnaire developers, vendors, and of course the regulators, just keep in mind this paper frame of mind and how it might be influencing your thoughts on what can be done. Boundaries are very much there to be pushed, so I think we should see how far we can take this latest technological revolution.
And thank you very much for your attention.
Thank you both so much for that excellent insightful presentation there. At this point, I would like to invite our audience to continue sending their questions and comments right now using that questions window for the Q&A portion of the webinar.
[Q&A section starts at 37:21]