Bring Your Own Device (BYOD) is the latest buzz-phrase in the eCOA industry, with the possibility of taking advantage of patients own devices to administer study questionnaires being touted as promising higher compliance and reduced patient burden compared to traditional eCOA studies. But some key challenges remain to the adoption of BYOD as a viable method for clinical trials, not least how patient’s themselves will respond. This session will present findings from usability testing of an app-based questionnaire in a broad sample of member of the general public which provides insight into the true challenges that face BYOD.
Hello again, everyone. I hope we’re all feeling wonderfully refreshed after lunch. You’ve probably heard, or you have heard in fact from some of the previous presenters, this term BYOD being thrown around. So I just wanted to make sure everyone was actually familiar with the term. So when we’re talking about electronic data capture, electronic clinical outcome assessments, traditionally we’re talking about providing participants in a clinical trial with a standalone device, whether it’s a tablet computer or a smartphone for example, and all participants within that study would receive the same device with the questionnaires already preloaded on it. So they have a consistent experience across participants and across questionnaires that they’re completing. BYOD—the acronym stands for bring your own device — exactly as the name suggests, is taking advantage of the smartphone that the participant has with them, and an increasing number of participants have with them at all times to administer the study questionnaires. So you’re getting the participant to traditionally download an app, but maybe also access questionnaires on a web system, for example, using their own device rather than providing them an additional device that they have to carry around with them or that they have to take home. I think this probably intuitively makes a lot of sense to us, we might as well take advantage of the computer that a lot of people are carrying with them at all times. And there’s been a few reasons why BYOD has become such a hyped term within the industry at the moment. It’s kind of the equivalent of patient centricity, as I mentioned earlier, is that real big hype phrase, you always have to have a session on patient centricity at a conference talking about healthcare. Well in the same kind of way if you’re having a conference about eCOA you always have to have a session about BYOD. And some of the reasons kind of driving this hype I believe are the idea of reduced hardware costs. So because you’re not providing devices for every single participant in the study, the theory goes that you’re not going to have obviously as high hardware costs because you’re not going to have to provision an individual device for every single participant within that study. Seems to make sense. There’s also this idea of a reduced burden for participants. So rather than get a device they’re completely unfamiliar with, they’re actually using the device that they use every day as it is. I’ve seen statistics thrown around the people interact with their smartphone 100, 200, 300 times a day, and they’re never within a few feet of their smartphone. So it’s something they're very very comfortable with, so why not take advantage of that rather than getting them to learn a whole new device, a whole new way of interacting with a piece of hardware. And then there’s also this idea of potentially it opens up the possibility of capturing some different data, some more novel data, taking advantage of some of the additional sensors that are on smartphones nowadays, and that’s not something I'm going to talk about in this presentation but I'm going to touch on it tomorrow when I'm going to dig a bit more into what we might do to make patient-reported outcomes a bit more interesting.
I think though, the real reason, despite these kind of hyped reasons for why BYOD is so popular as a concept at the moment, I think the real reason is simply Moore’s Law, which is technology gets cheaper, technology gets better, as time goes on. So this is some figures from Ericsson who are predicting that there’ll be over 6 billion smartphone subscriptions by the year 2020 with a projected world population of I think 7.9 billion or something. So smartphones are becoming ubiquitous. They’re so common, they’re reaching saturation levels in certain populations. And so why not just take advantage of those. We’re in a situation now where participants have a computer that we can take advantage of. And I think this is the real reason why BYOD is now becoming this buzzphrase that it is within the industry. The market is right, the hardware is there for us to take advantage of.
The regulators have suggested that they’re open to the idea of BYOD. They recognize that it’s kind of a natural evolution of eCOA, that we should be moving towards allowing participants to use their own device where possible. But they also have raised two key concerns around BYOD. And you’ve heard this turn around in the previous panel we had, the issue around equivalence. So are people going to answer the same on the wide range of different devices that might be used in a clinical trial when you’re allowing participants to use their own device. So will there be some kind of systematic bias introduced because a section of your population is using a small screen iPhone 5 versus the population that’s using a Galaxy Note, which has a much larger screen for example.
They’ve also raised this concern around inclusiveness and the fact that the might be excluding participants from a clinical trial because they don’t have access to really what at the end of the day is a very expensive piece of hardware. I think we can all agree that quite an unethical reason to not allow someone to take part in a clinical trial is because they don’t have a suitable smartphone to complete the questionnaires on. Both issues that I’ll touch on in a bit more detail. And to talk about the equivalence, and I don’t want this to be the focus of this particular presentation, but the issue around equivalence when it comes from paper to electronic comparison, so making sure participants are answering the same on electronic versions of paper questionnaires, certainly in my opinion I think we’ve largely answered at this stage. We were just recently involved with a publication with our friends in ICON—and Willie’s here as well—which was a meta-analysis looking at all published studies from 2007 to 2013, so all studies that compared paper to an electronic version of that questionnaire. I think there was about 72 studies where we found very high equivalence between the paper and electronic versions. And this echoes another meta-analysis that looked at studies published before 2007, found the exact same results, very high concordance between paper and electronic versions of questionnaires.
The subtle difference with BYOD is we’re now faced with a case of rather than going from paper to a single device in a clinical trial, which is that traditional provisioned model, we’re going from paper or an existing electronic implementation to potentially tens if not hundreds of different devices within a single clinical trial. And we obviously can’t test every single device that might be used. We can, though, potentially test a range of devices, so from a small-screen device to a large-screen device, and make some assumptions about what might be happening between that range. And I also think, to just hearken back to the meta-analysis we published, I think it’s a safe argument to make that going from paper to electronic is probably a bigger change than going from a slightly smaller screen size to a slightly bigger screen size. I feel like that’s not a particularly crazy argument to make. So based on all the evidence we already have of paper-to-electronic equivalence, what kind of assumptions might we be able to make about BYOD.
So that’s something we wanted to dig into in a bit more detail. But we also wanted to start exploring issues around usability and accessibility of an app-based version of a questionnaire that might be used in a clinical trial. So does this assumption around lowering burden on participants, does this ring true? Is there any particular technical issues that might arise during a BYOD study that we should be aware of. And will kind of fundamentally participants actually want to use, for example an app-based version of a questionnaire in a clinical trial where they might have to participate for months if not years at a time.
So we decided to run a pilot study in 20 participants, 20 members of the general public. We got a good range of gender and ages from 21 to 69, with a mean age of about 40, and a nice spread of self-reported comfort with technology. Similar to Chloe, I think we struggled to identify anyone who had almost zero experience with technology, we ran this in the UK. That’s a very fastly diminishing population, people who have no experience interacting with, for example, touch-screen devices. But we did manage to get a number of people who claim that they weren’t particularly comfortable using technology. And we observed and cognitively interviewed them as they interacted with an app-based version of a vaccine symptom diary. So it was a diary where you would report symptoms you’d had after you had a vaccine injection, talking about temperature, talking about rash, etc. Quite a typical thing you might use in a vaccine clinical trial. And we got them to interact with this vaccine symptom diary in a range of different smartphones, three specifically—the iPhone 5s being the small one, the Samsung Galaxy Note being the large device, and then a BLU Life Play which is kind of a cheaper Android device, as the middling device. So there’s a range of what we felt were probably pretty standard screen sizes. And then we also asked the participant to download the app on their own smartphone where possible.
So to kind of get over the equivalence question early on—because as I said that’s not the focus of this presentation—basically participants reported that they wouldn’t interpret or respond to questions differently on the different devices. Some nice quotes there. “They all look exactly the same. I’m really comfortable with smaller phones but would be happy to use all three and it wouldn’t affect the answers.” “There would be no difference in answers. I could comfortably do it on any device.” So participants saying that they wouldn’t answer differently because questions were being presented on different devices. In fact only three participants explicitly raised some concerns around this issue of equivalence across devices. One said, “I would probably answer the same in all devices, but if you were not used to a small phone you could miss something and answer differently.” Kind of makes sense. “You may concentrate more on the big screen if it was flat on a table, so could possibly give different answers.” I’m not sure if I 100% understand the logic behind that statement but obviously, they feel something about the way the questionnaire is presented could impact how they respond. And one participant felt that you may go into more detail on a device that is easier to use. For example, typing would be easier, which makes a lot of sense.
I think the important thing to point out here is that these are all issues around familiarity, being familiar with the device you’re using. In a real BYOD study, the whole logic of it is that the participant would be familiar with the device because they’re using their own device. So I just want to highlight something about the study is that we really took a worst case scenario approach, we didn’t offer any training. Yet a very fair point, so maybe to summarize for the microphone, the issue of familiarity will still be of a concern when we’re provisioning devices within a BYOD study, which we will have to do because not all participants will have an appropriate device, we’re assuming. So this issue of familiarity is actually still a concern, although ironically maybe not a concern for the BYOD element of the study, which is where the concern initially rose. But again as Rauha said, it does tie back to design best practices, and making your solution as usable as possible across all possible devices, including the potentially provisioned device.
AUDIENCE MEMBER: But that’s what you do now in your provisioned model, right? So you should know how to do that.
Indeed it is what we do now in the provisioned model, and we develop the solutions to be as usable as possible. But then when you see a difference between people using an unfamiliar provisioned device versus people using their own personal device that they are familiar with.
AUDIENCE MEMBER: I would say typically the devices are on a two-year contract. So for that changeover, there’s going to be a learning curve when your user is transferring from one phone to another.
Yeah, so you mean participants transferring from one phone to another phone, or when they lose their smartphone. Yeah, so I think that’s definitely something we need to be very very aware of, that participants will be changing phones during the course of a study, and I think you have to have a very robust plan in place for dealing with that, making it as easy as possible for the participants to get the app back onto their phone so you’re not losing any data, and I think that’s probably a challenge that we haven’t fully come to terms with as an industry, because we haven’t had the opportunity to properly roll out these BYOD studies yet, but that’s definitely a concern.
So this issue of equivalence as I said is not something I feel is a deal breaker, certainly something we have to be concerned about.
AUDIENCE MEMBER: I was just going to mention, in context from a technology point of view, a bit of my previous background’s hat on, and what’s happening in the cloud world at the moment, it’s really not something you need to worry about. You know, I could lose my phone right now, I can go into a shop, pick up another. In fact I could just log in with my laptop and pick up all of the data, every app, everything I’m connected to, because it’s all up there. So transferability across devices is only going to increase. Unless of course we’re talking medical devices, which are still stuck back in the 1960s in terms of design and usability. But in terms of what you’re discussing here, I think there are much bigger things for you to put your creative uses to than worrying about what the technology itself can do to support what you can get out of it.
I think that’s a really good point. You might be missing is actually the patient-centric bit, which is that you are a very tech savvy individual who knows how to easily transfer fro their old phone over to their new phone. Something I want to present in just a few slides. Questions that maybe we need to be thinking just a bit more careful about that. I completely agree the technology is fully capable of doing it, but are the participants.
Which his exactly my next slide. This was the most surprising finding from my point of view, the most interesting finding and I think really highlighted to me the very tech bubble that I live in. I work in a technology company, I’m exposed to very tech savvy individuals. Yet almost half of our participants had issues downloading and installing the app on their own device, which very much surprised me I have to say, and this wasn’t limited to more elderly people or people who rated themselves as less comfortable with technology, this was across quite a range of participants. And I think the reasons they struggled were particularly interesting. For example, forgetting the app store password. I think that’s a really fundamental thing that we kind of forget about. You need to be able to access the app store to be able to get the app onto your phone in the first place. If you don’t know the password to the app store, you’ve kind of hit a bit of a dead end there. There was also issues—again this is more about the technology—of incompatible software which you hopefully would work out through your inclusion/exclusion criteria and good screening early on in your study.
Network issues, a bit difficult to overcome that if the participant can’t access a good network while they’re at the site to download the app, for example, or if they just have limited connectivity at home. Insufficient memory on their phone, so they might have a large number of photos or music on their phone and not have a lot of space on their phone then to download additional app which needs a certain amount of memory to work properly. And then particularly interesting, a couple of participants were able to access the app store and download the app but couldn’t find it on their phone. I think in reality maybe we shouldn’t be too surprised about this, because this was some statistics from comScore that suggested that 65% of US smartphone users don’t download any apps during a particular month. So maybe no surprise that people are forgetting their password to the app store. So kind of highlighting that certainly the technology can do all of these things, but potentially participants are going to be stumbling at things that we—or should I say I—assumed wouldn’t be an issue at all. And just kind of highlighting the point that if you're approaching BYOD, and thinking of using it in a study, you need to have very robust ideas in place of how you’re going to be encouraging participants for example to find out their app store password before they come in for a site visit, to understand how to download apps and get apps onto their phone.
AUDIENCE MEMBER: We just had a usability testing study of a website for parents of children with leukemia. And when usability testing, I was quite surprised by the number of people who didn’t know how to use the capital letter lock to put a password in. So this is something we were having to teach parents time and time again because it was a case sensitive password.
Interesting. Yeah. I think it just highlights the assumptions we make from our day-to-day. So as I say, my day-to-day is working with very technologically savvy people. I have two smartphones, one for work, one personal, and you know, I’m downloading apps all the time. My assumption was that it wouldn’t be an issue. But this actually talking with the patients, or well, members of the general public, actually going out there and running this kind of work really highlights these things that you may have kind of almost forgotten and may not have considered.
In regards to preference of the app to paper, now I should probably caveat this with the fact that the paper version of this symptom diary was horrific, it was basically a stack of A4 pages, it was not user friendly in any way whatsoever. So unsurprisingly, all participants said they would prefer to use the app-based version of the symptom diary versus the paper-based version. But also 18 out of 20 of the participants said they would answer the same on the app as they would on the paper, they didn’t think the switch from paper to electronic would impact how they responded.
Also, seven out of the 20 participants said they would prefer to use their own device to complete the study if they were to use this app for clinical trial. They said they would be happy using their own device for a study. Ten out of the 20 participants said they would actually prefer the larger device. And only one of those people actually owned a larger device, which is to say nine participants said they would prefer to use a different device, i.e. the larger device, to complete the app during a clinical trial, which I think is very interesting when again we’re making the assumption that participants are going to want to use their own device for a BYOD study. If your provisioned device that you’re offering happens to be a very nice shiny large-screen Samsung Galaxy Note for example, potentially participants are going to prefer to get that. So to get the additional device rather than taking advantage of your own device, which is the whole assumption under BYOD as well. So raises some questions again about how you approach that provisioned aspect of a BYOD study. However, the kind of very positive bottom-line finding from the work we did was that all participants were able to interact with all versions of the app across all different screen sizes. So it was user friendly across a range of different screen sizes.
Just to touch on the issue of the availability of suitable devices and the possibility that not all participants are going to have a suitable device, I think this statistic from Pew Research Center, and I this is 2014 so these might have changed slightly. But they, in the US, reckon that 64% of all adults within the US have access to a smartphone. What was out interesting from my point of view was this split by age groups, where you have 18-29 having 85% having access to a smartphone—I was actually kind of surprised that was so low. But then 65+, only 27% having access to a smartphone. I think it’s important to reiterate the difference between a cell pone and a smartphone, you can have those candy bar phones or those flip phones, obviously saturation of those is much higher. Specifically talking about smartphones, ones that can take advantage of internet-enabled technology such as apps.
So if your study is having a range of age groups within it, which is pretty common in clinical trials, you’re really going to have to try and get a good understanding of not just the ages, but also demographics. Again, statistics which were very surprising for me. With people who earn $75,000 or more a year, 84% having smartphone, whereas those on less than $30,000 a year only 50% having a smartphone. Again that’s very big difference. And you’re going to see very big differences in the people presenting to your clinical trial in the availability of a suitable device. So it raises questions around, first of all, their comfort using an app-based system, but also how many provisioned devices you’re going to be providing to participants and how you’re going to calculate that. This calculation gets quite complicated when you start thinking about that breakdown of technology and availability by demographics. And also, availability by geographies. So this is again from Pew Research, the lowest 20 countries in the world I think, China came in at only 37% saturation of smartphones, something I found very surprising, but raises questions if you’re running a global clinical trial. We’re not running all our clinical trials in the UK and the US where smartphone saturation is relatively high, although we saw the differences across demographics. If you’re running these global studies, you have to take into account that smartphone saturation is not nearly as high there either.
That’s just to highlight that, in the most technically savvy country, you’re never going to be guaranteed 100% saturation in any decent sized population you’re going to be studying, so you’re always going to have some kind of robust plan in place for providing a provisioned device. And as has already been pointed out, you’re going to have to have some plan in place for helping participants who have either lost their device or move on to a new device, and there definitely is technological solutions to that, but they have to be user friendly and they have to be intuitive for participants.
This is research I saw summarized recently from Dr. Keith Meadows and DHP Research and Consultancy, and took an interesting approach similar to what Paul did, which was presenting research from different fields, but who are doing very similar things. So Paul was talking about the insurance industry, where potentially there is quite a bit we could learn about accessibility. This was from marketing, so getting feedback from participants for various marketing surveys. Summarizing some of the findings from a lot of the work they’ve done. Obviously marketing is a huge field, and there’s been a lot of data already generated. But at least in the marketing field it seems that response rates are lower with using a mobile device rather than, for example, a PC-based web system. I think one of the other interesting findings was break-offs, so failure to complete the survey are higher among smartphone users, and I think this is something we don’t have any hard data on when it comes to BYOD, but I think it would be very interesting to see, is there an impact on the compliance when you’re using a BYOD model where participants are using their own device versus a provisioned device, for example. And smartphone users take longer to complete a survey compared to a computer-based device. I think that’s maybe something we’d expect. But again, an interesting finding that suggests that there’s additional data out there that we can take advantage of to really inform our approach to BYOD.
So I think some of the outstanding questions that we have for the BYOD model. I think it is the future, at least in certain aspects of clinical trials, of certain types of clinical trials. And I think the equivalency, usability, and acceptability, generally that’s very positive. I think there’s still outstanding questions. I don’t think we’ve answered the equivalence question to the satisfaction of the regulators, for example. I think they’ll want robust statistical evidence for that point, which is kind of understandable. But I think from the usability and acceptability point of view, generally I think the BYOD app-based approach makes sense. However—and I think they’re quite important howevers—issues around technical support. how are you doing to support the patients from a technical point of view when you don’ know the exact phone or device they’re using. And you can’t just push that support on sites. You have to also support the sites, because the sites are at the end of the day members of the general public as well and may not be as comfortable with technology as your study team. So you really need to have robust plans in place for supporting participants in how to get apps on their phone in the first place, for example, as we saw that was a surprising challenging thing for a lot of our participants to do.
I already touched on this, but how will compliance be impacted. I think the assumption is that BYOD will push compliance upwards. We’re already at about 90% compliance when it comes to at-home diaries with provisioned devices for participants. We don’t have much further to go in regards to increasing compliance. I think it’s probably a safe bet that compliance is going to go down. But as we already saw with the mobile marketing data, there’s a suggestion that maybe you get more break-offs when participants are using their own smartphone devices, so more participants not completing the entire questionnaire. I think there’s probably some interesting work to be done there to really understand the impact on compliance of using an app-based BYOD model. And how will this patient preference piece, so whether participants will want to use an app on their own phone or actually et the provisioned device, how will this actually play out in the real world when you do have a provisioned device available for participants versus getting them to download the app on their own phone. Again, the assumption is that they will want to use their own phone where possible. And that feels like a safe bet, but we don’t know, we don’t have the real world evidence to suggest that that’s going to be the case, and that’s obviously going to impact how many provisioned devices you’re going to have to then provide for your particular study.
But I think these are all questions that can be best answered by talking to the patients first of all, which is what we’ve started to do in CRF, getting that feedback from participants to really understand their experience using an app. Running these kind of pilot studies. But also looking at some existing data that’s out there from different sources, so from insurance market which has already done a lot of work, from the marketing market which has already done a lot of work. There’s evidence already out there for participants and people using apps on their smartphones in very different cases, but that can be applied to BYOD and clinical trials. So I don’t think any of these are showstoppers for BYOD, I just think they are issues that should give us pause for thought and should open our eyes to the fact that we really, again, need to be having the patients at the centre of our thinking when it comes to BYOD. Just because it makes sense to us as tech savvy people who work in this industry, doesn’t necessarily mean that it’s automatically the best solution for all participants in a clinical trial. We need to be just a bit more careful in our thinking about it.
And that’s it. Did anyone have any questions?
AUDIENCE MEMBER: So you talked about compliance most likely going up. One of the concerns that some people have is that if you’ve not got a separate device that has an alarm enabled, patients might be more forgetful and might forget to complete it. I mean, if it’s your own smartphone you can just mute that alarm, you don’t have to—you can make it so it’s not going to disturb you. How do you think you’d get around that issue and making sure that patients don’t forget? I mean if I had another device and I knew, oh I must remember that, I think I’d just remember it more than remembering to complete an app on my own phone. So how do you think that you would get around that?
Yeah that’s a great point and I think you raised what I think is probably one of the key challenges raised against this compliance question when it comes to BYOD, the fact that we don’t have complete control over the device the participant is using when it comes to things like alarms. They can just turn the sound off, so alarms mightn’t trigger. I feel like one of the key ways of addressing that with so many things in this field that we’re working in and clinical trials in general is more around an educational piece, getting the patient to understand why this data is so important, why this alarm goes off every morning, which could get quite annoying, but there’s a reason why we’re asking the participant to do that. There’s a reason why we’re reminding the participant to do it every single morning. And you’ll also see from the data whether participants are being compliant so you can flag up patients who maybe their compliance is stating to slip and get sites to follow up with them to see if there’s any issues, to see are they struggling with the app for example, and encouraging them to stay compliant. So again I think it’s something we don’t have an answer to, we don’t know how participants will act with the app on their phone, particularly when we are triggering things like alarms. So it’s something that we’re yet to get a full understanding of.
AUDIENCE MEMBER: So I know from working with instrument authors, they can be particular. And so if they’re not willing to adapt to BYOD, which I guess some probably would not, do you think that’ll lead to new instruments being developed purely electronically that’ll become the lead measure of the endpoints?
Yeah, really good question. We have come across some authors already who have made clear that they are not comfortable with app-based versions of their questionnaires being developed. We have come across some others who are very proactive and are moving forward with developing app-based versions of the questionnaires as well. I think, exactly as you say, it’s one of those things that will just evolve as with a lot of things we’ve been taking about today and will continue to talk about, these things are just going to evolve over time. And if BYOD, as I believe it will, becomes more and more mainstream, if you want your questionnaire to be used in a clinical trial—and very often this is a source of revenue for people—you will need to have an app-based version of it or allow companies, vendors like CRF, to make app-based versions of the questionnaire. And if you’re not willing to do that for maybe concerns around psychometrics or whatever, then alternatives will come along. I think it’ll be kind of the brutal hand of the market will deal with that in the long term. It’s a great point, though, that some authors are iffy about it.
AUDIENCE MEMBER: Probably just in response to that. Most authors are more worried about the integrity of their instrument. So revenue is probably a second or third item. They would never admit to that anyway. But they are already doing that today without knowing that they’re doing it, because there is like seven, eight, nine, ten different vendors out there, and they all change the hardware over time. So implicitly they already have provided different versions of the instrument on different systems, different operating systems, different screen sizes. They may have tested each individual one—some do, some don’t. And that’s the work that we’re doing with you and with others, to basically bring them around to acknowledging that they have been doing it without knowing that they’re doing it. So once we put that together and can show like the publication that we did and do a couple more, I think we can show that the instrument or the instrument’s integrity is not at risk by providing eCOA BYOD solutions. And once we have that established, which we’re close to, I think it will be not a big issue.
AUDIENCE MEMBER: So I just had a question about, so you mentioned you know the hype of BYOD, but what do you think we have to do to make it reality? There’s all these kind of conversations you know, the regulators haven’t come out with that much. But is it the responsibility of sponsors to ask the regulators, or is it the responsibility of vendors to do more studies. And if it is the responsibility of vendors, if you think equivalence isn’t a real question anymore, or that we might need more evidence. What do you think we should be doing next in studies.
Yeah, so it’s a good question. I don’t think we can wait on the regulators, because I think the regulators are basically waiting on the industry to provide them the evidence that they want to show that BYOD is a viable solution, at least in certain cases. So I think it’s going to be a case of the vendors and the sponsors working together, running small-scale pilot studies, doing the kind of research that we’ve done with ICON. But then also potentially sponsors taking the risk to include BYOD as an element of a study for example, as maybe one arm or one country within a study or as part of a late phase where the regulations are maybe less strict. To capture the data to show that it works, and also to address some of these questions around the more logistical side of things. This is how we’ll do the provisioning to make sure no participants are left out of the study. This is how we’ll manage the change in devices mid-study, for example. So I think it’s probably reliant, and the risk—even though I don’t see it as being a huge risk—the risk lies more with the sponsors probably to actually decide to do that, to include BYOD as a part of their study, but obviously with the support of vendors to make that actually happen. You’re right that I don’t think the equivalence is a showstopper issue. I do still think we need to demonstrate that to the regulators statistically. But again that’s data that can be captured when you actually run a study, particularly you know if you’re just including BYOD as a part of a study and then you have the option of comparing to whatever else is used to collect the data in the study with provisioned devices. So I think it’s more maybe the logistical piece that also needs to be worked out by actually running the study in the real world to really see what happens when you deploy BYOD in an actual clinical trial.
AUDIENCE MEMBER: Just one little window of opportunity that was given in this morning’s talk in terms of personalization and how much we can adjust screen appearance, view, and interaction and so forth. And given the discussion here about these validated questionnaires and authors. And we also have a space of diaries, I’ve been involved with, that is not a validated questionnaire as such but it’s more factual data that is being collected, such as voiding or other events. So more facts, that has a little bit of a wider, a more open space for actually being adjusted, adopted in various different ways. So that also gives kind of the questionnaire author out of part of this discussion, but also again what you could be allowed to do in terms of personalization or adjustments on a screen. And then just another—it’s very practical question or challenge that we have run into, or I think need to be there is actually the cost of data transfers. You mentioned connectivity here for downloading the app, but then you also have uploading. Given what we pay currently honestly for data uploads and downloads, given the deployment. That cost will still be there one way or the other. Adoption and pickup of smartphones is one parameter but that necessarily it links to can you download an app, can you use it, but also do you have the data, upload, download in place, or still associated with cost, where can that be transferred, how can we handle that, which is a very practical challenge. Not that we shouldn’t go there, it can always be overcome, but one that we need to consider.
Most definitely, no I think that’s a very fair point. And again hearkening back to the patient point of view, we can’t have patients being out of pocket for taking part in our studies. And there are ways of calculating how much data is being sent and received by an app and the typical kind of data we’re sending because of the way it’s coded it’s just ones and zeroes basically it’s absolutely tiny. But at the same time, you’re right, we can’t have participants out of pocket. So I think there will have to be ways of reimbursing participants for maybe the average amount of data that’s expected to be sent for a study for example.
AUDIENCE MEMBER: So Paul, in the way that we now see clients come and ask for paper backups, do you think that as eCOA becomes really established and people go just for an electronic use, that that will then shift to being the paper backup and BYOD will be the future. Do you think people will transition away and move into another?
Yeah, I think that’s very possible. I think that’s kind of how we see technology evolve in general. I mean I think this whole issue of equivalence between paper and electronic was the big issue in eCOA for a number of years. Now that BYOD is being suggested as a viable alternative, we’re kind of moving away from that and now focusing on the issue of equivalence between different devices. I think we give it another five years, something else will have come along, whether data capture via Google’s brain chip is equivalent to data you get on your smartphone. So as technology evolves, the concerns shift along with it. And I think we hopefully will start to move away from paper backups, maybe in fact having BYOD as the paper backup. I think technology will address that desire that sometimes sponsors have to include paper backups, sometimes for quite strong reasons as we heard earlier. But I definitely think it’s an evolution. Technology is constantly evolving at terrifying rates that very often we don’t keep up with. But yeah, I think we will hopefully see that aspect being addressed.
Okay. On that note, thank you very much.
[END AT 38:10]