We’ve been talking about the future, and I thought I’d kind of kick off the questions by asking you folks to sort of look at the future from the perspective of where we actually are right now. So starting with Shimon, are there places in the PRO development process that you’re proposing where the regulatory bodies actually might have some objection?
Yeah, so I think the regulatory bodies don’t know how to deal with some of the data generated by alternative methods in PRO development or validation yet. And I think that calls for us to do some comparative methodological research to see, hey if we were to do this in a traditional manner with this patient population or if we were doing it online, interviews versus free text questions or any other methodological change, we’re going to need to kind of benchmark those side by side. And I think we’re going to find that there are cases where different methods work better. There are probably populations, as Paul pointed out, there are certain environments where new technological approaches offer a natural advantage that might apply for certain various sensitive topics that it’s easier for a person to complete in an environment where they don’t necessarily have a face-to-face relationship with the clinician, say. And there are going to be others where the traditional methods work better, and we need to study this.
I want to pass the mic to Paul. We have an industry that’s conservative. And it’s slow. It can take years to conduct a trial. How do we cope with consumer devices that are lasting a year or less if we want to do this electronically?
Yes, good question. We’re already running into that problem in the work that we’re doing today. We’re talking about equivalence, for one thing. And when we’ve decided where we stand on that debate, then we will probably have already decided that we can accept a certain degree of difference between devices and be confident that there’s no problem with it. That means that when, for example, your Android device that you design something for today suddenly goes out of circulation, which will happen very soon, you’re going to be okay because of the similar one that you’ve got confidence in. So we would need to understand where that ends, you know, as an industry. We need to be able to define where we’re not any longer comfortable with the difference that we’re seeing. But generally, I’m less frightened than the question implies by the incredibly rapid change of technology, because the main component doesn’t change as fast, which is ours, which is the human body and the expectations change, but slowly. So it’s not something that will tear away and leave us unable to gather data anymore. I think as long as we’re prepared for a period of being fluid, of as an industry developing effectively a digital strategy of understanding where we stand and what we’re comfortable with, then this period of invention will be very fruitful. Nothing too much to be afraid of.
And Jason, the whole rare diseases aspect of things, where you really are finding your way. Each one of the steps in development through your studies has questions related to them. And it’s very step by step. So how do you plan? You want as an industry to look at start to finish and to make some plans with lots of reasons. How do you do that when each of these steps is a question mark?
So I think that when we do a PRO development we know that there are going to be questions or steps along the process that are going to alter the next set of steps that we do anyhow. So we know that we’re going to start out by looking at the literature and talking to the experts. Then we’re going to go in and talk to patients and then we’re going to build a tool and see if the language works, and then we’re going to go in and do some psychometrics on it. Those remain the same. The details around them are going to alter. But they do now. So I think that if we demonstrate that there’s a path forward that’s very similar to what folks are used to using, but that there are different decisions or different methods that may have to be employed, and having a good idea of what those may be, you’re going to give more comfort to the client that what you’re ultimately building for them has a semblance of logic to it. Though you’re still willing—and having that up-front conversation that we need to be adaptive here for the rare situation. Now, after you go through one or two of these, you’re going to know a little bit more about what to expect and be able to provide a little better guidance, but there’s still going to be things that come up that you’re like, wow I didn’t see that coming, okay we have to go to Pluto to be able to measure these people. Whatever the case may be, you know, expressing that need for adaptability up front and giving them comfort of, we’ve done this before, we kind of know the big things that are going to happen, but when something shocking comes up we’ll work through it together and we’ll have a good plan.
Okay, any questions?
AUDIENCE MEMBER: So what is your thinking, that there will be an electronic format in which, when we develop a new tool now, 25 years from now, when somebody goes for that tool, it will be easy for them to deploy it in the technology that is going on at that period. I thought through this and I wanted to ask, and I think this time provides the opportunity. And if any of you can just address that, that would be very good. Thank you.
That’s a very good question. And if you think back to when I was making the parallel between ourselves and the electronic publishing industry, that’s precisely been the theme of the past five years or so. You might be writing some content that may not be life threatening, but it’s still very important to you. And you want it to live more or less forever, you want it to be able to be consumed and understood in the same way in whatever device is available in 25 years’ time. The separation of the content from the platform upon which it is placed has to do with the structuring of the data and making sure that, agreeing as an industry on what constitutes a well structured piece of content and later on your devices will be able to display it. Interestingly, I worked previously in insurance, I’ve done some work for some insurers, who will put a questionnaire out about your circumstances, and they will or will not insure you for whatever it is. And those questionnaires go under almost as much scrutiny as the ones that we’re talking about here. We could refer to them as instruments. And yet nobody—nobody in that industry has even thought that there might be a problem in moving those instruments from one place to another and showing them. It has never even occurred to anyone that it is an issue. So I keep reminding myself of that and thinking that we are probably looking several years down the line, we are looking at being able to display any of this content and being sure of the same, you know, that we understand the same thing when we read it or hear it.
I want to add when you look at time horizons of 25 or 30 years down the line, it’s very hard to predict what will be happening, you know, what sort of tools people will be interacting with. I think it’s pretty likely that people will still be reading and that written communication will still be useful here. But you know, if you look at how we use IRT right now, that has a very different flow for a questionnaire. And I can imagine very different psychological or statistical models that allow completely new kinds of interactions where maybe a PRO takes the form of something that from the patient’s perspective feels a lot more like a verbal dialogue with another person, probing on specific points. And I think that that’s a kind of interface technology that might develop, and I can imagine many other things and, you know, different pools of artificial intelligence resources that could be applied to PROs in novel ways. So I think we’re going to get there very gradually. I don’t think any of us can really predict where it’s going to land at that time frame. And because of that, I think we need to acknowledge that the instruments will need to be able to evolve, and we need to have a framework for collaboration and for the evolution of these tools to get there.
I don’t know that we’ll be at inter-cranial reported outcomes yet, but I agree with my panel experts her that, I think that a lot of what we have right now is going to remain the same. But I would expect the IRT advancements with adaptive testing. I come from educational measurement, and if you look at what our tools are doing, we’re basically 20 years behind educational measurements. So look at what they’re doing now and think of that now, but I also think that we’ll probably be doing a lot more extra-sensory data collection with the, you know, as we were talking about, you know, wearing some kind of device to collect a bunch of other information that goes along with our patient-reported outcomes. And I think that we’ll find some ways of making sure that the way we capture the information is just as easy and natural as possible. How that’s going to be captured, I think that I would agree it’s very hard to predict.
If you look back at the movie industry, as Paul was showing, and the advent of the close-up and what happens after that, what tends to happen is it’s built upon, you add to it. And so if you think about starting at a point here, your technology doesn’t preclude that as it advances, it just adds to it. So I would suggest that that’s what would happen, and I think you guys all agree with that. As you said, people will still be able to read, at least I hope so.
AUDIENCE QUESTION: This goes to the first presentation. So for patients who are registering themselves as part of a network to try and keep stock of their own disease and progression, once they enter a study, does it change the patient experience for patients who may be continuing to track themselves and, for example, the DLQI, if they go in and rate themselves daily as opposed to other patients in the study who they only will report that, you know, once every three months or whatever the sponsor designs, is there anything that we need to be concerned with as a sponsor to make sure that the patient experience stays the same?
So that’s an interesting question. I think the parallel thinking about that is, do you as someone operating a clinical trial need to be concerned with, say, what kind of medical care a patient might be receiving outside of the trial. And I think there’s certainly cases where that could be an issue. I don’t know whether a specific thing like reporting on the frequency of a PRO changes its meaning. As much as possible, when we do involve PROs in specific studies, if it’s useful in the context of a study to gather that data on some particular period, it’s probably useful to do that on a continuing basis outside of the study. So we would try to reconcile those things as much as possible. But yeah, I mean depending on the goals of the study, we would need to look at kind of those specific interactions.
Am I answering your question?
AUDIENCE MEMBER: No, you are, and this—I’m thinking of it also from the point of view of an inclusion/exclusion criteria. Now that there are more of these types of places where patients or potential patients can go and keep this information for themselves and also work and print out this information for their doctors, is that something we should maybe consider as an exclusion criteria or something to ask the patient if they are part of one of these other registries or networks.
Yeah, well I think it’s worth considering. You know, the fact that patients can find each other and go online, not just to our site but many others, and find, okay here are other patients reporting on treatment on this drug, which is actually an experimental study drug that, you know, they might be able to do things like unblind a trial in progress, and that’s certainly something that merits serious consideration in the design of a trial. You know, I think that can be a long and interesting discussion on its own, and some of that might come down to really setting up kind of a different social contract with study participants, where there is a certain expectation of confidentiality until a certain point. And that’s also going to raise all sorts of ethical issues about what information can and should be hidden from patients during a clinical trial and things like that. And I don’t think I know all the answers to that at all, but it’s a very interesting subject.
Sorry just quickly on that, if we’re talking about social media as a channel for people sharing information about what they’re going through. I’d like to see whether or not they share things which would bias any answers that they give. I would suspect them to be more likely to share things like, you know, it’s great to know there’s someone else going through this with me, it gives me a good feeling to know that you know what I’m experiencing, this kind of thing, but not the kind of thing that is likely to bias the responses. I think it’s worth doing some study into that, what kind of things do people share when they’re able to share. It’s probably not how to answer the questions. Worth doing some study on that, but it’d be a very difficult study to design though.
AUDIENCE MEMBER: Given the FDA’s more relaxed and flexible approach for rare diseases, could you argue that maybe they should take that approach for BYOD, because there are lots of benefits for BYOD.
That’s a good question. I think that there’s an empirical difference between the two, where we have or can procure the data to demonstrate the efficacy of BYOD, whereas you simply may not have the availability of patients to demonstrate it with a very rare disease, an ultra rare disease if you will. So I think that that’s going to be one core difference between them. In my opinion—and opinion is a big word there—I think that there’s also a bit of an emotional difference, in that they want to help patients with rare diseases get access to new and potentially curative drugs, etc. whereas if you talk about the ePRO argument, I think that they feel that this is an industry issue and not a patient issue. And so I think that there’s a big difference in their mindset on how easy they will be willing to make it for each of those two sides. There are by the way—as you probably have seen—a lot of rare disease methodological groups starting up. I believe ISPOR has one now, and there are some other groups that are starting up, to help provide more guidance. Because ultimately, although a lot of the FDA guidances touch upon rare diseases. When it comes to patient-reported outcomes and rare diseases, the language is like three sentences. It’s very minimal and there needs to be, eventually, more. But frankly, in this regard I hope they come out with more clear guidance on BYOD before they do on the rare diseases, because I think right now we’re all happy with how they’re working with rare diseases.
[END AT 18:30]