Using Predictive Modeling to understand the impact on assay sensitivity
The placebo response is a heavily studied and historically challenging phenomenon for drug developers. Strong placebo effect diminishes the ability to distinguish efficacy of an experimental drug, leading to phase II and III trial failures1– even for otherwise effective drugs.
Researchers have long devised strategies to overcome this source of noise in trial data analysis, starting with identifying high placebo responders. Historical attempts to do so involved excluding patients based on placebo run-in periods, in which patients showing an improvement upon administration of the placebo during the pre-study period were excluded from the study.
Though this strategy makes sense at surface level, its real impact on assay sensitivity and study precision and power have yet to be proved. In fact, there is already a strong case against excluding high placebo responders from clinical trials. This strategy limits the available patient population, making recruitment slow or, for rare diseases, even impossible, increasing overall trial costs and timelines. It also decreases translatability of data to the general population, an important requirement for regulatory agencies.
Despite this, the strategy remains tempting as the magnitude of placebo response and rate of trial failures continue to increase. Understanding the effectiveness of this strategy is difficult and traditionally has been estimated retrospectively by meta-analysis, which have shown that excluding placebo responders identified by lead-in phase fail to either decrease the magnitude of the placebo response2, nor increase the assay sensitivity (ability to distinguish drug response from placebo response)3,4. The advent of predictive modeling tools based on machine learning has, however, enabled the identification of placebo responders at baseline without a run-in period in both placebo and drug-treated patients. This provides an opportunity to take a new look at this important question.
We recently ran simulations to evaluate whether excluding strong placebo responders improves clinical trial assay sensitivity, as measured by the precision of the treatment effect estimation, using data from a Phase II randomized controlled trial in knee osteoarthritis. This analysis used the Placebell©™ approach, in which each patient’s expected placebo responsiveness was calculated using patients’ baseline characteristics – including personality traits, demographics, history and severity of disease.
The impact of excluding the top 5%, 10%, 15% and 20% of placebo responders was compared to exclusion of patients at random. Random exclusion resulted in a decrease in precision that is proportional to the number of patients excluded. The selective exclusion of high placebo responders only resulted in a marginal gain in precision compared to random screening. For example, removing the strongest 30% placebo responders increased precision from 0.69 to 0.75 compared to random screening.
These results were then compared with covariate adjustment, in which the predicted placebo responsiveness value calculated by Placebell©™ is used as a baseline covariate to account for the range in placebo response in a diverse trail population. The covariate adjustment increased the precision of the treatment effect by +37%. Combining covariate adjustment with exclusion of placebo responders did not further improve the precision of the treatment effect estimation.
These results support the experimental observations that excluding placebo responders has limited benefit, while accounting for placebo response-related variability can improve assay sensitivity. The covariate approach instead improves study power and the probability of study success without analytical or regulatory risk. Indeed, this approach is consistent with FDA and EMA guidance on the use of baseline covariates.These data were presented in a poster titled “Should placebo responders be excluded from RCTs?” at the ISCTM 2021 Autumn Conference, September 30 – October 2, 2021. Click here for the poster abstract and contact us for more information.
References:
1. Dumitrescu TP, McCune J, Schmith V. Is Placebo Response Responsible for Many Phase III Failures? Clinical Pharmacology and Therapeutics. 2019;106(6):1151-1154. doi:10.1002/cpt.1632
2. van Seventer R, Bach FW, Toth CC, et al. Pregabalin in the treatment of post-traumatic peripheral neuropathic pain: A randomized double-blind trial. European Journal of Neurology. 2010;17(8):1082-1089. doi:10.1111/j.1468-1331.2010.02979.x
3. Trivedi MH, Rush J. Does a placebo run-in or a placebo treatment cell affect the efficacy of antidepressant medications? Neuropsychopharmacology. 1994;11(1):33-43. doi:10.1038/npp.1994.63
4. Lee S, Walker JR, Jakul L, Sexton K. Does elimination of placebo responders in a placebo run-in increase the treatment effect in randomized clinical trials? A meta-analytic evaluation. Depression and Anxiety. 2004;19(1):10-19. doi:10.1002/da.10134
