Evaluation of methods for the inclusion of real world evidence in network meta-analysis-a case study in multiple sclerosis

Objectives: Network Meta-Analysis (NMA) is becoming a key component of submissions to reimbursement agencies world-wide, especially when there is limited head-to-head evidence for multiple technologies. However, almost all NMAs only consider Randomised Controlled Trials (RCTs) even though there may be considerable Real World Evidence (RWE) available - for example observational studies or registry-based studies. Evaluation of methods to enable the inclusion of RWE, especially in the light of the changing nature of RCTs from both a regulatory and reimbursement perspective, is considered here. Methods: RCTs and RWE studies were searched for using standard filters and databases up to, and including, the regulatory approval of Fingolimod by the European Medicines Agency (EMA) in 2011 for Multiple Sclerosis (MS). A number of NMAs were then conducted and which included; only RCTs, both RCTs and RWE (accepted at face-value), both RCTs and RWE but including an additional level in the NMA hierarchical model to represent the different study designs, and finally both RCTs and RWE but adjusting the RWE for potential biases. Results: Identification of RWE in addition to RCTs in this MS example significantly increased the number of studies (and comparisons) that were potentially included in the NMA. Whilst the inclusion of the additional RWE led to a reduction in the level of uncertainty surrounding most effect estimates, this depended on the method of inclusion adopted for the RWE, and the extent to which biases were adjusted for. Conclusions: This initial evaluation of methods for the inclusion of RWE in NMAs indicates that methods of adjustment for the potential biases in RWE can have a significant impact on the level of uncertainty. Consequently further work investigating both empirical evidence for such biases and methods of elicitation from experts on the extent of biases associated with individual RWE studies is warranted.