We selected national agencies that provided public information on the drug assessment process on their web pages, thus making it possible to collect the data needed for the analysis. With this aim, we searched European agency websites between January 2015 and October 2019.

Once the agencies were selected, from among the drugs assessed we chose the hematological cancer drugs for which the drug company had presented information on a MAIC comparison in their documentation.

Information was collected on whether each of the chosen drugs had been assessed by any of the 3 selected agencies. If this was the case, we recorded the date of the assessment, whether the pharmaceutical manufacturer had presented MAIC information to the agency, whether the MAIC data had been analyzed by the agency, and whether the results of the indirect comparison had been taken into account or discarded during the assessment process.

MAIC methodology is based on using the available IPD from a clinical trial and weighting them such that the mean of their baseline characteristics and standard deviations are consistent with the aggregate data of those reported in the literature for the alternative treatment of interest. Thus, the results of both treatments can be compared between balanced populations. Matching is achieved as follows: patients in trials with IPD are re-weighted by their odds of being potential participants in trials for which we only have published aggregate data8,9.

Comparisons are considered to be anchored if a placebo or a common comparator treatment vs the alternatives assessed is used. They are considered to be unanchored if direct comparisons are made between two treatment arms.

This methodology is only valid if the studies to be included in the indirect comparison have been identified via a systematic review of the literature. Following this step, the baseline characteristics to be weighted are determined. Finally, these balanced averages are used to compare IPD studies to the aggregate data from the studies addressing the alternative treatments.

Regarding sample selection, there should be no protocol differences that could affect the results and that cannot be balanced by matching. Trials with available IPDs should have similar or more inclusive inclusion/exclusion criteria than the aggregate comparison studies. In addition, there should be consistency between the type of results reported and the way they were obtained in the different studies10.

The baseline characteristics can be matched if the data for each baseline characteristic in the aggregate study are available in the study from which the IPD are taken.

The baseline characteristics from trials are matched as follows: individual patients from trials with available IPD are re-weighted to make the means and standard deviations of their baseline characteristics match those obtained from studies with only aggregated data available.

Thus, each trial is characterized with 4 component vectors [X, T, Y, Z], where X is also another vector describing baseline characteristics (e.g. age, sex, ethnicity, or previous treatments), T is the treatment received (T = 0, treatment with IPD; T = 1, treatment with aggregated data); Y is the result of interest which can be binary (e.g. response to treatment: no = 0, yes = 1) or continuous (e.g. progression-free survival or overall survival); and Z is the placebo arm (Z = 0) or the treatment arm (Z = 1)10.

The IPD study will have a vector (xi, ti, yi, zi) for each individual patient, whereas the vectors corresponding to the patients in the aggregate data study will be (x¯, ti,y¯, zi) because all these patients will be described by the mean of their baseline characteristics (x¯) and mean results (y¯).

Differences in the treatment effect (T = 0 vs T = 1) can be estimated from the weight wi, which is obtained as the odds of patients in the IPD trial participating in the aggregate study vs the odds of them participating in the IPD study. These estimated weights (wi) are used to ensure that the mean baseline characteristics are balanced between the two trials.

Thus, the following formula is used to calculate differences between treatments:

Where the first term represents the effect on patients receiving treatment in the IPD study, the second term represents the effect on patients receiving placebo in this study, and the third term represents the difference between the average effects of the treatment and placebo arms in the aggregate data study.

This estimator is effective if the baseline characteristics vector contains all the confounding factors. In this sense, the inclusion of the placebo arms is useful in that missing confounders can be taken into account10.

The main limitations of MAIC methodology include its inability to adjust for differences regarding treatments (e.g. dosage, administration, or co-medications) between trials and the impossibility of ensuring that all prognostic and effect-modifying variables are known or available. Thus, adjusting the population in an indirect unanchored comparison requires that the absolute results can be reliably predicted such that the degree of bias due to the imbalance from potential covariates that have not been controlled is acceptable. Any estimates or conclusions obtained from unanchored comparisons may be strongly challenged because the magnitude of the bias in these estimates is unknown, likely to be substantial, and may even exceed the magnitude of the estimated treatment effects.

The websites of the assessment agencies were reviewed and selected for analysis based on their fulfilling the inclusion and exclusion criteria. Three agencies were selected: The British National Institute for Health and Care Excellence (NICE), the French National Authority for Health (HAS), and the German Institute for Quality and Efficiency in Health Care (IQWIG).

In total, 13 treatments for hematological cancer disease were found on the websites of the three agencies. All pharmaceutical manufacturers had included information on a MAIC comparison in their documentation. Table 1 shows these treatments, the agency which presented the documentation, and the date of the assessment.

The HAS only received a MAIC for five treatments in their documentation: of these, only two were considered for the final assessment. The IQWIG only received a MAIC for three treatments, but none of these were used. Finally, the NICE received a MAIC for eight treatments, half of which were used in the assessment. Of these, 50% of those assessed by the NICE and 40% of those assessed by the HAS were considered by the agencies during their decision-making process on their inclusion and therapeutic positioning in the treatment sequence. None of the three MAIC comparisons received by the IQWIG were taken into account during decision making.

Figure 1 shows the assessment by the HAS, IQWIG, and NICE of these treatments with MAICs.

Figure 1.

Assessments of the 13 selected treatments.

(0,24MB).

All the MAICs included as documentation for the assessment processes were conducted without anchoring. The only exceptions were the MAICs included with bosutinib for chronic myeloid leukemia and inotuzumab for refractory acute lymphoblastic leukemia, both of which were received by the HAS. Although inotuzumab was taken into account in the assessment, bosutinib was not because the comparator was considered inappropriate.

Of the 16 assessments including MAICs, only three drugs were assessed by more than one agency. Of these three drugs, brigatinib and ibrutinib were assessed by the NICE and IQWIG, whereas venetoclax was assessed by the NICE and HAS. The remaining 10 assessments that included MAICs were assessed by only one agency; the HAS assessed avelumab, bosutinib, inotuzumab, and tisagenlecleucel; the IQWIG assessed pembrolizumab; and the NICE assessed carfilzomib, cemiplimab, ceritinib, daratu-mab, and lutetium (177Lu) oxodotreoxide.

Of these drugs, avelumab (HAS), carfilzomib (NICE), and daratumumab (NICE) were not considered for assessment because of problems related to the characteristics selected for matching. In the case of pembrolizumab (IQWIG), the MAIC was not considered in the assessment because it did not have sufficient statistical power to determine if the observed differences were caused by bias. In the cases of bosutinib (HAS) and cemiplimab (NICE), the MAICs were not considered in the assessment because they used a comparator that was considered inappropriate for the assessment (i.e. the indication of the comparator drug was not approved in the country at the time of the assessment).

Of the treatments assessed by more than one agency, the NICE and the IQWIG made the same decision to not to use MAIC in their assessment of ibritinib because in both cases the pharmaceutical manufacturer did not include the data corresponding to the comparison made.

In the cases of brigatinib (NICE and IQWIG) and venetoclax (NICE and HAS), indirect comparisons were taken into account by the NICE, but not by the other agencies. The IQWIG assessment of brigatinib did not take the MAIC into account because of a decrease in the comparator dose (ceritinib) at the time of assessment in Germany. The HAS raised questions about the selection of the characteristics used for matching during the assessment of venetoclax.

Figure 2 shows the reasons for excluding the studies that were excluded discarded.

Figure 2.

Causes for exclusion from the comparison.

(0,09MB).