GENESIS group of Spanish Society of Hospital Pharmacy works collaboratively in drug evaluation since 2005 using its MADRE method. This paper provides a review of economic aspects of 32 one-drug evaluation reports done collaboratively.

Some improvements can be proposed to facilitate drug economic evaluation in MADRE program, e.g. establishing criteria for drug price selection, decision making under uncertainty or cost-effectiveness thresholds in different situations.

Drug evaluation and selection is one of the key tools for establishing drug policy in hospitals. New drugs are evaluated by Pharmaceutical and Therapeutic committees in order to decide their possible inclusion in the formulary and the criteria for their use. This activity has been widely developed in Spain1 and in other countries as well2. Since the start of this activity, hospital pharmacists have been the leaders and they are the main writers of the drug evaluation reports used to support the decisions made by the Pharmacy and Therapeutics Committees.

Approximately 10 years ago in Spain, there was no centralized state initiative for drug selection so a horizontal collaborative system for drug evaluation was initiated. This system was conducted with the voluntary participation of hospital pharmacists throughout Spain. The GENESIS group (Grupo de Evaluación de Novedades, Estandarización e Investigación en Selección de Medicamentos, meaning group for drug evaluation, standardization and research in drug selection) was then created within the Spanish Society of Hospital Pharmacy (SEFH). The group was made up of pharmacists who were widely experienced in drug evaluation.

The GENESIS-SEFH group established a method for drug evaluation reports called the MADRE method (Método de Ayuda para la toma de Decisiones y la Realización de Evaluaciones de medicamentos, meaning method for assistance in making decisions and writing drug evaluation reports)3. The first version was published in 20054. A study conducted in 2006-2007 showed that MADRE was the reference method used in Spanish hospitals for this purpose5 and in subsequent years, it was also adopted by drug evaluation groups from different health services and regions6–11. At the end of 2012, a new version of the MADRE program was published3. Aspects such as disease description, data extraction in survival analysis, indirect comparisons, therapeutically equivalent alternatives and certain aspects of economic evaluation and budget impact were improved.

In 2013, a national committee was created for writing drug therapeutic positioning reports, called IPTs12; autonomic representatives and members of the Drug Spanish Agency and of the Ministry of Health participated. IPTs could be a reference or the main reference for evaluation committees at different levels13. However, up to now, economic evaluation has not been included in these IPTs. Therefore, GENESIS reports continue to be important documents for drug evaluation committees in hospitals, and on health services and autonomic levels.

The main instrument for the diffusion of MADRE and drug evaluation reports is the GENESIS-SEFH webpage (http://gruposdetrabajo.sefh.es/genesis/). This diffusion has contributed to the standardization and quality improvement of hospital drug evaluation reports. However, different analyses showed that there was some need for improvement5,14; their results helped to improve the method in the 2012 version. In one study5 1,805 drug evaluation reports conducted in 175 hospitals were reviewed. The findings showed different concluding decisions despite the fact that all the reports used the similar methodology. It was also observed that different evaluations carried out on the same drug and indication were conducted simultaneously, reflecting inefficient repetition of the same work.

Due to the aforementioned facts, the GENESIS-SEFH group decided to work collaboratively on reference drug evaluation reports which would be useful for the entire country. In 2010, the new collaborative process was initiated; different reviewers from different hospitals now work together on the evaluations. Firstly, an author writes the report which is reviewed by a tutor as well as by the members of the GENESIS coordinating group. This report is made available to the public and emailed to pharmaceutical companies and related scientific societies for allegation and comments. The comments are answered by the tutor. Both the accepted and rejected comments areincluded in the final published version of the report. This final version also includes the positioning of GENESIS-SEFH, the group which represents the scientific society.

As part of this dynamic and continuous improvement process, the GENESIS group has now placed great importance on the economic evaluation aspect. It is a key issue when deciding a drug’s place in therapy. Even though economic evaluations have undergone improvements throughout time, the need to include more explanations as well as tips and tools for improving the economic content of these drug evaluation reports has been recognized. With the objective of focusing in on these necessary improvements, we have decided to review and analyze the economic content of every drug evaluation report already made in a collaborative manner so as to identify the principal difficulties and to make adequate improvement proposals.

Every drug evaluation report carried out with the use of the GENESIS collaborative method and then published on the web page before February 2015 was reviewed.

Reports in which more than one drug was evaluated were excluded in order to avoid the difficulty of evaluating the report with different conclusions or methods for each evaluated drug and to avoid double counting of common parts. In addition, the MADRE method was originally designed to evaluate one drug per report, in spite of the fact that it can be used for evaluating various drugs simultaneously. In any case, this exclusion was not going to significantly alter the results because the multiple drug evaluations were only a small percentage of all the reports.

Data were extracted to an Excel page for analysis. Extracted variables were:

• -

  Descriptive: drug name, evaluated indication and date of publication on the web.

• -

  Variables related with sections included in the report: Whether or not all sections were included and which were missing. The MADRE method includes 9 sections: 1. Drug identification and authors, 2. Applications and evaluation process, 3. Descriptive area: medicine and health problem, 4. Pharmacological area, 5. Efficacy evaluation, 6. Safety assessment, 7. Economic evaluation, 8. Convenience, 9. Conclusions, 10. References. Each section includes different subsections.

• -

  Variables related to the economic evaluation section of the evaluation report:

  • •

    Whether or not all the MADRE economic evaluation sub-sections were included, the ones that were missing, and the ones that were not considered to be applicable to the study. These sub-sections are: 1. Cost and incremental costs, 2. Published economic evaluations, 3. Economic evaluation carried out by the author of the report (cost-effectiveness analysis), 4. Budget impact at a hospital level, 5. Budget impact at a primary care level, 6. Budget impact at a region/country level.

  • •

    Whether or not other non-drug costs were included; if so, identification of considered costs.

  • •

    Whether or not a cost-effectiveness analysis was carried out by the author; which main effectiveness measure was used; from where it had been obtained; and whether or not a subgroup analysis was conducted. The numeric result of the cost-effectiveness analysis, its confidence interval and units. In addition, whether or not a sensitivity analysis was conducted.

  • •

    Whether or not a published cost-effectiveness analysis was reported, who the author was, and what the results were.

  • •

    Whether or not the economic evaluation was considered when positioning the drug, and what the positioning was.

  • •

    Whether or not there was a proposed price for the new drug based on the economic evaluation. When the drug price had not yet been set in Spain, some reports included a cost-effective price proposal.

  • •

    Whether or not public comments regarding the economic evaluation part were received, who made them, what aspects did they make reference to, and whether or not the comments were accepted.

  • •

    Finally, the reviewer collected the main problems found in the economic section of the reports because they were patent in the report or because they were commented on in the report elaboration process.

By February 2015, 36 drug evaluation reports conducted by hospital pharmacists following the collaborative method were available on the GENESIS web page. Four were excluded from the analysis because they evaluated more than one drug: one evaluated oral anticoagulant agents, another bendamustine and rituximab, the third one evaluated bosutinib and ponatinib, and the fourth one evaluated ipilimumab and vemurafenib. Therefore, 32 drug evaluation reports were included in the analysis. Of these, four were in a draft phase; therefore, public comments had not been included yet. In four reports, off-label indications were evaluated.

The drugs and indications evaluated are shown in Table 1. Three reports were published in 2015, eleven in 2014, three in 2013, eight in 2012, four in 2011, and three in 2010. All parts of the MADRE method were included in each report.

With reference to economic evaluation sections:

• -

  Drug costs and incremental costs vs. comparator were included in each report.

• -

  An economic evaluation was conducted by the authors in 87.5% of the reports; theses were cost-effectiveness analyses in 24 cases and cost-minimization analyses in 4 cases. Of the four remaining reports, two failed to include a section dedicated to economic evaluation, while the third case included a comment saying that said type of evaluation was not applicable, and the fourth report included a budget impact analysis.

• -

  In 65.6% of the evaluations, a published economic evaluation was available and commented on in the report. In all but two of the reports, the published economic evaluation section was included (either including information from the publication or indicating that a search had been conducted but no published economic evaluation had been found).

• -

  In 90.6% of the reports, some type of budget impact was included. Hospital impact was included in 43.7% of the cases, primary care impact in 6.2%, and national impact in 71.9% of the cases.

With regard to costs, the drug costs were always included. Non-drug costs were only included on five occasions. In one case, these costs corresponded to surgery, arthroscopy and rehabilitation; other cases involved International Normalized Ratio (INR) test, febrile neutropenia, adverse effects and intravenous administration of drugs, and day care hospitalization. In other reports, some costs were identified but not quantified. In one additional report, a sensitivity analysis was conducted, including failure and hospitalization costs.

In relation to the cost-effectiveness analysis conducted by the author of the drug evaluation report, this was included in 24 reports, accounting for 75% of the evaluated drugs. Three reports did not include this section; one included a budget impact analysis in this section, and in 4 reports (15.6%), since no difference was found between treatments with regard to health effects, a cost minimization analysis was conducted. In Table 2, the effectiveness units used are shown. Health results were obtained from clinical trials except on two occasions. On one occasion, QALYs were obtained from another study, and in the other case, they were obtained from clinical trials with some assumptions.

Subgroup analysis was performed in 9 (28.1%) evaluations.

With regard to the mean cost-effectiveness ratio, in the 14 reports that calculated the cost per life year gained or per quality adjusted life year gained, this ratio was 108,088€. This ratio was less than 30,000 € on just one occasion; in five drugs, the ratio was approximately 50,000-60,000€, in one case between 70,000 and 80,000, and in 7 cases, it was more than 100,000€.

A sensitivity analysis was conducted in every report that included a cost-effectiveness analysis except in two cases. However, most of the time the analysis was not identified as a sensitivity analysis and it was a one-way sensitivity analysis. Variables analyzed in sensitivity analyses are listed in Table 3.

National Budget impact was present in 24 reports. Its methodology and uncertainty estimation differed between reports. Mean national impact was approximately 71 million euros with a wide range of variability; the median value was 24 million euros; minimum being 0.1 and maximum being 1,129 million euros. Thirteen reports also included national health impact, with health variables being very diverse; the life years gained were only estimated on 3 occasions.

Twenty-one evaluations included a published economic evaluation. This was carried out by the National Institute for Health and Care Excellence (NICE) on nine occasions, by the Scottish Medicines Consortium in 6 cases, by the Canadian Agency for Drugs and Technologies in Health (CADTH) in two reports, and one was made by the Australian Medical Services Advisory Committee (MSAC) while one was made by the World Health Organization (WHO); the rest were papers published by other professionals. Seventeen reports included cost-effectiveness analysis, with cost per QALY as the main result.

Twenty-three reports received public comments regarding the economic section, 22 of which were made by pharmaceutical companies. The comments made reference to costs and incremental costs in 17 evaluations, another 17 referred to cost-effectiveness analysis conducted by the author; on four occasions the comments were related to hospital budget impact, one was concerning budget impact in primary care and 10 regarding national budget impact. One drug evaluation report could have several comments regarding different sections. Main comments are included in Table 4. In three reports, no comments were accepted by the authors; in 13 reports, some comments were accepted and in 7 evaluations, every comment was accepted and included in the report.

In every report, except for three evaluations, economic aspects were clearly considered when deciding a drug’s place in therapy. In 8 reports, a drug cost was proposed.

At the end of the evaluation, a proposal was included in 31 of the 32 reports. In 5 cases, the recommendation was to not include the drug in the hospital formulary; in four cases the evaluated drug was considered to be a therapeutically equivalent alternative; in one case it was considered as an alternative treatment without considering it as being therapeutically equivalent, and in 19 reports the recommendation was to include the drug with specific criteria for its use. On two occasions the decision was postponed until more data were available.

Problems in economic sections identified by the reviewer are listed in Table 5.

Drug evaluation reports are now more comprehensive than those published before 2010. Before this date, some sections were not included and the content was quite simple14.

When drugs are evaluated, no published economic evaluation is available in approximately 35% of the cases. It is necessary to conduct an economic evaluation, not only because it is absent, but also because it is frequently difficult to adapt a published economic evaluation due to problems in evaluating its quality and the lack of access to some data. In this study, an economic evaluation was conducted by the authors in 87% of the reports, a higher percentage than in drug evaluations conducted between 2004 and 200714. A similar trend was observed with the budget impact analysis.

The main difficulties faced when conducting economic evaluations in these reports include the lack of: quality data in target population, comparative evidence with alternative drugs being used in clinical practice, long term results, quality of life data, drug price, and clear dosing strategy. Another difficulty concerns the availability of various drug prices

The lack of a drug price was partially managed in the new MADRE version, including recommendations for maximum drug price proposal calculation so as to make a drug cost-effective. This method is based on the estimation of the incremental QALYs obtained with one drug vs. an alternative one, and multiplying it by the threshold willingness to pay per QALY gained most frequently used in Spain, although it is not an official number.

The quality of the economic sections has increased; however, there is still room for more improvement. The following important ideas should be kept in mind:

• 1.

  To include other costs in addition to the drug costs. To make it easier, a table with the most frequently used costs can be included in the MADRE program. For example, in oncology, the cost of one hospitalization day, one hour cost in day care, cost of a neutropenic episode, of a neutropenic infection, and so on. In cardiology, the cost of a myocardial infarction, in hepatology the cost of a liver transplant, etc.

• 2.

  To include helpful tips on calculating survival means from survival curves, to facilitate the use of cost/LYG or cost/QALY gained, which is the only measure that permits comparisons between different pathologies and decreases subjectivity in cost-effectiveness classification, even with its limitations.

• 3.

  To include helpful tips for performing sensitivity analysis and for interpreting it, as well as tips for incorporating this uncertainty in the decision-making process. It could be interesting to give some examples of variables frequently tested in sensitivity analyses.

• 4.

  To include instructions regarding how to transfer QALY published data to our context.

• 5.

  To identify the main analysis from the analyses conducted as part of the sensitivity analysis in every report.

• 6.

  To fix criteria when it is possible to conduct a cost-minimization analysis instead of a cost-effectiveness analysis, considering statistical and clinical differences between treatments.

• 7.

  To establish criteria for selecting prices for calculations (official price for the hospital, price for the patient, reimbursed price, real hospital acquisition price, price as a foreign drug, etc.), if different criteria are necessary for ambulatory care drugs and hospital drugs, consideration of vial left-overs, etc. Uniformity in calculations is necessary for fair budget distribution.

• 8.

  To give recommendations regarding how to handle the lack of comparative studies when indirect comparisons are not possible.

• 9.

  To propose methodology for drug evaluations with poor or nonexistent evidence.

• 10.

  To establish procedures when final variables, such as survival, cannot be estimated.

• 11.

  To set criteria for dosing calculations according to weight, body surface area, etc. as well as the duration of therapy.

• 12.

  To establish recommendations for efficiency criteria in specific situations, such as orphan drugs, end of life, etc.

• 13.

  To give recommendations regarding a gradual introduction of the new drug when calculating budget impact.

• 14.

  To establish criteria for subgroup analysis.

• 15.

  To propose a cost-effectiveness threshold, even if it is not official, in order to base all the recommendations on the same criterion.

• 16.

  To explicitly show criteria for decision making.

In conclusion, reviewing the economic sections of the drug evaluation reports conducted by Spanish hospital pharmacists in a collaborative manner has helped to identify the main difficulties that arise when conducting this part of the reports. This review has also led to the proposal of at least 16 improvements for facilitating and increasing the quality of the economic evaluation section in the drug evaluation reports.

No author has any conflict of interest related with the content of this manuscript.