The TECNO group of the Sociedad Española de Farmacia Hospitalaria (Spanish Society of Hospital Pharmacy) has addressed the definition of a catalogue of indicators for performance, quality and safety in the use of technologies applied to the logistic activity of Hospital Pharmacy Units. The project was developed with a methodology of qualitative techniques by consensus, with the members of the TECNO Group participating as experts. Once indicators had been defined, a validation phase was conducted, and standards were established based on the result of the sampling carried out in the hospitals of the group members.
A total of 28 indicators were obtained, with their corresponding quality standards applied to the use of technologies in the processed for medication storage, dispensing and preparation.
The definition of quality indicators and their standards for measuring technologies in the use of medication represents a step forward in the improvement of their safety.
El grupo TECNO de la Sociedad Española de Farmacia Hospitalaria ha abordado la definición de un catálogo de indicadores de funcionamiento, calidad y seguridad del uso de tecnologías aplicadas a la actividad logística de los Servicios de Farmacia Hospitalaria. El proyecto se desarrolló con una metodología de técnicas cualitativas de consenso participando como expertos los miembros del grupo TECNO. Una vez definidos los indicadores, se realizó una fase de validación y se establecieron estándares en base al resultado del muestreo realizado en los hospitales de los miembros del grupo.
Se han obtenido un total de 28 indicadores con sus correspondientes estándares de calidad aplicados a la utilización de tecnologías en los procesos de almacenamiento, dispensación y elaboración de medicamentos.
La definición de los indicadores de calidad y los estándares de medida de las tecnologías en el uso de los medicamentos es un paso adelante para mejorar su seguridad.
The publication of this panel of indicators is considered relevant due to the lack of validated indicators published for technologies that have been implemented in many Spanish Hospital Pharmacy Units, with the objective of improving patient safety.
There has been a definition of indicators for structure, process-functioning and outcomes, which will contribute to the safety in the use of automated systems for logistical activities such as storage, dispensing, and preparation of medications.
The continuous monitoring of indicators will allow to learn about the efficacy of technologies, and will allow to identify any latent errors or system failures with risk for patient safety, thus avoiding their systematization.
IntroductionThe advances in terms of technologies applied to the healthcare setting have allowed to develop systems that lead to an improvement in the quality, safety and efficiency of processes, including those associated with the use of medications1.
The intense logistical activity by Hospital Pharmacy Units has promoted the updating of technical resources and processes, by incorporating technology in activities which were traditionally manual. Thus, as shown by the results of a survey, the implementation of New Technologies in Spain is essentially targeted to drug management, prescription and dispensing systems2,3.
The Spanish Society of Hospital Pharmacy, as well as other national and international organizations, has created specific work groups for this matter. Throughout its trajectory and true to its mission, the TECNO Group, created on October, 2004, has prepared support documents for the development of effective criteria and practices for the implementation of new technologies regarding the use of medications with efficacy and safety, as part of comprehensive patient care. Line 3 in their Strategy Plan 2013-2017, QUALITY, determines as an operational objective: “To define quality indicators for the use of new technologies”4. In 2010, an editorial published on the role of the Hospital Pharmacist regarding new technologies in the healthcare setting determined a definition of the Pharmacist role in terms of technology selection and evaluation, implementation, assessment of outcomes, teaching, training and research. These activities include determining indicators to ensure the quality and efficiency of processes and their monitoring and follow-up. The editorial also includes methods for quality assessment and orientation to outcomes and quality, among the knowledge and skills of the Pharmacist in charge1.
Patient safety is a critical component of healthcare quality, as the final aim of this technological development that allows to optimize complex processes. The responsibility for adverse events is assigned to deficiencies in the system, its organization and functioning, rather than to the individuals involved. Therefore, it is necessary to be aware of the errors that can be entailed by the implementation of these technologies in Pharmacy Units (PhUs)5.
The variability of activities and persons involved, as well as the conditions in which these activities are conducted, represent an evident risk that must be known and analyzed. The incorporation of technology does not eliminate errors, it will often replace them by others that will become systematic; therefore, quality criteria must be incorporated when planning to implement them and at the subsequent evaluation of their performance. Root Cause Analysis and Failure Mode Effects Analysis (FMEA) are tools that have been used to this aim in the evaluation of processes associated with medications, in order to identify potential errors and their causes. Even though there are a limited number of publications on this matter, with very variable results according to the methodology used, and therefore difficult to compare, the majority of articles published on medication safety are focused on dispensing errors5,6,7. The storage and dispensing settings represent the highest proportion of a PhU activity, measured in Relative Value Units (RVUs), and technology is having the greatest impact precisely upon these activities. Semi-automated systems for storage and dispensing, horizontal and/ or vertical carousels, have represented a major advance in hospital logistics, leading to the maximum optimization of the resources used for medication management, but it is convenient to follow up the quality of processes involved in order to guarantee the minimum number of errors3. Dispensing errors have been quantified in different studies and over the years as the most frequent, with rates from 2% to 31% in the dispensing by stock model. The incorporation of technologies in this activity has already demonstrated a reduction in levels from 1.7% to 8% after the incorporation of semi-automated systems to dispensing by stock. These studies have not only assessed the number of errors, but have also identified the stage of the dispensing process where they occur, and the factors involved. To detect these errors will improve the quality of the service offered by the PhU, and allow to establish preventive measures and work procedures that will lead to safer dispensing5,8,9,10. In the specific case of semi-automated dispensing systems (SDS), the TECNO Group, in collaboration with ISMP-Spain (Institute for Safe Medication Practices) prepared the document “Recommendations for the safe use of automated dispensing systems”. In this document, Essential Procedure 13 includes the assessment of SDS in the hospital programs for quality and risk management. As well as having procedures, it is recommended to evaluate and record any incidents that occur, in order to implement improvements, as well as to define quality indicators with continuous monitoring that will guarantee an adequate performance and use of SDS. Some of them are: contents of the SDS, stock, expiration dates, filling processes, preparation of orders or collection of medications7. Subsequently, TECNO and the ISMP-Spain analyzed the implementation of safe practices in the use of automated dispensing systems, based on this document. The results of this analysis show that the level of implementation of the practices recommended in the Essential Procedure 13 is of 48% for medium-sized hospitals (200 to 499 beds), 54% for hospitals with > 500 beds, and 60% in those with < 299 beds, with a mean 53% in the whole set of hospitals; this was one of the recommendations with lower percentage values of the 14 included in the document11.
The objective of this study is to define a catalogue of quality indicators in order to assess the use of Technologies applied to Hospital Pharmacy.
Methods- –
On September, 2013, in the setting of the Strategy Plan 2013-2017, the TECNO Group defined as an objective: to prepare a set of indicators in order to evaluate the use of technologies implemented in Pharmacy Units.
- –
A study based on qualitative consensus techniques was conducted, where the TECNO Group members participated as experts.
- –
For the definition of indicators, there was an identification of those processes where technologies have been incorporated. All group members were requested to provide the quality indicators used in their centres. Besides, a bibliographic search was conducted in order to identify those indicators already described in literature, as the basis for the definition of the indicators that were the objective of the study3,5,7,12. A fact sheet was completed for each indicator, in order to guarantee homogeneity in data collection and interpretation. This sheet included the name of the indicator, method of calculation, data source, collection frequency, and person responsible, among other data (Table 1).
Table 1.Fact Sheet for the Indicator.
INDICATOR NO. PROCEDURE / PROCESS: RESPONSIBLE: QUALITY CRITERIA: Exceptions to the criteria (if any): Sources of the quality criteria: Standard: INDICATOR (name): Calculation Numerator: Denominator: Data Collection Data sources: Sample size and type of study: Frequency: Responsible: - –
There was a validation stage for the catalogue of indicators defined, in order to evaluate the reliability and feasibility of the calculation of the indicators designed. Data were collected from hospitals with different characteristics, size, work procedures, and commercial solutions implemented.
- –
Finally, the standard value for each indicator was established, based on the results obtained in a sequential sampling over 3 months.
The logistic processes in the Pharmacy Unit that have incorporated technologies are: storage, dispensing, and preparation of medications. The most widely implemented systems are:
- –
Semi-automated systems for horizontal dispensing (SASHD)
- –
Semi-automated systems for vertical dispensing (SASVD)
- –
Automated dispensing systems (ADS)
- –
Automated dispensing systems for outpatients (ADSO)
- –
Medication re-packaging systems
- –
Traceability systems for drug preparation
Process and outcome indicators were defined for each one; the outcome was a list of 28 indicators with their related standards (Table 2). The estimation of quality standards was conducted with data provided by 5 hospitals in the TECNO Group: Universitario Ramón y Cajal from Madrid, Universitario de Getafe from Madrid, Virgen de la Arrixaca from Murcia, General Universitario from Ciudad Real, and Parc Taulí from Sabadell.
PanelofQuality Indicators for technologies in Hospital Pharmacy.
| TECHNOLOGY | QUALITY INDICATOR | CALCULATION OF INDICATOR | STANDARD |
|---|---|---|---|
| SASHD | Reliability of the drug inventory in the SASHD1 | No. of boxes without incidence / No. of boxes reviewed (%) | ≥95% |
| Match between the physical and the clerical stock in the SASHD2 | No. of units without incidence / No. of total units counted (%) | ≥90% | |
| Accuracy in the preparation of orders for clinical units3 | No. of lines without incidence / No. of lines sent to the SASHD (%) | ≥95% | |
| Agreement in empty places | No. of real empty places / No. of empty places identified by the system (%) | ≥95% | |
| Control of expiration date | No. of non-expired units / No. of total units counted (%) | ≥98% | |
| Quality in preparation of orders4 | No. of medication lines without errors in order preparation / No. of medication lines reviewed (%) | ≥98% | |
| No. of units without errors in order preparation / No. of total units reviewed (%) | ≥95% | ||
| SASVD | Reliability of the drug inventory in the SASVD5 | No. of boxes without incidence / No. of boxes reviewed (%) | >80% |
| Match between the physical and the clerical stock in the SASVD | No. of units without incidence / No. of total units counted (%) | ≥75% | |
| Accuracy in the preparation of Unidosis Carts6 | No. of lines without incidences / No. of lines sent to the SASVD (%) | ≥95% | |
| Agreement in empty places | No. of real empty places / No. of empty places identified by the system (%) | ≥98% | |
| Control of expiration date | No. of non-expired units / No. of total units counted (%) | ≥98% | |
| Quality in preparation of carts7 | No. of medication lines without errors in cart preparation / No. of medication lines reviewed (%) | ≥98% | |
| No. of medication units without loading errors / Total number of medication units reviewed (%) | ≥95% | ||
| ADS | Reliability of the ADS inventories in the re-stocking process8 | No. of medication lines without stock incidences in the re-stocking process / No. of medication lines re-stocked (%) | ≥80% |
| Reliability of the ADS inventories9 | No. of medications without stock incidences in the re-stocking process / No. of medications included in the ADS (%) | ≥90% | |
| Stockout for ADS | (Total No. of stockouts / Days analyzed in the period) / No. of stations (%) | ≤2% | |
| Collections without override by ADS | Total No. of collections without override / Total No. of collections* No. of ADS (%) | ≥70% | |
| Medications prescribed not included in the ADS10 | No. of medications prescribed not included in the ADS / Total No. of medications in the ADS (%) | ≤10% | |
| Collection / Re-stocking Ratio11 | No. of medication units collected / No. of medication units re-stocked in each ADS | Historic control | |
| Control of expiration date | No. of non-expired units / No. of total units reviewed (%) | ≥98% | |
| Errors in ADS re-stocking12 | No. of lines adequately re-stocked / Total No. of lines re-stocked (%) | Historic control | |
| ISMP Index of Safety in ADS | Results of the self-assessment questionnaire for ADS safety | Historic control | |
| ADSO | Accuracy in dispensing to outpatients13 | No. of lines without incidence in dispensing / No. of medication lines dispensed (%) | ≥99% |
| Accuracy in the automatic load of medications | No. of non-rejected packages / Total No. of packages to be loaded (%) | ≥95% | |
| Agreement between the ADSO stock and the management system | No. of existing units in ADSO / No. of units in the ADSO Management System (%) | ≥95% | |
| System stops in dispensing | No. of incidences blocking ADSO in dispensing / Total No. of dispensing lines (%) | ≤5% | |
| Quality in dispensing of medications to outpatients14 | No. of dispensing actions without errors / No. of dispensing actions conducted with the technology (%) | ≥99.5% | |
| RE-PACKAGING SYSTEMS | Losses in re-packaging | No. of units lost / Total No. of units to be re-packaged (%) | ≤1% |
| Quality in re-packaging | No. of units adequately re-packaged / No. of re-packaged units (%) | ≥99% | |
| TRACEABILITY SYSTEMS FOR DRUG PREPARATION | Discrepancies of medications in laminar flow cabinet | No. of preparations intercepted by the system / Total No. of preparations (%) | Historic control |
| Deviations in gravimetric control | No. of preparations detected with incorrect weight by the gravimetric control / Total No. of preparations (%) | Historic control | |
| Accuracy in administration15 | No. of errors in administration to patients prevented by the traceability system / Total No. of administrations (%) | Historic control |
SASHD: Semi-automated systems for horizontal dispensing ADS: Automated dispensing systems
SASVD: Semi-automated systems for vertical dispensing ADSO: Automated dispensing systems for outpatients
Notes on the Panel of Indicators.
| Semi-automated system for horizontal dispensing (SASHD): | |
| 1Reliability of the inventory | Any discrepancy found, qualitative as well as quantitatively, in the SASHD boxes. A systematic review of the SASHD must be established for its calculation. |
| 2Match between the physical and the clerical stock in the SASHD | Any quantitative discrepancy in the units counted in a systematic review of the SASHD. The standard value has been defined taking into account that +/-5% discrepancies in the units counted in relation to the clerical stock of the SASHD are not considered an error. |
| 3Accuracy in the preparation of orders for clinical units | “Incidences” are defined as lines automatically generated by the system in the “Report of Incidences”, when completing the preparation of orders (any type of orders: deals, urgent orders, re-stocking for SASVD and ASD, etc.). E.g. order “10”, dispensed “8”. |
| 4Errors in preparation of orders | Any discrepancy found, qualitative as well as quantitatively, between what has been requested and what has been dispensed in each order prepared. Stockout of a medication will be considered an error.One or the other indicator will be calculated based on the needs and the potential for calculation in each hospital. |
| Semi-automated system for vertical dispensing (SASVD) | |
| Reliability of the inventory | Any discrepancy found, qualitative as well as quantitatively, in the SASVD boxes. A systematic review of SAS-VD must be established for its calculation. |
| 6Accuracy in the preparation of Unidosis Carts | “Incidences” are defined as lines automatically generated by the system in the “Report of Incidences”, when preparing for loading each Unidosis Cart. E.g. order “3”, dispensed “1”. Stockout of a medication will be considered an error. |
| 7Errors in load preparation for the Unidosis Carts | Any discrepancies found, qualitative as well as quantitatively, between what has been ordered and what has been dispensed in each box of the Unidosis Cart. Stockout of a medication will be considered an error.One or the other indicator will be calculated based on the needs and the potential for calculation in each hospital. |
| Automated Dispensing System (ADS): | |
| Reliability of the ADS inventories in the re-stocking process | Any discrepancies found, qualitative as well as quantitatively, in ADS locations during the re-stocking process. This indicator does not include the expiration date control. |
| 9Reliability of the ADS inventories | Any discrepancies found, qualitative as well as quantitatively, in ADS locations during the re-stocking process. A systematic review of the ADS must be established for its calculation. |
| 10Override | Exceptional withdrawal of medications from the ADS previous to pharmaceutical validation |
| 11Collection/Re-stocking Ratio | The calculation must exclude those medications which the Hospital has defined as external to the ADS (e.g. saline therapy). It will include those medications prescribed but not included in each ADS, and which are dispensed by an alternative circuit to ADS re-stocking |
| 12Collection/Re-stocking Ratio | Each centre will define the optimal efficiency in re-stocking according to the maximum/minimum stocks defined for the medications included in the ADS and the re-stocking frequency determined for them. E.g.: For a 4:1 ratio (max:min) and minimum stock determined to cover the needs during 2-3 days, the optimal re-stocking frequency would be approximately every 6 days. |
| 13Errors in ADS re-stocking | Each centre will define the types of error they are interested to analyze and the methodology to be followed. It is considered essential to measure at least qualitative errors (wrong medication, medications mixed in the same location), which must be measured in the re-stocking process in real time. |
| Automated dispensing system for outpatients (ADSO) | |
| 14Accuracy in dispensing to outpatients | “Incidence” is defined as any discrepancy found, qualitative as well as quantitatively, in the dispensing lines. Stockout of a medication will be considered an error. |
| 15Quality in dispensing medications to outpatients | The type of error to be considered is not specified in the indicator (wrong medication, dose, quantity…) Each centre will define the types of error they are interested to analyze and the methodology to be followed. |
| Re-packaging System | |
| 16Losses in re-packaging | Medications that, after the re-packaging process, have lost their integrity or organoleptic characteristics (breakage, change in colour, pulverization…) |
| Traceability in preparation: | |
| 17Medication discrepancies in the laminar flow cabinet | Those intercepted by error in medication, dose, concentration, etc., will be considered discrepancies. |
| 18Accuracy in administration | At the time of administration to the patient, the system detects an error that can be “wrong patient” or “wrong order of administration”. |
According to the WHO, the best way to adopt solutions in order to reduce risks is to think in terms of system; therefore, it is essential for organizations to get involved in the implementation of quality guarantee systems, and to define criteria, objectives and standards.
An indicator is not a direct measure of quality, but a tool that allows us to assess actions, and indicates which aspects require a deeper analysis. There are different definitions of indicator. According to the JCAHO (Joint Commission on Accreditation of Healthcare Organizations), it is: “A quantitative measure useful for monitoring and assessing the quality of important aspects in patient care, organization and management. It points at the aspects where there might be an opportunity for quality improvement”13. Rule UNE 66175, Quality Management Systems. Guidelines for the implementation of indicators systems, defines it as: “Data or set of data which will help to measure objectively the evolution of a process or an activity”14.
Indicators are tools determined over time, which allow an improvement in the quality of processes. Having a catalogue of indicators will facilitate management and benchmarking, and ensure a homogeneous quality15.
The methodology for developing the catalogue was based on qualitative techniques by consensus, because there are no publications on validated indicators for the use of these technologies. This participative method is widely used in the setting of public healthcare, given the need to unify criteria in areas where it is not possible to generate scientific evidence15-16.
The following are considered qualitative techniques: open interviews, discussion groups, observation, and participative observation. Qualitative research collects the words by the subjects for their subsequent interpretation, without insisting on the statistical representation of quantitative techniques. The members of the group are required to make collective decisions, based on common agreements. In order to reach this type of agreements and decisions, there are different techniques for consensus that can help to a structured and systematic process.
In order to guarantee the validity of a consensus, the following will be essential:
- •
To determine the questions to be answered, and set up clear and specific objectives.
- •
To select the group of experts, in order to guarantee aspects such as a sufficient number of members, experience, prestige, interest for the subject, time availability, and lack of conflicts of interest.
- •
Scenarios must be methodically prepared, following a formal structured process.
These indicators have been confirmed in daily practice, verifying that Pharmacy Units have the information systems required to allow their monitoring. In the items “SASH: errors in the preparation of orders” and “SASVD: errors in the preparation of carts”, 2 indicators with similar characteristics have been defined, to allow each hospital to use the most feasible based on the technology available and their information system.
The objective of the TECNO Group has been for maximum values, trying to define a high number of indicators in order to include the highest number of activities and characteristics of the automated logistic processes, though avoiding to make one single definition, or even leaving some matters for each centre to decide, aware that the structure and work procedures are different in each Pharmacy Unit, and it is not always possible to apply common criteria.
In the case of automated systems for dispensing to outpatients, or traceability in preparation, their limited current implementation in Pharmacy Units makes it difficult to obtain indicators, and even more to define a quality standard; therefore, the catalogue leaves up to each Pharmacy Unit the definition of the types of error to be monitored, based on their own interests. It is considered necessary to continue along this line of work, as these technologies are implemented and others are incorporated, such as the use of robotic dispensing systems.
We must highlight a limitation: the standards were calculated with the data collected in 5 hospitals from the Group. A larger study would be required in order to verify them; and for this reason, the TECNO Group considers it will be necessary to develop a multicentre project with the Pharmacy Units that use technologies to calculate systematically and periodically these indicators, and share these results. An increase in sample size would allow to validate or re-calculate standards based on results, and thus continue moving forward in quality improvement.
With this definition of indicators, the TECNO Group takes one more step to ensure the best use of the technologies available. If so far it had defined the technologies and requirements that should be met in terms of structure, software, interfaces and services4, with the definition of this panel of indicators it meets the objective of establishing a continuous evaluation system which will allow to identify latent errors or system failures with risk for patient safety, avoiding their systematization. The definition of quality indicators for technologies applied to Hospital Pharmacy and their standards is a process for continuous improvement that contributes to a safe use of medications.
