The automation of hazardous drug preparation in hospitals using robotic systems is an effective strategy to enhance safety, quality, and process efficiency. However, its implementation introduces a novel user–robot interaction that transforms professional roles, tasks, and training needs. The acceptance of this technology depends on human, technical, and organizational factors.
At present, no validated tool exists to specifically assess user–robot interaction in the preparation of hazardous drugs, whether in industrial or healthcare settings. Therefore, this study aims to develop and validate a questionnaire to evaluate such interaction in hospital settings.
MethodsThis multicenter study will be conducted across five Spanish hospitals equipped with robotic systems for hazardous drugs compounding. The study population will include pharmacy technicians, nurses, and other healthcare professionals who operate these systems. The questionnaire will be developed using the Delphi method, a rigorous consensus process involving a panel of experts. Data collection will be conducted using the REDCap (Research Electronic Data Capture) platform. The validation process will assess: face validity (clarity and coherence of the questionnaire), content validity (relevance of the included items), criterion validity (correlation with an established external criterion), construct validity (consistency between the items and the construct under investigation), and internal consistency (measured using Cronbach's alpha).
ConclusionsThis study is the first initiative to develop and validate an instrument specifically designed to evaluate human–robot interaction in the preparation of hazardous drugs. Its implementation will support the identification of barriers and facilitators to the adoption of robotic systems in clinical practice, guide the design of specific training programs, and contribute to the optimization of medication preparation workflows in hospital settings.
la automatización de la preparación de medicamentos peligrosos mediante robots en los hospitales es una estrategia eficaz para mejorar la seguridad, la calidad y la eficiencia del proceso. Su implementación introduce una nueva interacción usuario–robot que transforma roles, tareas y necesidades formativas. La aceptación de esta tecnología depende de factores humanos, técnicos y organizativos, lo que hace necesario disponer de herramientas que evalúen dicha interacción en el contexto hospitalario.
Actualmente, no existe ninguna herramienta específicamente diseñada para analizar la interacción usuario–robot en la elaboración de medicamentos peligrosos, ni en el entorno industrial ni en el sanitario. En este contexto, el objetivo del estudio es desarrollar y validar un cuestionario para evaluar esta interacción en el entorno hospitalario.
Métodosse trata de un estudio multicéntrico que se realizará en 5 hospitales españoles que dispongan de robots para la preparación de medicamentos peligrosos. La población de estudio incluirá técnicos de farmacia, personal de enfermería y otros profesionales usuarios de estos sistemas. El desarrollo del cuestionario se realizará mediante la técnica Delphi con la colaboración de un comité de expertos. El formato de recogida de datos será en REDCap (Research Electronic Data Capture). La validación evaluará: validez de apariencia (claridad y coherencia del cuestionario), validez de contenido (relevancia de los ítems incluidos), validez de criterio (correlación con un criterio externo establecido), validez de constructo (coherencia entre los ítems y el constructo que se está investigando) y consistencia interna (grado de confiabilidad medido mediante el alfa de Cronbach).
Discusióneste proyecto constituye la primera iniciativa para desarrollar un instrumento validado que analice la interacción usuario–robot en la elaboración de medicamentos peligrosos en los hospitales. Su aplicación permitirá identificar barreras y facilitadores en la aceptación de estas tecnologías, orientar el diseño de programas de capacitación específicos y contribuir a la optimización de los flujos de trabajo en la preparación de medicamentos en entornos hospitalarios.
The Institute for Safe Medication Practices (ISMP) supports automated pharmacy compounding as one of the most effective methods in terms of quality, safety and efficiency.1,2 The use of advanced technologies, including robotic compounding systems (RCS), improves the monitoring and traceability of compounding processes, increases compound precision and accuracy, and enhances patient safety while minimizing risks to laboratory personnel.2
The adoption of robotic compounding systems in hospitals poses a new human-robot interaction scenario that requires a thorough understanding of human, technical and organizational factors. This transition not only transforms existing processes but also entails the emergence of new tasks and professional roles that require appropriate management (support and maintenance tasks, as well as the development of specific personnel training programs).
The successful integration of robotic systems within the clinical laboratory requires careful evaluation of human-robot collaboration, especially regarding user acceptance of these systems.
Among available models for evaluating human-robot interaction, the most widely accepted is the Extended Technology Acceptance Model (TAM23 or TAM34), which is an extended version of the original TAM model (Technology Acceptance Model5) developed by Davis et al. in 1989. This model is not intended to be specific to a single technology. Instead, it provides a general theoretical framework for assessing user acceptance across diverse technologies and scenarios using questionnaires as a measurement tool. The ETAM is widely used to assess variables influencing user acceptance of new technologies (information systems and automated dispensing systems, among others).4,6
More specifically, Brohl et al.7 developed the Human-Robot Collaboration Acceptance Model (HRCAM), an ETAM-based model adapted to human-robot interaction in the production industry.
After a literature review, we found no previous questionnaires designed to assess human-robot interaction for hazardous drug compounding in the hospital setting (with all its inherent organizational, operational and safety particularities).
The primary objective of this study was to develop and validate a questionnaire for assessing human-robot interaction in the compounding of hazardous drugs in the hospital setting.
MethodsStudy designA multicentre study will be conducted to develop and validate a questionnaire for assessing human-robot collaboration for hazardous drug preparation in hospital pharmacies. The participating sites will involve five Spanish hospitals, all of which have automated compounding systems installed in their Hospital Pharmacy Departments.
This project is planned to develop across four non-clinical phases, as described in Fig. 1.
Study population
The study population will comprise users of robotic compounding systems (RCSs) in the preparation of hazardous drugs in the hospital setting (pharmacy technicians, nursing staff, pharmacists responsible for RCS maintenance and other healthcare providers). All participants will be required to provide informed consent to take part in the study.
In line with the scientific literature reviewed, a different number of participants will be included across the different questionnaire development phases.8–12 More specifically, 10–15 participants will be selected for initial validation, whereas 30–50 subjects will be involved in the pilot study.10,12 For the final validation phase, a minimum of 5 subjects will be required per item.13,14 Assuming that the questionnaire will contain a comparable number of items to that of ETAM or HRCAM (i.e 20–25 items), the estimated sample size is espectet to range from 100 to 150 participants.
Initial validation: 2–3 volunteers will be selected per participating site (total n = 10–15).
Pilot study: volunteers will be recruited from all sites involved in the questionnaire development process (total n = 50–60).
Final validation: all users of RCS for hazardous drugs in Spain will be invited to take part in the study through the Spanish Society of Hospital Pharmacy's SEFH mailing list (total n = 100–150).
Across all phases of the study, participants will receive an email containing an invitation to take part in the study, explaining the study's purpose, the steps necessary to complete the questionnaire, and the estimated time needed for completion.
Questionnaire designThe questionnaire will be developed following the Delphi method, an information collection technique by which iterative feedback is obtained from a panel of experts. This method is recommended for designing questionnaires in innovative scenarios where limited evidence is available.15
According to the Delphi methodology, the panel is most frequently composed of 6–30 experts, with priority given to quality over quantity.15
In our study, the panel will include eight pharmacists with over one-year experience of the use of RCS for hazardous drugs at their sites.
An initial meeting will be held to define the questionnaire purpose and the scientific literature to be reviewed (ETAM, HRCAM).4,7 In this meeting, construct dimensions, initial questionnaire items, and questionnaire format (data collection platform, response type, Likert scale type) will be additionally established.
Two Delphi rounds will be performed electronically. In this round, each expert will express their degree of agreement with each of the items previously defined in the start-up meeting by rating them on a 5-point Likert scale. Qualitative comments on each item will be allowed.
Following each round, an analysis of the level of consensus on each item will be performed. A high level of agreement is established when ≥80% of experts “agree” or “strongly agree” with an item.15–17 When a low degree of consensus is observed, items will be removed from the questionnaire. Allcomments and decisions will be documented in interim reports. In the last round, the items excluded from the final version, along with changes to the first draft, will be identified and summarized in a final report. The final consensus version will be submitted for further validation.
This study will be conducted in accordance with the ACCORD guidelines for the consensus process phase.18
Questionnaire validationQuestionnaire validation is the process of ensuring the validity (extent to which the tool measures the intended variable), reliability or reproducibility (consistency and accuracy of results) of an instrument.12 A variety of validation methods are available, including statistical analyses, pilot tests and other strategies described below.9
The initial validation of our questionnaire will address two components:
Face validity: The questionnaire is evaluated for clarity, consistency of style, ease of understanding and layout10. Participants will be asked to evaluate each item according to the following questions: Is the item easy to understand? (Yes/No, free text if the answer is ‘No’) Would you change anything in the item? (Yes/No, free text if the answer is ‘Yes’) Do you find it easy to follow the order of the items? (Yes/No, free text if the answer is ‘No’).
Content validity, which refers to the extent to which the items in a questionnaire are representative of the entire theoretical construct or domain the questionnaire, is designed to assess12. For this purpose, the relevance of each item is assessed on a 4-point Likert scale (1 = not relevant, 2 = slightly relevant, 3 = relevant and 4 = very relevant). Based on these scores, the content validity index is calculated at item (I-CVI) and scale (S-CVI) level. According to the literature, an I-CVI ≥ 0.78 will be considered acceptable for an item to be maintained in the final version of the questionnaire10.
To evaluate the two components, a data collection form will be generated on REDCAp and sent out to a representative sample of RCS users at the participating sites.
Preliminary pilot studyTo minimise errors in questionnaire administration, the performance of the design resulting from initial validation will be tested in a small sample of participants. A pilot study will be conducted involving a larger sample of users from participating sites.
Participants will complete the latest version of the questionnaire electronically (REDCap) by using a 6-point Likert scale (1 = strongly disagree, 2 = largely disagree, 3 = slightly disagree, 4 = slightly agree, 5 = largely agree, 6 = strongly agree).
An analysis will be performed of the results obtained to identify potential areas for improvement and apply the necessary modifications.
Final validation of the questionnaireFinal validation will involve evaluating the following variables:
Criterion validity evaluates the extent to which a questionnaire evaluates how accurately a variable predicts the outcome of a related variable. Criterion validity is assessed by comparing the instrument against a related, previously validated indicator. The correlation between the scores obtained with our questionnaire and those obtained with the Spanish-validated version of the Affinity for Technology Interaction (ATI) scale will be assessed.12,19
Construct validity, defined as the extent to which questionnaire scores are consistent with other measures of the same construct,12 will be assessed by examining correlations between the scores obtained with our instrument and related variables, such as professional experience of RCS use and frequency of use.
Internal consistency, as an indicator of questionnaire reliability, evaluates the degree of correlation among items measuring the same concept/construct. This will be assessed using Cronbach's alpha.9
Using the SEFH mailing list, the Spanish Society of Hospital Pharmacy will send the final version of the questionnaire via e-mail, accompanied by an invitation to participate in the survey. Results will be assessed based on previously defined criteria and presented on a final report.
DiscussionDespite the growing interest in the automated preparation of hazardous drugs within hospital pharmacies, no questionnaires have been validated for evaluating human-robot interaction in this setting.
The questionnaire proposed in this study provides an innovative approach, since it integrates hospital-specific dimensions such as safety when handling hazardous drugs, process traceability, the need for specialized training, and perceived risks.
Moreover, the implementation of a rigorous development and validation process based on the Delphi methodology and other statistical methods,8,9,16 added to the participation of experts with direct experience of robot use in the hospital setting, will strengthen the methodological robustnes of the questionnaire. This approach ensures that the questionnaire only includes consensus items aligned with clinical practice, thereby reinforcing content validity and future applicability.
The ETAM model was previously implemented in a study aimed at assessing user-robot interaction during the implementation of a pharmacy dispensing robot.6 Another study used the ETAM model to examine user acceptance of bar-coded medication technology in a pediatric hospital.20 A limitation of that study was that the ETAM model was modified to adjust it to the technology evaluated without going through a methodologically consistent validation process; therefore, the reliability and validity of the model were not guaranteed.
The Delphi methodology favors professional consensus and ensures transparency in the decision-making process. However, our study may have some limitations related to the consensus process, including a potential expert selection bias. To overcome this limitation, we designed a meticulous validation process to ensure the validity and applicability of the instrument.
In conclusion, this is the first study aimed at developing a validated instrument to analyze human-robot interaction in the hospital setting. Hospital leaders responsible for implementing robotic hazardous drug compounding systems may find this questionnaire valuable for identifying barriers and facilitators to acceptance, designing targeted training programs, and optimising workflows. The ultimate purpose of this instrument is to maximize the benefits of automation in terms of healthcare safety, quality, and efficiency.
Contribution to the scientific literatureThis study develops a methodology for designing a questionnaire to assess human-robot interaction in the compounding of hazardous drugs in the hospital setting. Likewise, this is the first instrument designed to assess human-robot collaboration in the hospital setting.
FundingThe authors did not receive any funding for the preparation of this manuscript.
Use of generative artificial intelligenceThe authors declare that ChatGPT (OpenAI) was used during the preparation of this paper to translate and improve the writing style of some sections.
The authors thoroughly revised and validated the AI-generated contents to ensure their accuracy, coherence and originality and assume full responsibility for the contents of this manuscript.
CRediT authorship contribution statementCristina Fernández-López: Writing – review & editing, Writing – original draft, Visualization, Validation, Project administration, Methodology, Investigation, Formal analysis, Data curation, Conceptualization. María Calvo-Arbeloa: Writing – review & editing, Writing – original draft, Validation, Investigation. Paz Pacheco-Ramos: Writing – review & editing, Writing – original draft, Validation, Investigation. Carmen María Valencia-Soto: Writing – review & editing, Writing – original draft, Validation, Investigation, Formal analysis. Leticia Garrido-Sánchez: Writing – review & editing, Writing – original draft, Investigation. Sandra Fontanals-Martínez: Writing – review & editing, Writing – original draft, Validation, Resources, Investigation, Conceptualization.
The authors declare no conflicts of interest associated with this publication.
