The impact of adding usability testing to your eCOAs
7 mins
The digital revolution leaves no stone unturned, with the complex world of clinical trials being no exception. The introduction of eCOAs has transformed how clinical trials collect their data - replacing traditional paper-based methods with digital tools that improve patient protocol compliance and retention, increase data accuracy, facilitate regulatory compliance, and align with patient preferences for mode of responding to COAs.
The success (and challenge) of eCOA systems lies in how easy they are for participants to use and navigate. Often, the very aim of a trial is to include patients that come from diverse backgrounds - so how can we ensure the collection methods reflect this? Enter usability testing.
Conducting usability evaluations that prioritize the patient-perspective and experience is essential for ensuring user-friendly, intuitive, and effective eCOA systems that collect accurate data.
The importance of usability evaluations
1. An improved user experience
Clinical trials often by their nature include patients from a variety of backgrounds. An eCOA design that doesn't consider their perspective and experience can make it harder to use, and inhibit engagement.
eCOA platforms can be improved by testing a wide range of participants in usability testing. This includes both men and women, different age groups, and participants with different levels of education and impairments (e.g. motor, visual, cognitive). By doing so, it can be confirmed that these platforms work well for most participants in global clinical trials.
This enables users to navigate the system comfortably and provide accurate responses, despite the significant demographic variability that can exist even within a well-defined target patient population.
2. Ensuring data consistency and accuracy
The aim and responsibility of any clinical trial is of course to ensure the data it receives is accurate, and as we know, in the pharma and healthcare industry, the stakes could not be higher. Ultimately, usability evaluations that prioritize the patient perspective in these ways contribute significantly to the consistency and accuracy of the data collected through eCOA systems.
Increasing the usability of eCOA systems for all potential users by removing common but often unaddressed barriers enables participants to better comprehend and accurately respond to assessment questions, minimizing errors and inconsistencies in data entry.
3. Staying Linguistically and Culturally sensitive
Clinical trials often involve participants from diverse cultural and linguistic backgrounds. Usability evaluations ensure that the eCOA system is culturally sensitive, and aligns with the linguistic, cultural, socioeconomic, and educational nuances of different target populations. For example, we may think about how Arabic and Urdu, which have left-to-right writing, show things differently on screens of varying sizes. Scripts with complex characters such as Japanese and Chinese may also require a very specific approach for maximizing the usability of eCOA systems displaying them. It is also important to explore how access to and familiarity with technology impacts a user’s ability to successfully interact with an eCOA system, and make adjustments to solutions that improve their universality and ease-of-use for all populations.
This promotes inclusivity, mitigates misunderstandings, and enhances patient engagement.
A framework for usability evaluations
When considering usability evaluations for eCOA systems, several methods can be employed. Here are a few commonly used approaches, which can furthermore be deployed sequentially within a single UT methodology designed to 1) comprehensively mitigate common pitfalls of eCOA design and 2) improve the user’s experience by prioritizing the patient perspective:
- Face validation: COAs are increasingly developed solely for electronic administra-tion, and may not, in all cases, undergo rigorous validation prior to testing or use with participants. Face validation assesses the measurement properties of a COA and provides an opportunity to address any potential issues related to construct va-lidity, reliability, and general questionnaire design ahead of more formal types of validation (i.e. psychometric, linguistic).
- Linguistic validation: LV is a process that involves translating and validating assess-ments to ensure linguistic and conceptual equivalence across languages. This is par-ticularly important when the eCOA system includes validated questionnaires or scales that need to maintain their psychometric properties across translations.
- Heuristic evaluation: Heuristic evaluation involves experts examining the eCOA sys-tem against a set of predefined usability principles or heuristics. These experts eval-uate the system independently, identifying potential usability problems and provid-ing recommendations for improvement. Heuristic evaluations should also consider the potential need for cultural adaptation of the eCOA system, ensuring that the us-ability of a platform will extend into a global clinical trial context. Cultural adapta-tion involves aligning the content, visuals, and user interface with the cultural norms and expectations of each possible target population.
- Cognitive walkthrough: With cognitive debriefing, evaluators simulate users' thought processes and step through the system to assess its ease of use with partici-pants. This method helps identify potential usability issues, such as unclear instruc-tions, confusing navigation, or error-prone data entry points.
- User testing: User testing involves observing participants as they interact with the eCOA system, performing tasks representative of real-world usage scenarios and an-swering a set of usability questions customized to the platform and target popula-tion. By capturing user feedback, observing their actions, and noting any difficulties encountered, designers can identify areas for improvement and make necessary ad-justments to enhance usability.
- Multilingual user testing: User testing should include participants representing the target populations for which translations are needed. This allows designers to ob-serve how participants interact with the translated eCOA system, gather feedback specific to each language, and identify language-related usability issues. The increas-ingly global context for clinical trials necessitates an expanded UT approach in order to better accommodate the many linguistic, cultural, socioeconomic, or educational differences that can emerge.
Usability evaluations play a vital role in ensuring the success and effectiveness of eCOA systems in clinical trials. By considering the large array of factors that contribute to differences in the patient’s experience, usability evaluations can address the needs of diverse populations, enhance the user experience, and improve the quality of data collected. Incorporating a global clinical trial perspective into usability evaluations promotes inclusivity, cultural adaptation, and accurate understanding of assessment content.
By investing in usability evaluations, researchers can create user-centric eCOA systems that facilitate efficient and impactful clinical trials across patient groups, languages and cultures, benefiting both researchers and patients alike.
At RWS, we understand the importance of a well-executed eCOA process and have the expertise and experience necessary to guide you through it. Our end-to-end process management, eCOA solutions team, and linguistic validation experience ensure that your translations are done right. Contact us to learn more about how our team can help you navigate the complexities of linguistic validation and achieve success in your clinical trials. We are always ready to work with you to ensure that your clinical trial runs smoothly and successfully.
This blog was co-written by Dan Herron and Tim Poepsel, Survey Research Analyst Team Lead at RWS. Tim received a BA in Linguistics from Northwestern University and a PhD in Psychology and Language Science from Penn State University, where his research focused on 2nd language learning in adults. He currently works in the area of linguistic validation, and his research focuses on methods for incorporating the patient’s perspective into developing, improving, and administering clinical outcome assessments in a global context.
Dan Herron
All from Dan HerronRelated Articles
