When you think of a medical device, user experience (UX) might not come to mind as one of its key features. But UX is quickly rising in importance for medical device manufacturers, and global UX considerations are becoming an indispensable part of the typical medical device design process.
Why is that? For one, medical devices are changing. They’re going digital and becoming more “user-friendly,” more akin to what consumer devices look like. On the other hand, they feature more and more complex user interfaces (UIs). The scope of what constitutes a medical device has expanded over the years, and now includes a wide range of software and apps. Many of these are used by patients or caregivers such as family members, meaning they’re nonprofessionals rather than trained medical staff.
This means that well-tested UIs and the overall usability of devices are important risk-mitigation measures. Safety of medical devices is the industry’s number one concern. It’s critical to mitigate the risk of patient harm due to improper use caused by faulty device design. This is even more important because medical devices get used in sometimes high-risk, critical life-or-death situations, where simplicity and ease of use are paramount. According to the World Health Organization, medical error rates have been quoted to be in the region of 5-15% per hospital admission in the developed world. And an analysis of adverse events reported for medical devices post-release shows many of them are attributable to issues regarding the design of the UI.
While the medical device sector is playing catch-up to the technology sector in many ways, it is fairly ahead when it comes to usability, the key part of the overall user experience. Not surprisingly, usability considerations form part of the regulatory requirements in many countries. Centered on the concept of human factors and usability engineering (HF/UE), the focus is on the synergy of users with systems and their IUs, with the goal to optimize the design of displays controls, and other UI aspects and limit use-related risks.
In the US, the Food and Drug Administration (FDA) issued an updated guidance document in 2016 that requires manufacturers of higher-risk types of devices to provide a report in their premarket submission that contains human factors/usability information. This should include, for instance, a description of the device UI, potential use errors,and details of the human factors validation testing performed.
The FDA officially recognizes a number of standards published by national and international standards bodies, including the most recent one: ISO/EN 62366-1:2015 Medical devices, Part 1: Application of Usability Engineering to Medical Devices. This outlines activities throughout medical device development that culminate in validation testing, with the final UI design tested in simulated use environments.
The standard is also applicable in the European Union (EU), putting both markets in this respect on a similar regulatory footing. In the EU, manufacturers need to comply with this standard to obtain the Conformité Européene (CE) Marking required to release their devices in this market. The new Medical Device Regulation (MDR) 2017/745, which will come into force in mid-2020, addresses risk management and requires providing evidence of validation of reduced risk, which is the goal of usability testing.
How do medical devices get tested for usability?
Usability testing of medical devices differs slightly from the established practice in the IT industry or the consumer space. But the general principles are universal. In practice, manufacturers are required to address UI considerations early on in their device development process. In this context, user interface has a wider definition and includes all the points of interaction between the product and the user. For example, not just displays but also controls, packaging, labeling, instructions for use (IFUs), training, alarms and so on.
Formative testing comes first
In the design development phase, formative testing serves to confirm designs and to find potential use errors which may lead to unacceptable risk for the patient or user. It should be carried out on all aspects of the design, including IFUs and training documents. This testing addresses the twin goals of making the UI intuitive and easy to use while mitigating potential use-related risks. These goals can sometimes be in conflict because a design that is convenient for users may not always be the most secure one.
Formative testing is designed to reveal use issues that can only be found through observing user interaction with the device. It’s performed using simple mock-up devices, preliminary prototypes or more advanced prototypes as the design evolves. This type of testing may also help determine training requirements and inform the design of the labeling and training materials.
One popular concept that the medical device community borrowed from other industries is cognitive walk-through. In this process, test participants are guided through the process of using a device while being questioned and encouraged to discuss their thought processes (“think aloud”). This serves to uncover any difficulties or concerns as they occur. As such, this is often complemented by a heuristic evaluation that uses checklists to validate device interfaces for usability and safety.
In practice, formative testing is an iterative process during medical device development, potentially resulting in design changes. Once the design is fixed, devices proceed to the next stage, where they are tested in more real-life scenarios. Formative testing can be normally done with five to seven people from each identified group of users.
Summative testing
comes next
Summative testing, also called simulated use testing or human factors validation testing, is the second key part of assessing the usability of medical devices prior to market release. It’s a final demonstration of their use safety for both IEC 62366 and the FDA human factors pre-market evaluations. The idea is to engage representative users to perform a set of scenarios under simulated use conditions that are representative of real-life conditions.
Participants follow use scenarios that represent typical interactions with the device, and should include all the tasks that have been categorized as high risk. The tests should include a post-test dialogue with the study team to establish the cause of any failures and difficulties.
Importantly, this testing should also include validation of the IFU and accompanying documentation with the intended users, in their language, using materials that are representative of the final text and layout. Such testing will observe if and how users can follow relevant instructions in order to correctly use the product, and how they understand the information provided for their safety. IFUs often get tested separately, because it’s not realistic to force test participants to use them during the simulated use testing. Summative testing normally requires at least 15 people in each identified category of users.
The FDA requires that testing participants reside in the US. This is because, according to the FDA, studies performed in other countries or with non-US residents may be affected by different clinical practices that exist in those areas, different units of measure used, language differences that change the way labeling and training are understood, etc. The labeling and training should correspond exactly to what would be used for the device if marketed in the US — and thus, testing needs to be conducted in the US. Conversely, for CE Marking in the EU, usability testing should be done in-market, though not necessarily with participants from all EU countries or covering all EU languages.
General recommendations
User experience or usability of medical devices is a concept that is developing rapidly, and has been codified in relevant guidelines or standards only recently. But its importance is rising with most device manufacturers. When considering the global aspects of medical devices, these are some of the general recommendations:
•Guidelines encourage usability evaluation and testing to occur early on in a typical device development process. This is a chance to include users and testers from respective target markets to validate in-market device usability early on. They should be included ideally in the formative testing phase, but at a minimum in the summative.
•Usability testing works with a set of specific personas; descriptions of a few typical intended users of the device based on empirical evidence. Some aspects of these personas are country-specific, reflecting the roles and responsibilities of medical staff in individual healthcare systems, as well as specific demographic aspects of nonprofessional users, such as patients. These personas need to be taken into account.
•A fair amount of usability insight comes from researching existing sources, such as incident reports, academic literature or databases. It’s useful to research such sources internationally. This may reveal different use patterns and reported incidents that, while specific to one or more countries or locales, should be incorporated into the device. In this sense, post-market surveillance plays a big role in getting feedback from actual device use and reported adverse events, which should inform necessary design and UI changes.
•Labeling, instructions for use and other product information should be included in the usability testing process. This testing is normally conducted well before the given product and its accompanying information are final, using a working draft of the labeling, for example. Only the subsequent summative testing is typically done with a final mock-up as users will see it.
However, since usability testing of medical devices is seen and conducted as an iterative process, it makes sense to involve local users of localized devices to get an international perspective of the product information early on.
In practice, this step is an ideal complement to any readability testing of IFUs that is typically conducted later. This testing is designed to assess how easy it is to read and understand the material. As in usability testing, readability of medical device labeling needs to be tested with a sample of likely, representative users of the device. In the US, the FDA suggests that the reading level of IFUs should be no higher than the eighth-grade level, the average reading level of adults.
In this sense, formative testing of the IFUs for a device can reveal areas of improvement even if the overall readability is as required. It may show, for instance, that changes need to be made to improve images or layout, or to add new instructions that clarify some procedures.
Usability testing is one area where medical device manufacturers spend increasing amounts of time and effort to not only meet regulatory requirements, but also to improve the overall design and UI of devices for easier — and safe — use. Incorporating global UX considerations early on is thus emerging as the next big idea that will ensure devices are truly usable across markets, locales and languages.
