The assessment and mitigation of safety-related risks is arguably one of the most important parts of the medical device development process. Every medical device, regardless of its nature or intended purpose, carries health and safety risks of some sort that absolutely must be addressed as thoroughly as possible before the device is assessed in an agency submission and ultimately released to the market. This is to assure the protection of patients and those who care for them while fulfilling a myriad of regulatory requirements set forth by the FDA and in international safety standards. Human factors are a key aspect in evaluating and managing the safety risks of medical products and having a clear understanding of use related risks constitutes a strong foundation for effective risk analysis. Dr. Mary Beth Privitera is the principle of HFE/Research for HS Design, co-chair, and faculty at the Association for the Advancement of Medical Instrumentation Human Engineering Committee, and editor of the recently published Applied Human Factors in Medical Device Design. Outlining the contents of this human factors toolbox, Dr. Privitera dives into the processes of safety-related risk assessment and particularly the importance of its intersection with human factors in use-related risk analysis activities.
Pre-Market Studies
Use-related risk analysis is the first step in the safety assessment process and entails identifying parts of the device or its operation where there lies the potential for use error. Covered in Applied Human Factors in Medical Device Design by Sophia Kalita and Melissa Lemke, use-related risk analysis begins by defining the intended use of the device, discerning where safety risks might arise, and determining whether those risks are severe enough to be deemed unacceptable. Task analysis, discussed by Dr. Privitera in previous hsDNA episodes, provides the “backbone” for this process. This type of analysis entails breaking down the tasks
that a user must complete to operate the device in great detail, viewing the user interface from the perspective of the patient or caregiver rather than as a member of a design team. A key element in this process is the approach of identifying the perceptual, cognitive, and action aspects of device use. This entails identifying what the user must see or hear, comprehend, and then act on in order to operate the device. When applied to use-related risk analysis, having a strong, thorough task analysis record to refer to will not only assist in the identification of potential risks but also in the determination of root causes.
Once risks have been identified and management measures are deemed necessary, root cause analysis, outlined in the textbook by Tim Reeves, Christina Mendat, and Eric Shaver, is the next important order of business. This step includes identifying the causality, the why a use error occurs, and what are the primary factors in the design of the user interface which contribute to those errors, and how they may be mitigated or prevented altogether. The foremost methodology for defining root causes is to, per Dr. Privitera, “ask five whys focused on the user interface.” “If you ask why five times, then you’ll eventually get to the true root of the answer,” she explains, continuing by noting that the answers to those “whys” become increasingly applicable as one continues to ask.
Interestingly, Dr. Privitera reveals that there has been a shift over the years regarding use errors and key terminology often found in risk analysis. The “old opinion,” allowing for the term user error and “new opinion” of use error. Previously, user error was viewed as a failing on the part of the healthcare provider; they made the mistake and thus were at fault for it. This opinion is cautionary at best in that blame for design weaknesses are now pushed onto the user wherein often the user had no choice by design. As ideas about human factors and their place in medical device design have evolved, human error has come to be viewed as the
consequence of a particular design issue, with responsibility falling onto the manufacturer or developer rather than the user, a view that is reflected in current guidance documents. Humans will always make mistakes; however, devices should be designed to make mistakes as hard as possible to make. The term useR error is never appropriate in human factors studies. Human factor studies evaluate the user interface, not the user.
With use-related risk analysis performed and root causes identified, risk control measures may be assessed for residual risk. This feeds into the overall risk evaluation and eventually added to the usability file. While this promotes safety on the onset, safety-related practices do not end when the product has been released, rather they continue in constituting a rigorous and continuous process of post-market surveillance.
Post-Market Surveillance and Legacy Devices
Once a device is in the field, use-related information should be captured. This can include any use related errors or challenges that occurred during its operation. While the primary reason for this is to provide good quality customer service and support to users, Dr. Privitera emphasizes there is also the importance of utilizing this information in design processes as well, asserting that this aspect has been “glossed over” in years past. Capturing post-market information that is useful in identifying design weaknesses can be a challenge if detailed information is not captured. When device-related issues are voluntarily reported, they are often stored in databases like FDA’s MAUDE database, the Manufacturer and User Facility Device Experience, where they may be referenced and considered during the design process. Many devices have
known use-related errors that developers are expected to account for and circumvent in their new device releases. Indeed, it is expected that these databases, as well as the existing marketplace more generally, will be examined during the design process.
Legacy devices, or devices of unknown provenance, utilize post-market safety-risk analysis in order to update files and adhere to contemporary standards. In these cases, the use specification must be defined, post-market analysis performed, hazards and risk controls attended to; in short, legacy devices must pass all the same evaluations as devices currently being released to the market. The challenge with this is that a device may have reports into the tens of thousands, each of which must be attended to with equal attention. Further, databases themselves have not always been particularly easy to navigate, previously lacking filters that
make identifying intended users and other relevant information much harder. With an attitude focused on patient safety, these records can impact the design of novel devices.
In the event that a new version of a legacy device is being designed, post-market analysis is essential in a number of ways. FDA guidance conveniently provides a database list that should be utilized during development for the purposes of identifying existing use related issues. However, these are not the only databases with which to search. For example, customer complaint files could be examined to glean how the device has performed in the market thus far. If use errors are present, risk controls must be evaluated to determine whether they are adequate, and residual risk evaluations must be performed. When releasing a new device based off a legacy device with no changes to the user interface, Dr. Privitera describes the process as “an exercise in diligent research and documentation,” confirming safety through the analysis and updating usability files. Making significant changes to the user interface, however, complicates matters somewhat. In these cases, post-market analysis of previous iterations is highly important in determining which aspects of the user interface have become habitual and should, therefore, be maintained in order to prevent the negative transfer of behavior (e.g. inadvertent operation of a critical control and/or not meeting the user’s expectations). In summary, for some cases, features of a legacy device’s operation become known behaviors, and any modification to these behaviors may actually invite user error rather than offset it.
Additionally, further human factors work and safety-related analyses may need to be performed to ascertain whether changes made will negatively impact the overall user experience. The rule of thumb when changing user interface, Dr. Privitera says, is “ to develop a clear human factors strategy to assess usability, safety, and risk in consideration of other like devices and those devices that may work in concert with the device under development.”
Addressing safety-related risks is one of the most important aspects of a device’s development. Yet, it is not a one-and-done step on the development journey. Instead, it is an ongoing process that Dr. Privitera asserts should begin as early as possible and must continue well after the device is released to the market in the form of surveillance. This begins when conceptualizing a device, designers should be considering the ways in which an error could occur and how it might be prevented. In her words, “It’s more of an iterative approach where you’ve got to follow through with the details.” For more information about human factors and use-related risk analysis, take a look at Dr. Privitera’s Applied Human Factors in Medical Device Design, available on Amazon.