Skip to main content

A blog on why usability is key to ensuring your medical device is ready for clinical evaluations

Engaging your end-users at key stages is an important part of a user-centric development of your medical products. Clinical evaluations are expensive and time-consuming so early-stage voice of the customer research and usability engineering can help to ensure that you are on the right path. Developing medical products that are inherently safe by design, enhances the user experience with your product while meeting the regulatory requirements and your commercial needs.

At eg technology, we live and breathe integrated end-to-end development. Voice of the customer, human Factors and usability engineering are integral processes and drive insight-led, effective product development, ultimately leading to better, safer products.

Voice of the customer is a critical step in gathering insight into user need to establish a solid basis for user and product requirements, while usability engineering aims to assess the medical product user interface in terms of use error.

Usability testing does not and cannot replace clinical evaluations and the two should not be conflated.

Usability testing versus clinical evaluations

Both are critical parts of any medical device or in-vitro diagnostic development programme, and both involve interacting with your end-users, but they are asking fundamentally different questions. The purpose of usability testing is to assess the product’s user interface in terms of use risk whereas the purpose of a clinical evaluation is the ultimate validation of clinical safety & efficacy. This is why the full usability process, up to and including summative usability testing should be completed before conducting any clinical evaluations with your product. During usability tests, use of the product is simulated, without delivering any actual diagnostic or therapeutic effect, whereas in clinical evaluations the clinical effect is delivered, and its efficacy assessed. As such, summative usability studies typically involve circa 15 participants per user group, whereas clinical evaluations often require thousands of participants.

In a usability test, we want to observe how users might use the product imperfectly, to determine what use errors occur because of our design; in a clinical evaluation, we want to remove the chance of use error occurring, to determine the clinical effectiveness of the product when used perfectly. Variation is deliberately tested by observation and assessment is more subjective in usability studies, whereas variation is deliberately minimised or controlled in clinical evaluations and the aim is to produce objective evidence.

Why is usability engineering important?

Human factors and usability are becoming more and more prevalent in the design and development process. So much so, that many organisations are placing a focus on the human factors element of their product development.

Humans are fallible and poorly designed UI can lead to error and potential harm. An example of this was the UI of a blood glucose monitor, on which the decimal point, separating numbers, was difficult to decipher. The consequence of this was at home patients misreading their blood glucose levels as 22mmol/L as opposed to 2.2mmol/L and therefore administering a much larger dose of insulin than was required, leading to diabetic coma. This shows how a small usability oversight can have catastrophic consequences.

Problems such as these are becoming increasingly problematic, as at-home devices and remote monitoring become more prevalent, increasingly complex and operated by unskilled users. Clinicians are also having to deal with an increasing number of devices with complex or conflicting user interfaces. This is why it is critical to engage users as early into the development process as possible. The sooner any issues are identified, the sooner the risks can be mitigated, and changes incorporated. The benefits of this are not only from a safety point of view but also commercial. A demonstrably safe, intuitive, and effective product that has been developed with the user, will be less likely to receive negative client reviews, fewer recalls and have fewer associated training costs.

Challenge assumptions

Early user input from voice of the customer research will force you to challenge assumptions. By considering all potential use scenarios, you can then factor them into your development and design out any potential errors, early. User expectations have shifted with the prevalence of wearable devices, especially as many now have the appearance of being ‘medical’ – such as health trackers. Users, therefore, have an expectation as to how the UI should look, feel and operate, as well as how the device itself looks, how it charges and how it fits into their daily lives. This expectation is obviously dependent on the situation in which the device is used, and by whom, but as consumer devices evolve, medical UIs are expected to assimilate.

User studies should therefore allow you to assess expectations and challenge assumptions within your development. When we were working with CFT, the general assumption was that the device should be entirely handheld and portable, but with user insight, we soon realised that in an A&E environment, that handheld wasn’t the best option. Without user and situational feedback, we would have continued with a development that would have yielded a product that was not suitable and would therefore have caused costly redesigns. This consultation process not only mitigates financial impacts but can actually provide a competitive edge within the market, as your device will be something that users need and want! Check out the full CFT case study here to look at the full development process.

What do the regulations and standards ask for?

IEC 62366 is the recognised standard for managing use risks. With the recently implemented MDR and the imminent IVDR on 26th May, more emphasis has been placed on usability. The process for each is similar, with the overarching requirement being that you demonstrate your device is both safe and effective for use, in the intended situation, by the intended user group.

Considerations need to be made regarding the device’s potential for harm, whether it will be used by laypeople and whether the UI is complex or novel to the user. The scale of development work, therefore, needs to be relevant to the scale of risk. By demonstrating you have not only planned but also implemented the integration of human factors or usability studies into your development, you are on your way to achieving compliance. Of course, with usability spanning the entire development, traceability is key and being able to provide the required documentation and ensuring usability feeds into your final engineering file, you should be on track for developing a final product that is not only safe and effective to use, but also market-ready and compliant to the required usability standards and regulations.

What the usability process involves

1

Insight & Concept Development

In this stage, it is important to really get to know and understand your users, their motivations, requirements, the use environment and even their capabilities and limitations. This primary user research stage overlaps with the former ‘Voice of the Customer’ work that is ideally done in the early stage of product ideation as part of your market research.

Once the use specification has been detailed, you define the risk and UI specification to ensure that any known issues and critical tasks are identified and written into your programme. The output of this stage is a living document that should evolve with your development as updates are incorporated.

2

Design Implementation & Verification

The key part of this stage is the evaluation and mitigation of any use risks identified. By moving through prototype phases, results of any early user testing known as formative evaluations can be fed back into the design through iteration.
3

Transfer to Manufacture & Validation

Finally, the last stage of the process focuses on validation through summative evaluation and the production of the usability engineering documentation. If you can prove that any risks identified in stage one have been assessed and reduced and you can prove that the product is safe and effective to use, by the intended users, in the product’s use environment, then this can be fed into the usability engineering documentation.

Where does usability engineering sit within your overall product development process?

The three main stages within a development programme are concept generation, detailed design and development and transfer to manufacture. However, there is more to a programme than these core steps. Other elements happen in parallel, but human factors and usability need to interact constantly. The three usability stages we have already detailed need to align and integrate seamlessly with the main development stages in order to get the best product at the end of the programme.

V-Model

A large number of product recalls are attributed to poor product design. It is therefore imperative that your user interface is based on actual user needs. By challenging assumptions (you’d be surprised at how even the most confident assumptions can be incorrect!) and integrating user insight into your design throughout, risks can be mitigated from the start. It is important to show your iterative design, illustrating how you have mitigated and controlled risks to an acceptable level before clinical evaluations.

Best Practice

Traceability is vital within the design and development process. During the insight and concept development phase, you should be conducting primary research to ensure that your user interface specification is based upon actual user needs (as mentioned above). This research forms part of your User Requirements Specification (URS), which in turn is used to define your Product Requirements Specification (PRS), which is derived from the functional & technical requirements of your product. Having this process fully detailed provides full usability traceability from user insight to product validation.

We have discussed using the V-Model for development verification and validation in a previous blog, dedicated entirely to URS and PRS, so please check that out here. The V-Model will provide you with a process against which you can verify against your product requirements specification and validate again your user needs, demonstrating that you designed the right product and designed the product right.

Finally, to fully demonstrate safe and effective use, multiple evaluation methods should be employed. There are two core techniques used to evaluate use risk in this process. Use risk analysis aims to capture foreseen risks with low likelihood and user studies aim to capture the unforeseen use risks. There are a variety of different tools and methods which can be used depending on factors such as the phase of development, level of details required, and the nature of the UI, but together these two techniques are what make the process effective at mitigating use risk.

To read how we employed a fully integrated product development programme, incorporating design, development and usability, please check out this human factors case study.

Watch the below workshop we carried out with NIHR (National Institute for Health and Care Research) which focuses on the full end-to-end process of product development, from a usability point of view.

Remain part of the conversation. Subscribe to our blog by clicking on the subscribe button at the bottom of your screen!

For more information on getting your product to market or to chat with one of our team about your product design and development requirements, please contact us:

Via email on design@egtechnology.co.uk, by giving us a call on +44 01223 813184, or by clicking here.