Nine Questions for Medical Device Software and SAMD Manufacturer

What Regulations, Standards, Guidelines Apply to Medical Device Software and SAMD?

​

The story of compliance starts at the top with the FDA regulation regarding the quality system of the medical device as documented in FDA 21 CFR Part 820 that is a part of Title 21 Chapter I Subchapter H, which relates to Medical devices. There are 31 sections in Part 820. Among all these sections, §820.30 that describes Design Control is the most important for the specifying, designing and developing software.

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

As far as medical device software is concerned, there are three legs under this regulation – Quality, Risk and Software Development management. Quality and Risk apply to all medical devices whereas the Medical Device Software related documents are applicable only to software components. We need to tailor the requirements for documents and processes described in Quality and Risk management guidelines for software components.

​

For Design Control FDA has a complete document published in 1997. It is called Design Control Guidance For Medical Device Manufacturers. Though in the 20 years since this document’s release, there have been massive changes in the technology in general and software in particular, this document provides some high-level guidelines which are still very relevant. This document also refers to ISO 13485 for detail description in various sections.

​

ISO 13485 is the primary standard for implementing a Quality Management System (QMS). Most of the sections under FDA 21 CFR Part 820 map to one or more sections in ISO 13485. The current version of ISO 13485 is released in 2016. The first version based on ISO 9001 standard was published in 1996 and referred by FDA in “Design Control Guidance For Medical Device Manufacturers” document. The earlier 2003 version has been used for all medical devices currently available in the market.

For Risk Management ISO 14971 is the primary standard for all medical devices. The latest major version was released in 2007, with some minor updates published in 2009. ISO 14971 and ISO 13485 refer to various sections of each other, which shows the interdependence of these two standards.

 

So far, all the documents mentioned here are relevant to all medical devices including the medical device software. The next set of documents are specifically applicable to medical device software.

​

What is SaMD?

As per FDA’s website the term Software as a Medical Device is defined by the International Medical Device Regulators Forum (IMDRF) as "software intended to be used for one or more medical purposes that perform these purposes without being part of a hardware medical device."

​

Let us first define what is NOT SaMD. In the past most medical device software were embedded in the hardware device like firmware. Currently there are the class of medical device software that run on mobile device or Cloud but relies on hardware devices to acquire medical data either machine or sensor generated or manually entered. This class of software are not SaMD and they are referred in general as Medical Device Software. The thumb rule is, if a software is needed for the intended use of the hardware device then it is not SaMD.

​

On the other hand, a new class of pure-play software applications are being developed which rely on data from various sources and does not need a particular hardware device from the same manufacturer as the software producer. Even if a software needs a hardware device but the hardware device does not need the software for its intended use, it is an SaMD. Especially the software that runs on general purpose computers and uses general purpose hardware (like off-the-shelf camera) are also SaMD. An example of SaMD will be an Artificial Intelligence (AI) based diagnostic software which uses patients various pathological test results to predict a future likelihood of a critical condition like sepsis or cancer. Or another software that uses digital X-Ray or MRI image to diagnose various medical conditions is an SaMD. There are some great examples of SaMD and Not SaMD in this FDA site.

​

For defining a regulatory framework for medical software, FDA relied on IMDRF: International Medical Device Regulators Forum in which FDA is a member. In their own words, IMDRF is a voluntary group of medical device regulators from around the world who have come together to accelerate international medical device regulatory harmonization and convergence. IMDRF developed a regulatory framework for Software as Medical Device (SaMD) – Clinical Evaluation that FDA issued as part of its Guidance for Industry and Food and Drug Administration Staff on December 8, 2017. This is the same document as of IMDRF – N41 SaMD Clinical Evaluation. This document uses three other IMDRF documents as a reference. IMDRF – N10 SaMD: Key Definitions describe the definitions of key terms. For quality management as applicable to SaMD, IMDRF – N23 SaMD: Application of Quality Management System documents the guideline.

​

IMDRF created the guideline for Risk Management for SaMD in a curiously named document: IMDRF – N12 SaMD: Possible Framework for Risk Categorization and Corresponding Considerations. Interestingly it does not refer to ISO 14971 Risk Management document. This document defines various risk scenarios and includes the classification of SaMD software based on their risk profile.

​

 

 

 

 

 

Can Medical Device Software and SaMD Use Agile Methodology for Development?

This question of the suitability of Agile methodology to develop Medical device Software is a recurring one for more than a decade. An apparent conflict between the FDA regulations requirements of detail documentation and Agile methodology’s perception of low (or no) documentation is the constant source of this question. The short answer to this question is ‘Yes’ and the long answer is ‘Absolutely, definitely YES’.

For SDLC – Software Development Lifecycle or as it is known now as Application Lifecycle Management (ALM) of medical devices, the definitive guide is IEC 62304 Medical device software – Software life cycle processes. However, this does not specify what development methodology to use. In fact, according to FDA:

​

“The quality system requirements do not dictate the types of design process that a manufacturer must use. Manufacturers should use processes best suited to their needs. However, whatever the processes may be, it is important that the design controls are applied in an appropriate manner.”

FDA Design Control Guidance for Medical Device Manufacturers, p. 8

​

“Based on the intended use and the safety risk associated with the software to be developed, the software developer should determine the specific approach, the combination of techniques to be used, and the level of effort to be applied. While this guidance does not recommend any specific life cycle model or any specific technique or method, it does recommend that software validation and verification activities be conducted throughout the entire software life cycle.”

General Principles of Software Validation (GPSV), p. 1

​

In the last decade, organizations are using the Agile SCRUM methodology as effective development method for diverse types of software in various industries. Several research groups have documented the enhanced quality and success rate of Agile based projects over those using a more traditional Waterfall methodology. So, it is natural to use Agile for Medical device Software as well so long it can be aligned with IEC 62304, the guiding standard for medical software development.

​

Association for the Advancement of Medical Instrumentation (AAMI) developed a guide for using Agile methodology for medical device software development and mapped them to the IEC 62304 requirements in a document AAMI TIR45: 2012 Guidance on the use of AGILE practices in the development of medical device software. FDA explicitly recognized agile as an acceptable standard in January of 2013, referring to AAMI TIR45: 2012 and added it to its list of the approved standards to be used by the manufacturers. The FDA acceptance takes care of the question of using Agile methodology for Medical device Software once and for all.

​

How To Read FDA Quality Regulation for Medical Device Software and SaMD?

The Quality Management Design Control as described in FDA 21 CFR Part 820.30, requires the Medical Device Manufacturer to have plan and implementation of some distinct process areas. We explain each of these in the context of software development processes as follows:

• Design and development planning

process refers to creating a document defining all the complete processes of design, development, verification, validation, and documentation of the software product.

• User Needs and Intended Uses

Though explicitly User Needs is not specified as a process area, either in FDA 21 CFR Part 820.30 or ISO 13485, it in several descriptions in conjunction with ‘Intended uses’ as the conformance items to which the medical device is validated. It is to include User Needs as separate items to track and manage as part of the development. In software Engineering terms the User Needs correspond to Marketing Requirements that can have various sources including Market needs, Regulations, Competitions, Security, or Performance.

• Design Inputs

We can derive Design Inputs from the User Needs as Functional Requirements. These are the requirements that are to be designed and implemented as Medical Device Software. In Agile these to the ‘epics.

• Design Outputs

These are the product design elements derived from Design Inputs. For Medical Device Software this might include the user Journey, user experience (UX) design, user interface (UI) design, state machine, database designs, security, design elements.

• Design Review

We need to include this step at every stage of the development process. Each of the User Needs, Design Inputs, Design Outputs is to be reviewed and approved by the authority in the manufacturer organization.

• Design Verification

Design Verification proves that we have developed the product right. Verification plays an integral role in confirming that we have created the design outputs as per the Design Inputs. The in the Medical Device Software context includes the review and of the product design, source codes and various like unit, integration, and the acceptance criteria. V is a static process which means we need to run the software for verification.

• Design Validation

Design Validation proves that we have developed the right product. need to validate the product against the User Needs, Design Inputs and requirements needs and intended Against all the different like the , integration, and the acceptance criteria the product software to run to see if it passes all .

• Design transfer

For design transfer means documenting the process of transferring the medical device from prototype to manufacturing. For most modern software, this is equivalent to the the software to deployment often on cloud infrastructure with hardened (highly secured and reliable) environment. this will include the same set of validation process to run to prove that the transfer has not broken any part of the system.

• Design changes

Changes in the product design can happen during the development as well after the release. The source of can be due to variation in the user needs, intended use, regulation, or technology. They can impact the current product from negligible to extensive. Change management needs a complete process for review by a Change Control Board (CCB), impact analysis to Design Inputs, Outputs, and plan to implement them with the minimum interruption.

• Design history file

The design history file keeps the complete history of all the activities during the development process. It might be to keep all in one file other and tools will be an acceptable form of the design history file. Alternatively, can keep all the audit trails and histories of all activities.

 

 

 

 

 

 

​How to Use Agile Development Methodology with Quality Management System?

To match the Design Control concepts to the Agile Development Methodology, we need to start with explaining some basic Agile concepts in the Medical Device Software context. The detailed overview of Agile methodology is out of the scope of this article; I have included some excellent resources in the Bibliography in the end.

​

​

​

​

​

​

​

​

​

Backlog

A backlog is a prioritized list of items which needs time and resource to take care. The backlog items can include a requirement (user need, design input, design output), change requests (design change, enhancements), issues, bugs. The Agile term for Backlog requirements is ‘User Story.’ For this article, I will be using a common term ‘requirement’ to refer to User Needs, Design Inputs and User Stories. 
Product Backlog is a laundry list of all requirements, change, issues, and bugs that we can implement in some releases in the product. This might even include low priority items which might never see the light. Product Backlog is a dynamic list which the product manager (product owner as per Agile terminology) augments and reprioritizes continuously.

 

Releases
The product is released when it ‘ships’ to the users who can start using it. In some industries, product releases might happen as frequent as daily! For Medical Device Software, each product release goes through a lengthy FDA approval process and hence it needs a lot more planning and documentation. For many Medical Device Software, since it requires synchronization with hardware development, the typical release period might be 6 to 24 months.

Release Backlog is a subset of Product Backlog that the product manager selects to include in a particular release of the product. The Agile methodology allows the product manager to change this list during the development of the release. The product manager might add and remove items to this list and reprioritize them based on changed circumstances.

 

Sprints

A Release consists of multiple Sprints which produce ‘potentially shippable product.’ In other words, each Sprint is a mini release. However, for Medical Device Software, a Sprint is not a shippable product, or a release since each product release needs to go through an FDA approval process. Sprints allow the development team to continuously improve the product and incorporating any changes throughout the development timeline. Since a Sprint includes multiple review, verification and validation processes, the Sprint duration might be 4 to 6 weeks.
Sprint Backlog is a subset of Release Backlog which a Sprint implements. The product manager selects the highest priority backlog items from the Release Backlog to be included in a Sprint Backlog. Once the Sprint starts, the Sprint Backlog cannot be changed. However, the product manager can change any future Sprint Backlog before its start. 

​

Definition of Done (DoD)
When can we declare a Sprint is to be ‘Done’? Even when the developers commit their codes and a build of the software is produced without any error, from a Sprint point of view it is not done yet. The development team needs to complete several additional activities before it can be called ‘Done.’ This list of activities is called ‘Definition of Done.’ For Medical Device Software this list might include the following:

• Verifications - each Design Input and Design Output (requirement), included in the Sprint are reviewed and verified

• Code completed and committed 

• Peer Code Review performed

• Refactoring completed

• Unit tests – created, run and passed

• Product builds without errors

• Validations - All manual and automated acceptance tests are created, run and passed

• Any configuration or build changes documented

• All Documentation updated as applicable

 

Sprint Review
When a Sprint ends, the Sprint Review event allows the Agile team to demonstrate the completed requirements to the product manager and other stakeholders with evidence of the completion of DoD.
 

How Quality and Risk Management Processes Work for Medical Device Software and SAMD?

​

​

​

​

​

​

​

​

​

​

The Risk Management is the background in which the Medical Device Software development takes place. ISO 14971 is the definitive guide for Risk Management for all medical devices. We need to contextualize it for Medical Device Software. FDA has its guidance for the industry called “Q9 Quality Risk Management” where they discuss Risk and benefit assessments, Risk control, Risk communication and Risk Review. The high-level risk management processes as described in FDA and ISO documents look like the following. As we can see, they are not the same but share some common elements.
 

Risk Assessment

The questions to ask are –
1. What might go wrong?
2. What is the likelihood (probability) it will go wrong?
3. What are the consequences (severity)?

 

Risk identification is a systematic use of information to identify hazards referring to the risk question or problem description. This addresses the first question of “What might go wrong?”
 

Risk analysis is the estimation of the risk associated with the identified hazards. It is the qualitative or quantitative process of linking the likelihood of occurrence and severity of harms. This refers to the second and third questions “What is the likelihood (probability) it will go wrong?” and “What are the consequences (severity)?”
 

Risk evaluation compares the identified and analyzed risk against given risk criteria. Risk evaluations consider the strength of evidence for all three of the fundamental questions.

 

Risk Control

Once risk is identified, its probability and severity are determined. The next question is how to control or mitigate the risk. Risk control will try to answer the following questions:
•    Is the risk above an acceptable level?
•    What can be done to reduce or eliminate risks?
•    What is the appropriate balance among benefits, risks and resources?
•    Are new risks introduced as a result of the identified risks being controlled?
 

Risk reduction focuses on processes for mitigation or avoidance of quality risk when it exceeds a specified (acceptable) level (see Fig. 1). Risk reduction might include actions taken to mitigate the severity and probability of harm.

Risk acceptance is a decision to accept a risk. Risk acceptance can be a formal decision to accept the residual risk, or it can be a passive decision in which residual risks are not specified. For some types of harms, even the best quality risk management practices might not entirely eliminate risk. In these circumstances, it might be agreed that an appropriate quality risk management strategy has been applied and that quality risk is reduced to a specified (acceptable) level.  

FDA being so concerned about the safety of any medical device, it is no surprise that Risk Management plays a central role in the whole compliance process. However, it is not a stand-alone process, but it takes place at every stage of the design and development of any Medical Device Software. For example, to answer the fair question – “when the right time is to identify a risk?” the answer is at every step and level of Design Control and Quality Management process. The following diagram shows that the risk identification should be systematically done while performing every Design Control process including those for User Needs, User Inputs and User Outputs.

​

​

 

 

 

 

 

 

​
 

 

 

The best place to identify Risks is while we create new User Needs, Design Inputs or Design Outputs. It provides the right time and also the correct context to ask the three questions described above. By doing risk assessment, if we can identify new Risks, they follow their Risk Management processes. It is important to note that while doing risk control step in the Risk Management process, for mitigation of the risk, new requirements – User Needs, Design Input or Design Output might be created. This interdependence between the QMS and Risk Management allows a systematic way of identifying and controlling Risks.
 

How to Define Traceability for Medical Device Software and SAMD

There are many different objects and items are being created, tracked, managed, and processed as part of QMS, Risk and Agile management processes in a Medical Device Software development project. What will be the best way to answer the following questions: 
•    How are they related to each other?
•    Which other items get impacted if we change one?
•    Which items fell through the crack?
•    In case of an issue, how to do the root-cause analysis?
To answer all these questions, we can use a handy tool called Traceability Analysis. In traditional Requirement Management, Traceability allows us to define a relationship between two items of the same or different kinds. These may be a User Need, Design Input, Design Output, Backlog Item, Testcase, Issue, Change or Document. For example, we can list all related items (Design Input, Design Output, Backlog Item, Testcase, Issue, Change) in the context of a single User Need. Now when that User Need changes, we know what other items need to be reviewed and determine if they are any way impacted. This is called impact analysis. Similarly, we should be able to list all User Needs, which do not have any items related to them. Which either means that one such User Need is not required or it has not been treated systematically to follow the Design Control process. These items are called Orphans in the context of Traceability. 

 

 

 

 

 

 

​

​

 

The traceability can be unidirectional - impact flows in the direction of the arrow strictly from one item to another. For example, unidirectional traceability from a User Need to Design Input means the change in the User Need impacts the Design Input. Instead the change in the same Design Input does not impact the User Need. In some cases, traceability can be bi-directional where a change in either of the items impacts the other related item. The example is the relation between two Design Inputs where a change in each can impact the other.
 
How to implement Medical Device Software and SAMD Development and Compliance System?
The system for Quality, Risk and Agile Management system can be implemented manually using various textual and spreadsheet documents. In that case, it will be our responsibility to define and adhere to each of the processes and ensure that all team members follow the process without any shortcuts. As we have seen, there are a few inter-related processes that need to cross the organizational boundary of a medical device manufacturer. For example, the Quality Assurance (QA) department is responsible for Quality Management and Risk Management. The software development team does the Agile development. Manual implementation of all these processes, although quite doable, will be slower and prone to mistakes and often slips.
It will be much easier to implement using a tool or integrated tools that can share various information for all the processes. The tools that enable automation of the processes will remove the uncertainty of process enforcement and also record many of the critical information like who, what and when automatically. Though a single tool which can implement all the procedures and traceability is preferable, using multiple integrated tools is the second best alternative and can reduce much of headache of the manual implementation.  

Conclusions
The development and compliance of Medical Device Software and Software as Medical Device (SaMD) need not be a hairy mess. Systematic implementation of various processes and enforcing them with zeal will make us the winner. 

• Create processes that we can manage and follow. Do not wait for a perfect process. A working process is better than no process.

• Document every aspect of the quality, risk and development processes. Document Who-what-when for every step.

• Perform Verification and Validation for every aspect of the Medical Device Software at every step, multiple times if necessary.

• Include Risk Management at every step of the Design Control and development process.

• Use traceability between different items to do impact analysis, orphan analysis, and root-cause analysis.

• Use tools to automate as much of the processes as possible.

​​

Bibliography
1.    FDA - 21 CFR Part 820, US Food and Drug Administration.
2.    Design Control Guidance For Medical Device Manufacturers, US Food and Drug Administration, 1997.
3.    ISO 13485:2016, International Organization for Standardization (ISO), 2016.
4.    ISO 14971: 2007, International Organization for Standardization (ISO), 2007.
5.    Agile Manifesto
6.    Official Scrum Guide, Ken Schwaber and Jeff Sutherland, 2017

​

​

​

​

​