This article is the third in a three-part series, where I reflect on the challenges we have experienced in MEDEI while implementing the medical device software life cycle standard into our agile software development.
In this series, I describe my experience with implementing the required procedures for ISO-62304, and which tools and methods I use to define a structured way for developing high quality software for healthcare. The purpose is of this series is not to act as a guide, but rather to give inspiration to others embarking on the same journey.
Part III: There is no golden rule – just documentation.
In the previous two posts, I have covered the more theoretical parts of this series. I’ve considered the extensive amount of documentation often required for medical device software development and the vital processes of integrating risk-analysis into activities related to design and development. It’s now time to discuss the challenges related to the actual software development.
One of the major changes we’ve had to make in our workflow has been to improve software testing and source code reviewing. This is a part of optimising our activities to better comply with the medical device software “V-model”.
Complying with the “V-model” can be an immense task for a development team when the knowledge of validation, verification, integration, or unit-tests is minimal – which it was in our case. The purpose of all of these tests is to verify that the design and architecture implemented coheres with the one that has been documented.
However, when you first read the ISO 62304 standard, many of the clauses used to describe a software, and the different parts to be verified, can be easily misinterpreted.
Not many of our developers had previously been involved in development projects requiring implementation of any kind of verification. Some had written unit-tests, but none as important as those tests we have to implement into a software with a safety class C (more on that below).
Both software safety classifications, and methods of verification highly affect the total development time. It was therefore crucial for us to integrate as much of the verification work as possible into the development sprints themselves in order to spare time on the long run.
Software segregation and classification
When designing a software artefact, the initial system architecture along with initial risk analysis made on the design, should reveal the safety classification of the software system (and its items). It is highly important to have the safety classification of a software system established before initiating detailed system design, or development.
The standards define the difference between each safety classification, but what is more important to note here is how segregation between software-parts and their classification works.
The 62304 standard uses definitions like product, system, components, item, and unit, to describe parts of a software and the design requirements for each part. At first these definitions can seem redundant. However, considering how complex software architecture can become it becomes more clear when put into perspective.
A product can consist of both hardware and software components. A software system is a complete system consisting of one or more software components. A software component can either function as a stand-alone component, or require integration with another component. A well known example of two software components functioning together to create a software system is a front-end component and an API component.
Software items are used to define any parts of a software which are underlying a component. Lastly, a unit is the smallest part of a software item, which often is defined as a “class” (dependent upon the programming language) as well. When designing a software system, it is essential to clarify the safety class of each software item. This is mainly due to the fact that items classified as either B or C, have stricter design and unit-level verification requirements than class A items. This allows for lowered design requirements for items not critical in safety of the user or patient, and requires extended design requirements and verification for those items classified as B or C. Thereby also affecting the development time, and verification time spent on each item.
If you have a system consisting of both A, B and C items, you must allow for, and define, segregation between all relevant software items, and clarify the classification of each item, whether it’s A, B, or C.
From personal experience, I know that verification and general software testing can take a lot of time. Especially if unit-tests are to be implemented in every single class, you can almost “double-up” on every estimate for each task. Therefore, be wise when designing your architecture (i.e. allow for segregation between items with different safety classification). Not not only will it save you time during development, but total time spent on testing and verification as well.
At MEDEI we have defined SOP’s that describe how we are to handle verification of software systems containing all, or any of the three safety classes. For those containing items with software safety class C, we must perform at least two types of unit-level verification – which are unit-tests and code-reviews. This is done continuously during development (sprints), where our JIRA workflow has been highly customised to comply with these procedures. Together with JIRA, and Git, we have managed to streamline a workflow which allows for unit tests to be both reviewed, during code reviews, and afterwards automatically tested and verified using Jenkins.
Of course, unit-testing and automation of such tests highly depends upon the source code language. However, most of the systems we’ve designed and developed up until now, have allowed us to perform automated unit tests.
Depending on the size of the sprint, we plan code reviews to be performed either once or twice a week, where all participants of the sprint either meet physically, or online, and review all source code, for each of the tasks containing safety class B and C items.
Afterwards, all tasks are placed into the “building & testing” where Jenkins automatically performs unit tests every evening, and returns a report to the team when done. Not only does this ensure that verification on unit level is performed sufficiently, but integrates it into our development workflow as well, allowing us to minimise the total time spent on unit-level verification.
Final testing and release
Before release, system verification and validation must be performed according to test cases specified in our verification and validation test plans (can however be distinguished for class A software). Depending upon the system being tested, test cases can either be completed on-site, or ad-hoc.
You could say that the verification and validation tests are the last pieces of the puzzle. However, there are other phases involved in the complete software lifecycle which I haven’t touched upon, which cover release, change requests, problem resolution, support, and maintenance. These phases are rather trivial as much of the activities involved can highly vary between manufacturers and types of software.
I must admit that although I’ve spent hours and hours in frustration trying to understand the many trivial paragraphs found in the ISO 62304 standard, I’m glad that it’s written in this manner as it allows for a certain degree of freedom when designing an artefact.
Although the different medical device standards clearly define what you should do, they do not describe how things should be done. I therefore must emphasize once again, that it is not a matter of how you we do things, weather we use word- or paper-documents to document our architecture, or weather we use JIRA or Post-it notes to keep track of your development tasks.
What matters is that you have to be able to document what you’ve done. Because as long as you can clearly state that something was completed in a certain way at a certain time using specific methods/tools, per your plan, you should be at least half way there.
Designing a medical device software is not an easy task, and requires both knowledge within the area and out-of-the-box thinking in order to fulfil the extensive requirements, and cohere to the wildest of expectations. It doesn’t take a genius to realise that much of the solutions being used within the medical field today are highly sophisticated software solutions.
Software is not only modernising our workflow within the healthcare industry, but assisting with creating new treatment methods as well. Software has become, and will continue to be the cornerstone of almost all workflow within the healthcare industry. However, many solutions are already becoming out-of-date, and we all know how IT-infrastructure handles out-of-date solutions – they start to fail.
At MEDEI we are determined to overcome this. We believe that if we are to improve people’s health and well being, we need to keep up with technological evolution, and be more efficient when implementing new treatment methods.
By designing and producing quality software solutions, and assisting others in doing the same, we are not only contributing to a more modernised healthcare industry, but doing so at the same level as other industries.