Software Integration and System Testing

Different levels answer different questions

Define the system boundary explicitly: firmware, application, cloud and supporting services may be separate software systems with interfaces between them. A test spanning several systems needs a clear requirement allocation. A software-system pass does not establish physical delivery accuracy or clinical suitability.

See MTL-116 — Verification and Validation for the wider evidence strategy.

Integration and software-system testing have different scope

Clause 5.6 addresses software integration and integration testing for Classes B and C. In summary, integrate according to the integration plan, verify the integration, test the assembled items, evaluate test adequacy, retain results and address problems. Regression testing considers whether integration has introduced defects into previously integrated software.

Clause 5.7 addresses software-system testing for Classes A, B and C. Establish tests that cover the software requirements, record results, handle discovered problems and evaluate the testing. Following changes, repeat affected tests and perform appropriate testing to demonstrate that unintended effects have not been introduced.

These are summaries, not a clause-by-clause compliance matrix. Use the controlled licensed standard for exact obligations, class applicability and records. Integration and system testing may be organised together where appropriate, while keeping their objectives and coverage identifiable. MTL-312 — IEC 62304 Software Lifecycle provides the wider process context.

Make the integration sequence deliberate

Begin with the architecture and identify which interactions could defeat a requirement or risk control. Introduce important interfaces early enough to expose incompatible assumptions. Connecting everything at the end makes fault isolation difficult and leaves little time to correct architecture.

1. Identify the items, versions, interfaces and dependencies in each increment.
2. Set entry conditions: accepted units, controlled interface definitions and a usable environment.
3. Specify normal and abnormal interactions with objective expected outcomes.
4. Record which dependencies are real and which remain simulated.
5. Define exit criteria and the regression set before adding the next increment.

A useful increment might connect acquisition firmware to the communication parser before adding storage and presentation. Another might integrate authentication and authorisation before exposing clinical data. The right sequence follows the architecture and project risks; no single sequence suits every device.

Test the assumptions at each boundary

Derive acceptance criteria from approved requirements. “Reconnects successfully” is incomplete when reconnection can lose a record, duplicate an action or display stale information. Record the expected state before, during and after the fault. Cybersecurity tests complement these checks where malicious manipulation changes the threat or acceptance objective.

Match the environment to the claim

Maintain a record of deviations from the intended configuration and explain which conclusions each deviation affects. A simulated sensor may support an alarm-logic test but cannot establish sensor accuracy. Keep final evidence tied to the release candidate or justify its applicability to that baseline.

Test requirements as observable behaviour

Build a coverage view linking each software requirement to tests and results. Include normal workflows, boundary conditions, required fault responses and non-functional requirements. Separate “not executed”, “blocked”, “failed” and “passed”; a test existing in the repository does not establish execution.

For each test, define initial state, configuration, inputs, actions, expected outcomes and objective acceptance limits. Use independent measurements where feasible. If the implementation and expected result use the same faulty calculation, both can agree while the requirement is violated.

Risk controls deserve explicit verification of their effectiveness. A displayed warning alone may be insufficient if the requirement also prohibits an operation or preserves a safe state. Check the whole specified response and its dependencies.

The record was saved, but the acknowledgement was lost

A fictional connected injection device sends a completed-injection event to an app. The app forwards it to a service. This telemetry path does not command drug delivery. The device retains the event until acknowledgement. Each component passes its unit tests, but the service stores duplicate records when a lost acknowledgement causes a retry.

For this teaching example, approved requirements specify: one logical record per event identifier; retries preserve that identifier; pending events survive restart; the app distinguishes pending from synchronised data; and synchronisation completes within 60 seconds after a stable connection returns under the defined test load. These are illustrative product decisions, not universal medical-device requirements.

Integration tests expose the retry and deduplication contract. Software-system tests show that the complete workflow meets persistence, timing and status requirements. Device-level testing must separately establish that this communication failure cannot compromise device functions. Passing telemetry tests does not prove a dose was physically delivered.

The fix might introduce idempotent handling of event identifiers. Review identifier uniqueness, retention and collision behaviour, then test the changed service and all affected clients. Do not close the defect after testing only the new server branch.

Keep evidence reproducible and regression justified

• Identify requirements, interfaces, risk controls and test objectives.
• Record software builds, firmware, dependencies, hardware, settings and test-tool versions.
• Retain procedures, expected outcomes, actual results, logs and reviewer conclusions.
• Link failures to anomaly records, corrective changes and repeat testing.
• Explain remaining limitations and unresolved anomalies before acceptance.

Choose regression from the impact assessment: changed behaviour, shared dependencies, affected interfaces and risk controls. Include unaffected functions where a common resource or configuration could cause an unintended effect. A database-library change can affect record integrity without changing application requirements.

Keep a rationale for tests not repeated. Retesting only changed source lines is usually an inadequate analysis of interaction risk. Preserve automated evidence beyond routine CI retention and ensure a reviewer can reconstruct which configuration produced each result.

Submission detail and development evidence are distinct

Section VI.H of FDA’s June 2023 guidance recommends summaries of unit, integration and system testing for Basic documentation, together with system-level protocols and reports. Enhanced documentation adds unit and integration protocols and reports. Present expected and observed results and pass/fail conclusions, with failures and their disposition explained.

Basic and Enhanced are submission documentation levels, not IEC 62304 safety classes. A shorter submission does not justify inadequate engineering records. This lesson addresses testing evidence; MTL-318 — FDA Device Software Functions — Premarket Submissions covers the broader package. FDA guidance is generally nonbinding; applicable regulatory and QMS obligations still apply.

Review a test report that says “all interfaces passed”

The team tested a single event over a stable connection, used a mock service that always acknowledges immediately, and recorded “pass” without identifying the app build. The report concludes that no records can be lost or duplicated and that no further tests are needed because the FDA submission is Basic.

1. Identify four unsupported conclusions or evidence gaps.
2. Write three fault tests with objective expected outcomes.
3. Allocate those tests to integration or software-system testing and explain the environment needed.
4. Propose a regression set after changing the retry logic.

Can the evidence support the release decision?

• System boundaries and interface responsibilities are explicit.
• Tests address software requirements and important interaction risks.
• Fault and recovery outcomes have objective acceptance criteria.
• Target-specific claims have suitable target evidence.
• Configuration, execution and results are traceable.
• Failures, unresolved anomalies and regression decisions are evaluated.
• Submission summaries accurately reflect the underlying records.