MTL-131: AI and Machine-learning Foundations for Medical Devices

What you will learn

By the end of this topic, you should be able to explain how an AI model differs from the medical-device product around it, describe the principal AI lifecycle stages, distinguish training from inference, recognise common model limitations and identify the controlled evidence needed to move from an experiment to a medical-device function.

The model is only one part of the medical device

Safety and performance depend on the intended purpose, data pipeline, user interface, operating environment, human decisions and lifecycle controls—not only on model accuracy.

AI lifecycle principle

Define the claim, control the complete system, generate independent evidence and monitor the product in real use.

Core concepts

Artificial intelligence

A broad term for machine-based systems that infer from inputs how to generate predictions, content, recommendations or decisions.

Machine learning

A family of methods in which model behaviour is developed using data rather than expressed only as hand-written rules.

Training and inference

Training estimates model parameters from data. Inference applies the resulting model to new inputs during development, validation or use.

Model and product

The model transforms inputs into outputs; the product adds acquisition, preprocessing, presentation, workflow, security, logging and human interaction.

Locked and changing models

A locked model does not alter its learned parameters during use. A planned update process can still release new controlled model versions.

Performance context

A metric has meaning only for a defined population, dataset, operating point, use environment and clinical decision.

A practical lifecycle

Treat AI development as a controlled product lifecycle with explicit hand-offs and traceability.

Frame the clinical problem

Define the intended user, population, input, output, clinical role and harm if the output is wrong, late or unavailable.

Typical evidence: Intended purpose, use scenarios and AI function description.

Specify the data

Define sources, inclusion criteria, labels, partitions, quality checks and populations that the data must represent.

Typical evidence: Dataset specification, lineage and data-quality report.

Develop the model

Control experiments, features, code, parameters, random seeds, libraries and decision thresholds.

Typical evidence: Development plan, experiment records and model card.

Integrate the product

Implement preprocessing, interfaces, workflow, user information, exception handling and safe degraded behaviour.

Typical evidence: Architecture, requirements and interface specifications.

Verify and validate

Test software implementation, technical performance, subgroup performance and clinical use against predefined acceptance criteria.

Typical evidence: Protocols, locked test results and clinical evidence.

Release and monitor

Baseline the complete configuration and monitor real-world inputs, outputs, failures, drift and user outcomes.

Typical evidence: Release record, monitoring plan and change triggers.

Controls to build in

An AI function should be reproducible, bounded and testable before it becomes a controlled medical-device baseline.

Control the complete data-to-output pipeline, including preprocessing and postprocessing.
Separate development, tuning and final evaluation data to reduce optimistic performance estimates.
Version data, code, model artefacts, parameters, thresholds, dependencies and labelling.
Define invalid, low-quality, out-of-distribution and missing-input behaviour.
Trace clinical and risk-control requirements through model and product verification.
State what the model must not be used for as clearly as its intended use.

Evidence to retain

AI system description

The model type, product boundary, inputs, outputs, dependencies and human role.

Dataset record

Provenance, selection, labelling, quality, partitioning, representativeness and limitations.

Model-development record

Experiments, selected configuration, rationale, software environment and reproducibility information.

Evaluation report

Predefined metrics, uncertainty, subgroup results, failure analysis and acceptance decisions.

Common pitfalls

“The algorithm is highly accurate.”

A single aggregate metric does not establish clinical value or acceptable risk.

“The data scientists can manage it separately.”

AI safety requires product, clinical, usability, risk, software, quality and post-market integration.

“The model is locked, so it cannot change.”

Inputs, prevalence, upstream systems and use conditions can change even when model parameters do not.

“More data always improves the model.”

Additional data can reproduce bias, leakage, poor labels or an irrelevant target at greater scale.

Action checklist

Define the intended medical purpose and the model's precise product role.
Identify inputs, outputs, users, affected decisions and foreseeable harms.
Specify data sources, labels, partitions and population coverage.
Control model development and the surrounding software pipeline.
Set performance, subgroup and failure-behaviour acceptance criteria before final testing.
Baseline the released configuration and establish post-market monitoring.

IN DEPTH

Understand what performance numbers actually mean

Learning a relationship is not establishing a clinical explanation

A supervised model learns a relationship between inputs and supplied labels. It can exploit any reproducible signal, including an unintended shortcut. A chest-image classifier might respond to a portable-scanner marker associated with disease in its training set rather than the relevant anatomy. Strong internal performance therefore does not establish that the learned relationship will hold elsewhere. Compare with a simple baseline, inspect plausible shortcuts, test on independent settings and analyse failure cases. Explainability tools may help investigate behaviour, but an attractive heat map is not proof of a clinically valid mechanism.

Separate ranking, decisions and probability

A score can rank patients well without being a well-calibrated probability. Discrimination describes separation between groups; calibration concerns agreement between predicted probabilities and observed outcomes. The operating threshold turns a continuous score into an action and determines the balance of missed cases and unnecessary alerts. Choose that threshold during development using clinical consequences and a predefined objective, then freeze it for evaluation. An area under the ROC curve summarises ranking across thresholds; it does not tell the user the false-alert burden at the threshold actually deployed.

Keep the experiment independent of its examination

Training data fit the model; development validation data guide choices such as architecture, features and thresholds; a final test set evaluates the selected system. Terminology varies, so define these roles explicitly. Repeatedly viewing final test results and adjusting the model turns that set into development information. Separation must match the source of dependence: images from one patient or repeated visits cannot automatically be treated as independent samples. A reproducible experiment also needs the preprocessing, software environment, model weights and output rules, because changing any of these can change predictions.

WORKED DECISION

Why 95% specificity can still create many false alerts

Teaching scenario

A fictional screening model is evaluated on 1,000 independent people, 100 of whom have the target condition. At its fixed threshold it identifies 90 of the 100 cases and incorrectly flags 45 of the 900 people without the condition. These invented figures teach interpretation; they are not acceptable-performance criteria.

1. Build the confusion matrix

There are 90 true positives, 10 false negatives, 45 false positives and 855 true negatives. Sensitivity is 90/100 = 90%; specificity is 855/900 = 95%. Positive predictive value is 90/135 ≈ 66.7%. Accuracy is 945/1,000 = 94.5%, but it hides the ten missed cases and the workload caused by false alerts.

2. Change the deployment population

If prevalence falls to 1%, then among 10,000 people there are 100 cases and 9,900 non-cases. Assuming sensitivity and specificity remain unchanged, the model produces 90 true-positive and 495 false-positive alerts. Positive predictive value falls to 90/585 ≈ 15.4%. The assumption must be tested: changing populations may alter sensitivity and specificity too.

3. Make a product decision

The team evaluates whether the service can investigate the expected alert volume, what happens to missed patients and whether another operating point is justified. It reports uncertainty and checks the complete workflow. It does not select a new threshold on the final test set and present those same results as independent confirmation.

EVIDENCE IN PRACTICE

Example model evaluation summary

This abbreviated teaching example shows the reasoning to capture. Adapt it to the product, risk and quality-system procedures, and link to the underlying evidence.

Configuration: Model, preprocessing and threshold identifiers; exact dataset partition and evaluation code version.
Performance: Confusion-matrix counts, metric definitions, confidence intervals, prevalence, subgroup results and exclusions.
Interpretation: Expected missed cases and review workload for the intended setting; assumptions requiring external evaluation.
Release decision: Named reviewers assess predefined clinical and technical criteria; unresolved limitations are recorded.

PUT IT INTO PRACTICE

Make the decision yourself

A rare-condition dataset contains 990 non-cases and ten cases. A model predicts ‘no condition’ for everyone. What does its 99% accuracy conceal, and what should be reported?

Write down your decision, the missing evidence and the next action before opening the answer.

Read the model answer

It misses all ten cases: sensitivity is 0%, specificity is 100% and positive predictive value cannot be estimated because there are no positive predictions. Report the underlying counts, clinically relevant metrics and uncertainty rather than treating accuracy as evidence of usefulness. Compare the system against the actual clinical task and baseline. A high score on a metric dominated by non-cases does not show that the device identifies the patients it is intended to help.

Apply this to your project

Use the example record above to document one real decision. Identify the assumption most likely to change the conclusion, the evidence needed to test it and the person responsible for the next step.

Read alongside this lesson: IMDRF: Good Machine Learning Practice — connect independent evaluation with the intended clinical use

REFERENCES

Authoritative starting points

This module provides educational guidance, not a product-specific regulatory determination. Confirm the legislation, guidance and submission expectations applicable to each intended market.

KEY TAKEAWAY

Engineer the AI system, not just the model

A credible medical AI product connects clinical intent, controlled data, reproducible development, safe integration, independent evaluation and lifecycle monitoring.

Continue through the MedTechLearning AI-enabled medical-device pathway to connect this topic with the wider lifecycle.