Data Science and AI / ML Engineers in Medical-device Development

The data-science and AI / ML engineering role

The role is to create and sustain an algorithm whose behaviour is appropriate for a defined medical purpose. This includes data definition, provenance, preparation, feature engineering, model development, evaluation, implementation and monitoring. A strong metric in a notebook is not enough; the released system, users, workflow and consequences determine whether performance is acceptable.

Start with the intended decision, not the available dataset

Define the target condition or quantity, intended population, input source, operator, use environment, output, clinical interpretation and action. State whether the function screens, detects, diagnoses, predicts, recommends, controls or merely supports workflow. Identify foreseeable misuse and the consequence of false, missing, delayed or overconfident outputs.

Keep the algorithm boundary aligned with MTL-102 — Intended Purpose, Users and Use Environments.

Govern data as a design input

• Record source, collection context, consent or legal basis and permitted purpose.
• Define inclusion, exclusion, labelling and adjudication methods.
• Represent relevant sites, devices, users, demographics, disease states and confounders.
• Control missingness, duplicates, outliers, transformations and synthetic data.
• Separate training, tuning and independent evaluation without leakage.
• Version datasets, code, labels, features, model parameters and environments.
• Protect privacy, security and access throughout the pipeline.

Use MTL-112 — Privacy and Data Protection by Design for product-level data protection.

Make model development reproducible and reviewable

Document baselines, architecture, features, optimisation, hyperparameters, stopping rules and selection criteria. Retain experiments sufficiently to explain why the chosen approach was selected. Control software libraries, compute environment and random seeds where they affect results. Evaluate robustness to plausible input variation and fail safely when inputs are invalid or outside the supported domain.

Evaluate the claim, not a convenient average

Report confidence intervals, calibration and error distributions where relevant. Examine performance at sites, devices and populations not used for model development. Use MTL-120 — Statistical Methods and Measurement Assurance.

Look for bias across the complete lifecycle

Bias can enter through the clinical question, access to care, sampling, measurement, labels, missing data, model choice, threshold, interface and deployment. Predefine consequential subgroups and analyse whether performance differences could cause unequal harm or benefit. When data are sparse, state uncertainty and define limits rather than treating an absence of evidence as equivalence.

Engineer the algorithm as part of a medical-device system

Specify inputs, preprocessing, timing, output semantics, confidence, error handling, logging, hardware and software dependencies, cybersecurity controls and user presentation. Verify that the production implementation matches the evaluated model. Assess automation bias, alert fatigue and how users respond to uncertain or conflicting information.

Connect the model to MTL-108 — Software Lifecycle and MTL-105 — Systems Engineering, Architecture and Interfaces.

Control model and data changes

Establish the released model, code, data specification, thresholds, dependencies and performance evidence as a baseline. Assess retraining, new data, label changes, model compression, platform migration and threshold adjustment for impact on intended use, risk and regulatory status. Predefine a change protocol only where the applicable regulatory pathway supports it; do not treat continual learning as uncontrolled deployment.

Monitor clinically meaningful performance after release

Define signals for input drift, data-quality change, subgroup performance, output distribution, override, complaint and adverse outcome. Monitoring must respect privacy and distinguish algorithm failure from workflow or population change. Set thresholds, investigation ownership and actions before signals arise. Feed learning into risk, labelling, support and controlled improvement.

Common misconceptions