MTL-606: Choosing and Validating Injection-Tracking Technologies

What you will learn

Compare technologies against a defined use case and evidence requirement.

Part 2 of a three-module pathway: measurement, technology validation and evidence planning.

Start with the decision and required information

“Track injections” is not a complete use case. Specify who will use the information, what action they may take and what happens if the information is wrong, late or absent. A personal diary, a clinician outreach queue and a research endpoint have different evidence needs.

Example use case: identify an unconfirmed weekly administration so that a care team can contact the patient during working hours. Required information includes the expected regimen, event status, observation coverage and source. The workflow must allow the patient to explain a planned pause or an unrecorded dose. It should not automatically advise another injection because a record is missing.

Record the intended population, compatible injectors, medicine presentation, environments, user tasks, response deadline and boundaries. Decide whether the required evidence concerns an attempted injection, a completed mechanism cycle, administration timing or physical delivery.

Compare methods by what they can observe

This is a comparison of technology classes, not validated performance rankings or claims about particular products. Actual performance depends on sensor design, algorithm, placement, device compatibility and use conditions.

Method	Potentially useful evidence	Failure modes and validation priorities
Device sensing	Mechanism position, activation, contact, movement or a defined sequence.	Challenge attachment errors, off-body use, incomplete cycles, leakage and device variants. Mechanism completion does not automatically prove delivered dose.
Smart labels / packaging	Identity, opening, removal, scan or package state; capabilities vary.	Challenge multiple removals, opening without use, later administration and missed scans. A richer sensor label needs its own event validation.
Acoustic detection	Recognisable sounds and temporal patterns.	Noise, other injectors, phone position, microphone permissions, replay and sound changes across lots or models.
Visual observation	Visible handling, device state or injection sequence.	Occlusion, lighting, camera angle, clothing, bystanders and changes in appearance. Visible steps may not reveal internal delivery.
Contextual inference	Patterns from activity, location or other indirect data.	Confounding routines, changes in habits, missing context and population shift. Validate the inference against independent event labels.
Self-report	Patient account, including reasons and symptoms unavailable to sensors.	Recall, back-entry, accidental taps, burden and reporting incentives. Preserve report time separately from claimed event time.

Prioritise reliable, automated capture of injection-related data, reducing dependence on manual entry and patient recall. Select technologies that collect the information required for the intended use with minimal patient effort, and validate their accuracy, completeness and performance under realistic conditions. Use patient input selectively to explain exceptions, such as a clinician-directed pause, with clear rules for reconciling it with automatically recorded data.

Define performance before running the study

Requirement area	Metric or question	Planning detail
Detection	Sensitivity = true detected events / reference events.	Define event matching and a timing tolerance; count missed reference events.
False events	Positive predictive value = true events / all detected events; false alerts per patient-week.	Include realistic non-event exposure. Specificity needs a meaningful negative opportunity definition.
Timing	Error relative to reference event time; relevant tail of error distribution.	Prespecify acceptable delay, clock drift and handling of offline uploads.
Attribution	Rate of wrong-patient, wrong-device or wrong-medicine association.	Challenge pairing, reassignment, shared phones and replacement devices.
Completeness	Proportion of expected records reaching the clinical view; observation coverage.	Separate detector performance from upload success and participant retention.
Interpretability	Can intended users distinguish confirmed, inferred, reported and unknown states?	Representative-use testing and consequences of mistaken interpretation.

Set numerical limits from the decision and risk analysis, not from the best result the prototype achieved. Predefine sample size, confidence bounds, relevant subgroups and event prevalence. Repeated injections within one person are correlated; 500 injections from five people do not establish the same generalisability as a representative patient study.

Test realistic use and failure

Use conditions: dexterity, vision, language, body habitus, clothing, distractions, carers and low digital confidence.
Environment: noise, lighting, travel, storage, temperature, phone models and intermittent connectivity.
Device behaviour: priming where relevant, incomplete injection, early removal, repeat handling, training devices and disposal.
Technical faults: low battery, disabled permissions, full storage, restart, clock changes and incompatible software versions.
Burden: extra steps, charging, attachment, camera setup, reminders, discomfort and time to resolve failures.

Recruit people whose abilities and circumstances represent intended use. Retain difficult cases and explain exclusions. For learned algorithms, separate development and locked evaluation data by participant and relevant source; avoid leakage through repeated recordings of the same device or session.

Validate from physical event to clinical display

Stage	Evidence to retain	Fault to challenge
Reference event	Independent synchronised observation and adjudication rule.	Reference disagreement or uncertain ground truth.
Sensor and algorithm	Raw signal, event classification, confidence and version.	Off-body activation or a partial sequence.
Local record	Stable event ID, source timestamp and identity binding.	Power loss during storage or device reassignment.
Transfer and service	Acknowledgement, integrity, deduplication and reconciliation logs.	Lost acknowledgement, retries and out-of-order upload.
Clinical display	Correct patient, regimen, event status and freshness.	Delayed data displayed as a new injection or unknown shown as missed.
Clinical response	User interpretation, escalation and documented disposition.	False alert prompts an inappropriate additional dose.

Verification shows specified requirements are met. Validation establishes that the resulting information and workflow are suitable for intended users and decisions. An algorithm test is only part of this evidence. Use an independent reference appropriate to the claimed event; document its own accuracy and blind adjudication where feasible.

For study data collection, FDA’s December 2023 DHT guidance provides a relevant framework for fit-for-purpose selection, verification, validation and usability. Its scope is remote acquisition in clinical investigations; it is not a universal device clearance standard.

Uncertainty, security and lifecycle control

Preserve a visible unknown state. Decide how the system behaves when confidence is low or coverage is absent, including when human review is required. Explain what the system can and cannot establish at the point of use.

Protect patient-device binding, authentication, authorisation and record integrity. Test replayed events and unauthorised reassignment alongside accidental duplication. Limit data collection to the purpose: audio and video may capture other people and private surroundings. Define retention, access, deletion, audit history and recovery responsibilities for the chosen service.

Control sensor, algorithm, firmware, app, cloud and display versions as one evidence baseline. A changed injector click, phone operating system, model threshold or cloud schema may invalidate assumptions. Assess each change, repeat affected verification and validation, and monitor field performance by relevant subgroup. Monitor missingness and false alerts as well as successful uploads.

Worked example: the detector passes, the workflow fails

Illustrative study: an independent reference identifies 200 injection events. The detector matches 190 and adds 10 false events. Sensitivity and positive predictive value are each 95%. Only 180 of the 190 matched events reach the correct clinical record within the required window; timely end-to-end capture is therefore 180/200, or 90%. These point estimates are examples, not acceptance criteria.

A delayed batch upload then appears as several injections “today” because the display uses server receipt time. Even a technically accurate detector cannot support accurate timing in that workflow. Correct the timestamp semantics, show data freshness, and repeat the affected end-to-end tests. Separately assess whether the observed false-alert rate is acceptable for the proposed clinical response.

Exercise: name three additional pieces of evidence before release. Suggested answer: confidence intervals and representative users; realistic non-event exposure and patient-level false-alert burden; and usability evidence that unknown status does not trigger inappropriate repeat dosing.

Practical takeaway: comparison and validation planning

Use the matrix before choosing a supplier and again before freezing the verification plan. Record evidence gaps explicitly; do not replace them with a qualitative score that hides a critical weakness.

Candidate / claim	Reference and test population	Acceptance and decision
Device sensing / mechanism-cycle event	Independent observation of normal, incomplete and off-body cycles across compatible variants.	Risk-based sensitivity and false-event limits with confidence bounds; delivered dose remains a separate claim.
Acoustic / event timing	Synchronised reference recordings across specified phones, noise and positions.	Timing error and missed/false-event limits; reject use conditions outside supported performance.
Hybrid workflow / patient outreach	Representative service simulation with delayed, conflicting and missing data.	Correct interpretation and safe escalation; quantify burden and inappropriate alerts.

Download / open the technology comparison and validation-planning matrix.

References and related learning

Sources checked 24 October 2026. The matrices are engineering planning aids, not claims that any technology class has proven clinical superiority.