AI at Accurx

Every AI feature we build in our medical device products goes through a rigorous safety process before it reaches a single patient. This is:

• Clinical ownership and sign-off. Our in-house Clinical Team owns and signs off the safety criteria for every AI feature, using recognised clinical risk standards (ISO 14971 and the NHS clinical risk management standard for health IT, DCB0129). As part of this, a cross-functional team spanning clinical, engineering, security, legal and other disciplines maintains a hazard log: a systematic record of every clinical risk we've identified, how serious it is, and the steps we've taken to reduce it. This is kept up to date based on real usage.

• Testing to statistical confidence. We test on large samples of real-world, anonymised patient data where possible, using sample sizes large enough that we can state our confidence with a defined margin - for example, 95% confidence that performance meets or exceeds an agreed clinical threshold.

• Independent clinical review.  Clinicians evaluate the AI's outputs against our safety criteria and are explicitly asked to flag anything that concerns them - including things we might not have thought of ourselves.

• Post-market surveillance. Once a feature is live, we monitor real-world performance to detect issues early, feed learnings back into our hazard log and safety controls, and keep improving over time.

How our AI is currently used is to generate questions or information to support processes like triage - it does not make care decisions on its own. AI can make mistakes, and here’s how we look to reduce them:

• Live guardrails. We build safety checks into our products that review AI outputs. When appropriate, these double-check the output and are designed to block anything that doesn't meet our standards before it reaches a patient or user.

• Graceful fallback. Features are designed to fail safely. For example, if AI outputs take too long to load or hit an error, they're simply skipped so the patient can still use our platform without disruption.

• Incident management. If an issue is ever identified, we have an established incident process involving the clinical team to assess the potential impact on users and patients, and provide updates to practices, where appropriate.

Our AI features are designed to support your team, not replace their clinical judgement. Accountability for clinical decisions stays with your clinicians in the following ways:

• Human-in-the-loop by design. A healthcare professional stays in control, reviewing and confirming AI-generated outputs before they inform patient care.

• AI supports, it doesn't decide. For example, Smart Triage Questions generates follow-up questions to help gather more information. It is not intended to diagnose, recommend treatment, or make the triage decision.

• Transparent about limitations. We are open about the fact that AI can make mistakes - that's exactly why the human review step exists.

Hallucination - where an AI model generates plausible-sounding but incorrect output - is a known risk with the type of AI models we use (large language models). We don't claim it can't happen. Instead, here’s how we focus on reducing the risk:

• Rigorous pre-launch testing. On anonymised real-world data, where available

• Live guardrails. These are built into our products that check for harmful or unexpected output, before it's shown to anyone

• A clinician in the loop. Always acting as the final decision maker

• Product-specific checks. For example, for our Smart Conversation Topics, we purposefully test for hallucinations and ensure results meet our clinical safety thresholds. For our triage questions, we check that AI-generated questions are relevant to the patient's problem and haven't suggested a diagnosis or treatment.

No. Patient personal data is not used to train the underlying models. We guarantee this through:

• Rigorous procurement. Our infrastructure goes through a strict procurement process specifically to meet our security and data privacy requirements.

• Controlled environment. The model is hosted within Accurx's controlled environment and only receives the specific data needed to respond to each request - restricting it to only the data it needs.

We protect patient data using robust infrastructure and continuous testing by independent security experts. We do this by:

• Red teaming. Our security team deliberately tries to make the system misbehave, so we can find and close gaps before any real-world use.

• Penetration testing. Independent security specialists attempt real-world attacks before anything goes live, so any weakness is found and fixed before it could be exploited.

• Controlled access. The model runs inside a controlled software layer that ensures it can only ever access the data it's supposed to, and nothing more.

• Secure development lifecycle. All code passes strict static and dynamic security checks.

Patient data only flows through trusted, vetted infrastructure that's been through our security procurement process. We guarantee this through:

• Controlled hosting. The model is hosted within Accurx's controlled environment and only receives the specific data needed to respond to each request, which limits exactly what it can access.

• DPIA template provided. We provide a Data Protection Impact Assessment template so your team can review and sign off the specifics for your own governance process.