Is our AI feature a medical device?

It depends on its intended purpose. Software intended to support diagnosis or treatment decisions can fall under medical-device regulation. Get a regulatory assessment before building such features.

Can we use hosted LLMs with health data?

Possibly, with strict safeguards — but health data is special-category data under GDPR. Many teams start with use cases that don’t require patient data at all.

Industry · AI by industry

AI in healthcare software: where it helps, where it’s risky.

Healthcare has enormous administrative load and strict rules for anything that touches diagnosis or treatment. The most valuable — and safest — AI work today reduces paperwork and helps staff find information, while clinical decisions stay with clinicians.

By Nythrex EngineeringUpdated 24 September 2026 2 min read

Use cases

Administrative document processing

Referrals, insurance forms and invoices into structured data for staff review.

Documentation drafting

Draft visit summaries or letters from structured notes, reviewed and signed by clinicians.

Patient communication

Appointment reminders, preparation instructions and FAQ answers from approved content.

Internal knowledge search

Protocols, procedures and administrative rules with cited answers for staff.

Scheduling assistance

Help staff find slots and manage changes across calendars and resources.

Coding support

Suggest administrative codes for review, never auto-submitted.

Flagship use case: staff knowledge assistant

Staff often search through protocols, internal rules and administrative procedures. A retrieval-based assistant with exact citations, limited to approved internal content, saves time without making clinical decisions.

Staff interface

Intranet
Teams
EHR sidebar (where possible)

Assistant

Approved content only
Exact citations
“I don’t know” path

Controls

No patient data by default
Access by role
Audit logs

Content

Protocols
Administrative rules
Procedures

Riskier than it looks

Use case	Why it’s risky	How to handle it
Symptom checkers and triage for patients	May qualify as a medical device; safety risk	Regulatory assessment first; clinical validation; conservative scope
Diagnostic suggestions	Medical-device regulation and liability	Only with proper regulatory pathway and clinical partners
Sending patient records to external models	Special-category data under GDPR	Minimisation, regional processing, DPAs, DPIA — or self-hosted models

A sensible first project

1
Choose administrative work
High-volume paperwork or internal knowledge search.
2
Map data flows early
Involve your DPO; decide what data may be processed and where.
3
Proof of concept on approved content
Measure accuracy and citation quality with staff.
4
Roll out with training
Clear guidance on what the tool is — and isn’t — for.

Frequently asked questions

GuideCustomer data in LLMs: GDPR guidePersonal data and LLM APIs: lawful basis, DPAs, transfers, data residency, retention and minimisation — a practical checklist for product teams.GuideEU AI Act for product teamsEU AI Act for teams building on LLMs: risk categories, provider vs deployer, chatbot transparency, high-risk areas and dates after the 2026 AI Omnibus.ServiceRAG developmentRAG development: AI assistants that answer from your documents and data with citations, permissions and measured accuracy — built in your cloud.GuideDocument AI: extract data from anythingHow modern document AI works: OCR plus LLM extraction, business-rule validation, human review for exceptions and integration with your ERP or CRM.

Want a second opinion on your project?

Tell us what you’re building and where you’re stuck. We’ll reply within one business day with the most practical next step — even if that step isn’t us.

Start a project