Skip to content

ArticlesAnalysis

Can You Run HIPAA And GDPR Compliant AI Without Sending Data To The Cloud?

A jurisdiction-by-jurisdiction crosswalk of what HIPAA, GDPR and Australian Privacy Act APP 11 require of AI inference, what self-hosting genuinely removes from the risk register, what it leaves with you, and the six controls that produce auditor-ready evidence.

A navy hero thumbnail showing the number zero beside the words third-country data transfers, the claim the post tests against HIPAA, GDPR and APP 11.
On this page
  1. Key takeaways
  2. What Do HIPAA, GDPR And APP 11 Actually Require?
  3. What Does Self-Hosting Genuinely Remove From The Risk Register?
  4. What Does Self-Hosting Not Fix?
  5. What Is The Six-Control Proof Stack?
  6. How Do You Evidence This To An Auditor Or A Customer Security Questionnaire?
  7. Where Do Local Models Fit On DGX Spark?
  8. What Is The Flowtivity Play?
  9. Frequently Asked Questions
  10. Do HIPAA or GDPR actually require self-hosting?
  11. Is self-hosted AI automatically compliant?
  12. Do we still need a Business Associate Agreement on our own hardware?
  13. What about GDPR transfers if our office is in Australia?
  14. How long does the evidence pack take?
  15. Takeaways

Last Updated: 19 November 2026

Key takeaways

  • ▪ Yes, you can run HIPAA and GDPR compliant AI without sending data to a model vendor, but self-hosting removes only part of the obligation.
  • ▪ HIPAA asks for five technical safeguards under 45 CFR 164.312, plus a signed Business Associate Agreement with anyone who touches PHI.
  • ▪ GDPR asks for a lawful basis, an Article 28 processor contract, Article 32 security measures and a lawful transfer mechanism under Article 44.
  • ▪ Australian APP 11 requires reasonable technical and organisational steps, and APP 8 keeps you accountable for what an overseas recipient does.
  • ▪ Self-hosting does not fix application logs, backups, support access, telemetry, endpoint devices, staff access or model licence terms.
  • ▪ What an auditor wants is not a diagram. It is a control matrix with dated evidence behind every row.

Yes, you can run compliant AI inference without shipping data to a model vendor, and for many Australian businesses it is now the cheaper and cleaner path. The trigger this week was HackerNoon publishing its complete guide to HIPAA and GDPR compliant inference on 17 November 2026, a clear signal that compliance-driven buyers are hunting for on-premises answers. The number that should stop you is 92 percent, the share of 32-token inputs that the Vec2Text research reportedly recovered exactly from embeddings, including names lifted from clinical notes. Expect four to eight weeks of engineering to stand up the controls, plus a permanent load to keep the evidence current. Self-hosting is not a certificate. It is a shorter supply chain that you then have to prove you govern.

What Do HIPAA, GDPR And APP 11 Actually Require?

Three regimes, three shapes. HIPAA is prescriptive about technical safeguards and contracts. GDPR is principle-based, with explicit articles on processors, security and transfers. The Australian Privacy Act sits on a reasonableness standard that scales with data sensitivity and entity size. None of them require data to stay in your building. All of them require you to describe, operate and evidence the controls that protect it. That is why "the data never leaves" answers only one clause of a much longer questionnaire.

HIPAA is the easiest to map because it names the controls. 45 CFR 164.312 sets five technical safeguard standards: access control, audit controls, integrity, person or entity authentication, and transmission security. Access control requires technical policies so only people or software with granted rights reach electronic protected health information. Audit controls require mechanisms that record and examine activity in systems holding ePHI. Integrity stops improper alteration or destruction. Authentication verifies that whoever is asking is who they claim to be. Transmission security guards data in motion.

Then the contract question. HHS is blunt: the Rules apply to covered entities and business associates, and a covered entity engaging a business associate must have a written contract requiring compliance. Business associates are also directly liable for certain provisions. So the moment a model vendor, a vector database host or a support engineer can reach PHI, you need a Business Associate Agreement. Self-hosting inside your own cloud account can remove a vendor from that list entirely, which is the biggest administrative win available.

GDPR is structured differently. Article 28(1) lets a controller use only processors that provide sufficient guarantees for appropriate technical and organisational measures. Article 28(2) blocks sub-processors without prior written authorisation, the clause that causes most grief in AI procurement. Article 28(3)(h) requires the processor to give you what you need to demonstrate compliance and to allow audits, including inspections. Article 32(1) requires appropriate security, naming pseudonymisation and encryption, judged against state of the art, cost and risk. Article 44 restricts third-country transfers unless Chapter V conditions are met, including onward transfers. Article 9(1) prohibits processing health data unless a Article 9(2) condition applies.

Australia adds another flavour. APP 11.1 requires an APP entity holding personal information to take reasonable steps to protect it from misuse, interference and loss, and from unauthorised access, modification or disclosure. APP 11.2 requires reasonable steps to destroy or de-identify information no longer needed, subject to record-keeping and legal-retention exceptions. The OAIC is explicit that reasonable steps include technical and organisational measures. APP 8.1 then requires reasonable steps to ensure an overseas recipient does not breach the APPs, and s 16C keeps you accountable for what that recipient actually does.

stat-grid infographic: the three compliance regimes side by side, HIPAA 45 CFR 164.312 technical safeguards, GDPR Article 32 security of processing, and Australian Privacy Act APP 11 security of personal information
At a glance: three regimes, three shapes of obligation, one shared demand for evidence.

What Does Self-Hosting Genuinely Remove From The Risk Register?

Self-hosting removes four things that are hard to paper over. It removes the model vendor from the processor chain, so there is no third party sub-processor to authorise, audit or chase for a data processing agreement. It removes the third-country transfer question for inference itself, because prompts and retrieved context never cross a border to reach a model. It removes the vendor's retention schedule, so a 30-day default stops being a policy conflict. And it removes version drift, because you pin the weights and keep evaluation results per version.

The processor chain point deserves emphasis. Under GDPR, every additional processor is another Article 28 contract, another set of guarantees to assess, and another party that must seek your authorisation before engaging its own sub-processors. Under HIPAA, every additional party touching PHI is another Business Associate Agreement and another name on the breach notification path. A cloud inference API typically adds the model provider, its infrastructure provider and often a logging vendor. Self-hosting on infrastructure you already control collapses that chain to you and your existing cloud provider. For the wider argument about jurisdiction rather than control, see data residency versus AI sovereignty in Australia.

before-after infographic: cloud inference processor chain versus self-hosted inference chain, showing the contracts and third-country transfers that disappear when the model runs on your own infrastructure
How it works: the same prompt, two supply chains, and the contracts that vanish when inference moves in-house.

What Does Self-Hosting Not Fix?

Plenty. Self-hosting changes where inference happens. It does not change how your application behaves around inference. Application logs still capture request bodies when debug logging is left on, and those logs often ship to a third-party observability tool. Backups still copy the vector index and prompt store to object storage, often in another region with a different retention policy. Support access still exists, because someone has to patch the GPU node. Telemetry still phones home unless you turn it off. Endpoints still hold the browser session. Staff still have accounts. And the model licence still binds you wherever the weights run.

The licence point catches people out. Meta's Llama licence, for example, requires redistributors to provide a copy of the agreement and prominently display "Built with Llama", and it disclaims all warranties while making the licensee solely responsible for judging whether the use is appropriate. Those obligations survive self-hosting because they attach to the weights and outputs, not the hosting location. A review that stops at network diagrams will miss this entirely. The six leak paths that survive an air gap are covered in what local AI actually means when it is offline.

Staff access is the quietest gap. HIPAA's access control and audit control standards both assume you know who has access and can show what they did. If three engineers share a root login to the inference node, you have neither. The same applies to GDPR's integrity and confidentiality principle and to APP 11's reasonable-steps test. Self-hosting concentrates risk into your own identity plane, which beats spreading it across vendors, but only if that plane is governed.

What Is The Six-Control Proof Stack?

The proof stack is the minimum set of controls that answers a HIPAA safeguard review, a GDPR Article 32 assessment and an APP 11 reasonable-steps test with one evidence set. Network isolation keeps inference endpoints off the public internet. Identity and least privilege give every agent and human a named account with minimum scope. Audit logging records who called which model, when, and under what authorisation. Metadata-only request logging keeps prompt bodies out of logs while preserving operational visibility. Retention and deletion rules cover vectors, caches, backups and fine-tuning data. Model supply chain control pins versions, records licence review and keeps evaluation results per release.

Each control maps to a specific obligation, which is what makes the stack defensible. Network isolation and authentication satisfy HIPAA 164.312(a) and (d), and GDPR Article 32(1)(b) on confidentiality. Audit logging satisfies 164.312(b) and supports Article 32 demonstrability. Metadata-only logging supports data minimisation and shrinks the blast radius of a log breach. Retention and deletion satisfies APP 11.2 and GDPR Article 5(1)(e). Supply chain control answers the Article 28 sub-processor question and the APP 8 accountability question at once. If you are building agents rather than a single model endpoint, the on-premises patterns are set out in self-hosted AI agents on premises.

steps-and-tiles infographic: the six-control proof stack for compliant self-hosted AI inference, from network isolation through identity, audit logging, metadata-only logs, retention and deletion, to model supply chain control
How it works: six controls, each mapped to a named obligation, each producing an artefact an auditor can inspect.

How Do You Evidence This To An Auditor Or A Customer Security Questionnaire?

You evidence it with artefacts, not adjectives. Build a control matrix with one row per obligation, one column for the control that satisfies it, and one column for the evidence proving the control operates. Then collect that evidence on a schedule. A network diagram is not evidence. A firewall export showing no public route to the inference subnet, dated this month, is evidence. A policy document is not evidence. A sample of audit logs showing named users and model calls, with retention configured, is evidence.

The artefacts that carry the most weight are consistent across all three regimes. A signed Business Associate Agreement or Article 28 data processing agreement with your infrastructure provider. A security risk analysis for HIPAA and a DPIA for GDPR where high risk applies. A sub-processor register listing every party in the chain and the contract covering them. A retention schedule naming each data store, including vector indexes, caches and backups, with a deletion mechanism and a test result. A model register with pinned versions, licence notes and evaluation results. And an access review showing who holds privileged access and what was removed.

Customer questionnaires are where this pays off commercially. Most enterprise buyers now ask the same twenty questions about AI vendors, and the ones that sink deals concern sub-processors, training data retention, cross-border transfer and audit rights. Answer those four with documents rather than assurances and you clear procurement faster than a competitor on a hosted API.

Where Do Local Models Fit On DGX Spark?

Two DGX Spark boxes running DeepSeek V4 Flash locally is a realistic inference tier for a mid-sized business, not a toy. The Spark form factor puts a meaningful amount of unified memory and GPU compute on a desk-sized footprint, so you can hold weights and a working context window on hardware you physically control. DeepSeek V4 Flash suits agent workloads because its latency profile favours multi-step tool calling over one-shot chat. The trade-off is a throughput ceiling, so you route heavy reasoning steps to a frontier API under a BAA and keep sensitive, high-volume steps local.

The economics matter as much as the compliance story. Self-hosting shifts spend from per-token billing to capital and power, which moves the break-even point. If your monthly hosted bill is steady and high, the local path pays back faster than finance teams expect. If volume is low or spiky, a hosted API with a BAA stays the sensible default, and the HackerNoon guide cites Superlinked's analysis putting that crossover at roughly USD 2,000 to 3,000 a month. That figure is vendor-reported, so model it against your own numbers. The full cost model is in local LLM inference economics.

What Is The Flowtivity Play?

Flowtivity builds this stack for Australian businesses that need compliant AI without a vendor in the middle. We run dual DGX Sparks with DeepSeek V4 Flash locally, so the architecture we recommend is the architecture we operate. Engagements are staged, because compliance work fails when it starts with a policy document instead of a working system. Every engagement produces artefacts your auditor or a customer's procurement team can inspect, and your counsel signs off before anything becomes policy.

1. Compliance control mapping. A fixed-scope review mapping your AI workflows against HIPAA 164.312, GDPR Articles 28, 32, 44 and 9, and APP 11 and APP 8. You get a control matrix with obligations, controls and the evidence you are missing.

2. Self-hosted inference build. A private inference tier on your infrastructure or ours, with network isolation, scoped service identities, audit logging and metadata-only request logs. Versions are pinned, evaluated and licence-reviewed, and you get the evidence pack with the endpoint.

3. Agent governance and observability. For teams already running agents, we instrument tool calls, retrieval steps and approval points so every action is attributable to a named identity. That closes the staff-access and audit-trail gaps self-hosting leaves open.

4. Auditor and procurement readiness. We prepare the artefact set, run a mock security questionnaire and sit with your team through the first real audit or enterprise review. The output is a dated evidence pack, not a slide deck.

Frequently Asked Questions

Do HIPAA or GDPR actually require self-hosting?

No. Neither regime mandates self-hosting. HIPAA requires safeguards and Business Associate Agreements. GDPR requires a lawful basis, processor contracts, security measures and lawful transfers. Self-hosting is one way to satisfy those obligations with a shorter supply chain, and a hosted API with the right contract can also be compliant.

Is self-hosted AI automatically compliant?

No. Self-hosting removes the model vendor from the processor chain, which is a real risk reduction. It does not fix application logs, backups, support access, telemetry, endpoints, staff access or model licence obligations. You still need the six controls and evidence that they operate.

Do we still need a Business Associate Agreement on our own hardware?

You need one with every party that can reach protected health information. If the hardware sits in your facility and only your staff have access, the model vendor drops out of scope. Your cloud provider, backup provider and any support engineer with remote access may still need a BAA.

What about GDPR transfers if our office is in Australia?

Article 44 restricts transfers of EU personal data to third countries, including onward transfers. Running inference locally in Australia does not solve that on its own, because the data still left the EU to reach you. You need a Chapter V transfer mechanism for the inbound flow, and APP 8 accountability applies on the Australian side.

How long does the evidence pack take?

For a mid-sized business with two or three AI workflows, four to eight weeks is realistic if the infrastructure exists. The long pole is usually retention and deletion testing across vector stores and backups, not model deployment. Starting with the control matrix shortens it, because you stop collecting evidence you do not need.

Takeaways

  • Self-hosting is a legitimate compliance strategy, but it answers the transfer and processor-chain questions, not the whole questionnaire.
  • HIPAA, GDPR and APP 11 converge on the same demand: named controls, operating evidence and a contract with every party that can reach the data.
  • The six-control proof stack is the minimum defensible set, and each control should map to a specific obligation in your matrix.
  • Counsel signs off on the legal position. Engineering's job is to make the controls real and the evidence current.

This is an engineering and architecture view, not legal advice. Your counsel signs off on the compliance position before anything is relied on.

  • HIPAA
  • GDPR
  • AI compliance
  • self-hosted AI
  • Australian Privacy Act
  • on-prem AI

What's actually useful in AI, every Friday

We follow everything happening in AI and send what's worth your time: new models, AI agents, and how to put them to work. One email a week, unsubscribe anytime.

One good place to start

What would you like to take off your plate?

Bring a process that feels repetitive or harder than it needs to be. We’ll help you find a practical first step.

Book a free consult

A free 1-hour conversation with AJ. No pressure, no pitch.