Does Your Local AI Really Stay Offline, And How Would You Prove It?

Last Updated: 31 October 2026

No, local does not automatically mean private, and the way to prove it is to watch the network rather than read the settings page. Local means the model weights run on hardware you control. It says nothing about telemetry, update checks, sync features, model downloads or the tools your agent calls. Those are separate decisions, and each one is a place where data can leave.

This week two explainers landed on the same point from different directions. Notebookcheck published a guide on making Ollama and LM Studio actually stay offline, and the practical guidance is blunt: with sync, online tools, master telemetry and AI observability including replay turned off, prompts and responses stay on your machine. A Mac-focused open-source agent called Local Code shipped as another local-first option, and Ability AI is marketing self-hosted business agent operations. The tooling is maturing. The proof is still the buyer's job.

This post covers what actually phones home, how to switch it off, how to prove it with evidence, why the agent layer is now the bigger risk, and what an air-gapped setup looks like on our own hardware.

What does "local" actually guarantee?

It guarantees that inference happens on your hardware, which is a real and valuable property. It does not guarantee that the request never leaves the machine, because the endpoint your application talks to is just an address. A localhost URL can still be proxied to a cloud model, and a runtime can be configured to fall back to a hosted provider when the local model is unavailable or when a feature needs it.

So the first check is not "is the URL localhost" but "which model is loaded, in which runtime, and what is the fallback". If the answer includes any hosted provider, the stack is hybrid, and the privacy claim needs to be written as hybrid rather than local. That is not a criticism. Hybrid is often the right design. It is a criticism of describing it as something it is not.

What actually phones home?

Six things, roughly in order of how often they surprise people. Telemetry and usage analytics. Sync and cloud workspace features, which are on by default in some desktop tools. Update checks, which are small but reveal that the machine exists and is running. Model downloads, which are large and obvious but still outbound. Online tools and web search integrations, which send queries out. And retrieval connectors, which can push document content to an external embedding or search service.

The guide's list maps to the first three: sync, online tools, master telemetry and observability including replay. Turning those off is necessary and not sufficient. The remaining three are configuration decisions inside your own application, and they are where most of the risk sits.

There is a seventh path that gets less attention: the agent's tool calls. If your agent can send email, query a CRM or call a payment API, then the data leaves through the tool, not through the model. The model can be perfectly local while the workflow posts a customer record to a SaaS endpoint. That is the leak that matters for most business deployments.

How do you switch it off?

Start with the runtime. In LM Studio, turn off sync, online tools, telemetry and observability including replay, and if you expose the local server, enable its authentication token so other processes on the machine cannot call it anonymously. In Ollama, the server is keyless by default on port 11434, so bind it to localhost only and put a reverse proxy in front if more than one application needs it. For any other server, vLLM, llama.cpp or a proxy, check that the base URL points at your own host and that no fallback provider is configured.

Then harden the host. Default-deny outbound traffic at the firewall, allow only the destinations the system genuinely needs, and log every denied attempt. If the machine needs nothing, give it nothing. A workstation that runs models and nothing else should not be able to reach the internet at all.

Then handle model provenance. Download weights once, verify the hash, store them locally and serve from disk. A stack that re-downloads on every start is not offline, even if it never sends your prompts anywhere.

How do you prove it?

With evidence, collected in this order. First, write down the intended egress: the list of destinations the system is allowed to reach. Second, enforce it with a default-deny rule. Third, log DNS queries and firewall decisions for a week. Fourth, capture packets during a representative workflow and read the destination addresses. Fifth, pull the network cable and run the workflow again.

The cable test is the one that convinces a client. If the workflow completes with no network, inference is local. If it fails, you have found a dependency you did not know about, which is exactly the information you need. Run the cable test on the model path and the tool path separately, because they fail for different reasons.

Record the result as a short statement: what the system can reach, what it did reach during the test, and how you verified it. That document is worth more than a vendor's privacy page, because it describes your configuration rather than someone else's policy.

What does air-gapped look like on our own hardware?

We run two DGX Sparks with DeepSeek V4 Flash locally, and the useful discipline is to treat the pair as an appliance rather than a workstation. The model server binds to the internal network only. The agent runtime talks to it over that internal link. Tool credentials live in a gateway, not in the agent's context. Outbound access from the model host is denied by default, and the only allowed destinations are the ones required for patching, on a schedule.

The measured result is 35 tokens per second on a single stream and 62 across four concurrent streams, which is enough for drafting, extraction and summarisation work. That number matters for this post because it sets the ceiling on what you can do offline. If the local stack cannot handle the workload, teams route around it, and routing around it is how data leaves.

The honest limit is capability, not connectivity. Vision-heavy tasks and the hardest reasoning still favour larger hosted models, so most real deployments end up hybrid by design. The point is to make the hybrid explicit: which tasks stay home, which leave, and what evidence you have for each.

What does this mean for Australian businesses?

Data sovereignty questions arrive in procurement, in client due diligence and in board risk reporting. "We use a local model" is not an answer to any of them, because the question behind the question is "where did our data go". The answer that works is a configuration statement with evidence attached.

Under the Privacy Act 1988, APP 8 governs cross-border disclosure of personal information. If a stack sends prompts or documents to an overseas service, that is a disclosure and it needs to be assessed, disclosed and contracted for. A genuinely offline path removes that analysis for the tasks it covers, which is a meaningful simplification for regulated clients.

It also changes the sales conversation. When a prospect asks whether their documents leave the building, the answer becomes a demonstration rather than an assurance. That is a different kind of trust, and it closes deals that a policy page does not.

If you are weighing the on-prem stack against a hosted one, our post on running agents fully on-premises covers the cost and capability trade-offs. If you want the egress test run against your own environment, talk to us.

Frequently asked questions

Does running a model locally mean my data stays private?

Not automatically. Local means inference happens on your hardware. Telemetry, sync features, update checks, model downloads, online tools and agent tool calls can still send data out. Privacy requires configuration plus evidence that the configuration holds.

How do I check whether Ollama or LM Studio is sending data?

Log DNS queries and firewall decisions, capture packets during a representative workflow, and read the destinations. Then disconnect the network and run the workflow again. If it completes offline, inference is local. If it fails, you have found a dependency.

What settings should I turn off first?

Sync, online tools, telemetry and observability including replay. Then bind the local server to localhost only, enable authentication if the runtime supports it, and default-deny outbound traffic at the firewall.

Can an agent be local while its tools are not?

Yes, and that is the most common real-world setup. The model runs locally while a tool call posts data to a CRM, email provider or payment API. The leak is in the tool layer, so audit the tools, not just the model.

Is a fully air-gapped stack practical for a business?

For drafting, extraction and summarisation, yes. For vision-heavy work and the hardest reasoning, capability is the constraint, so most deployments end up hybrid. The important thing is to make the hybrid explicit and document which tasks stay home.