A Mysterium VPN scan published September 10, 2026 found 36,769 self-hosted AI endpoints reachable from the public internet. Researchers identified those endpoints across model servers, workflow tools, and vector stores - and only 741 of them, or 2.02%, returned an HTTP authentication challenge. Teams self-host specifically to keep sensitive data away from third-party APIs. Many of them are doing the opposite.
This is not a story about a bad actor exploiting a rare misconfiguration. It is a story about a default.
Why self-hosted AI keeps leaking despite good intentions
The pitch for self-hosting is straightforward: if the model runs on your hardware, your customer data never touches OpenAI's or Anthropic's infrastructure. Everything runs on infrastructure you control, so customer data never touches a third-party API - privacy is not a promise in a contract, but a guarantee you get only when the data physically never leaves your network. That logic is correct, but it assumes you can actually enforce the boundary.
The problem is that the default settings of the most popular local inference tools fight against you.
Ollama ships with no authentication of its own, and its defaults expose the API to every network interface - anything that reaches the port can use the model.
The specific failure mode is Ollama's Docker documentation, which shows
the port-publish shorthand -p 11434:11434, which binds the API to every network interface. Rootful Docker then bypasses the ufw firewall rules that operators believe are protecting them - servers turn up exposed this way even when no setting their owners assume is responsible was ever touched.
Around 175,000 Ollama servers were publicly reachable on the open internet in January 2026, across more than 130 countries, nearly all of them without their owners' knowledge. The Mysterium VPN scan in September found a tighter but still alarming number across a broader category of AI services - model servers, vector stores, agent platforms together. Two scans, eight months apart, same structural finding.
Active scanners know this.
GET /v1/models is the OpenAI-compatible model-listing endpoint that dozens of self-hosted inference servers expose. If it answers without authentication, the host is running a model that anyone can query - free compute for the attacker and a potential pivot point.
Ollama's /api/tags endpoint lists locally installed models; Ollama binds to localhost by default but is very commonly exposed to the network by accident. A response there is a strong signal of an unauthenticated local LLM.
SANS ISC documented both patterns being hit repeatedly in scan traffic, probes cast at every host, not targeted attacks.
What the attack surface actually looks like once you self-host
Self-hosting an open-weight model does not remove risk - it transfers ownership of that risk to you.
With a managed AI service, the model infrastructure sits behind the provider's security boundary. Once the model runs locally, the enterprise owns the model artifact, the inference runtime, authentication, network exposure, patching, guardrails, tools, credentials, and every agent connected to it.
That list is longer than most teams realise when they spin up Ollama on a dev server to avoid sending documents to an external API. And the failure mode compounds when agents enter the picture. Hosting an AI model internally does not, by itself, prevent unauthorized access, rogue agent actions, or excessive exposure of sensitive data. Organizations still need to govern which information each user or agent can retrieve, which sensitive details it can see, and which actions it can perform.
SecuPi announced a data security platform for self-hosted AI environments on September 15, 2026, five days after the Mysterium VPN scan published. The platform combines runtime attribute-based access control, real-time monitoring of sensitive-data activity, a kill switch to block malicious agent activity, and field-level protection through format-preserving encryption, tokenization, and dynamic masking. The fact that a commercial control layer built specifically for self-hosted inference exists and is landing Forrester recognition signals that the gap is real, not hypothetical.
The comparison table below shows what you own versus what a managed provider owns across the dimensions that matter for a privacy-sensitive team:
| Dimension | Managed API | Self-hosted (well-configured) | Self-hosted (default config) |
|---|---|---|---|
| Data leaves your network | Yes, encrypted in transit | No | No - but port may be public |
| Auth responsibility | Provider | You | You (and you probably skipped it) |
| Patch cadence | Provider | You | You (often never) |
| Network exposure | Provider controls | Your firewall | Docker may bypass your firewall |
| Agent action guardrails | Provider + your prompts | You build them | None |
| Audit log | Provider dashboard | You build it | Nothing |
The right-hand column is where most quick self-hosting setups actually land.
The honest case for self-hosting, and when it still holds
For any team handling regulated or sensitive data, self-hosted AI automation is no longer the cautious, lower-quality choice it was in 2023. Local models are good enough for the extraction, classification, and routing that make up most privacy-sensitive automation, and the stack to run them is mature. The capability argument for self-hosting is now solid. The operational argument still has conditions.
Self-hosting options do not address the operational realities; they need dedicated IT resources to handle model deployment, updates, and performance tuning. For a team with two engineers who already run infrastructure, a locked-down Ollama instance behind a VPN and an authenticated reverse proxy is genuinely safer than sending documents to a cloud API. For a team that copies a Docker command from a README and moves on, it is the opposite.
The minimum viable safe self-hosting checklist:
Bind Ollama to
127.0.0.1, not0.0.0.0- addOLLAMA_HOST=127.0.0.1before the first runPut a reverse proxy (Nginx, Caddy) with HTTP basic auth or mTLS in front of any port that leaves localhost
Test your own exposure:
curl http://<your-external-ip>:11434/api/tags- if it responds, you are in the scan resultsRun
GET /v1/modelsagainst your external IP for any OpenAI-compatible inference server you operatePin your Ollama version and subscribe to its release feed; CVE-2024-37032 (Probllama) affected all versions before 0.1.34 and was fixed in Ollama 0.1.34.
Keep AI service processes out of the security team's blind spot - the risk is that powerful AI is being downloaded and operated as software infrastructure, usually outside the visibility of the security team.
An AI teammate embedded in your Slack workspace - a tool like Beagle, which runs on managed infrastructure with access-scoped to the channels it is invited to - sidesteps this class of problem entirely for most team-knowledge and document-lookup workloads. The privacy trade-off there is different: you trust the provider's data handling. The point is to make the trade-off consciously, not to assume that "local = safe."
Beagle in action#legal, 3:22pmSelf-hosted AI security: common questions
Does self-hosting AI guarantee my data stays private?
No. Self-hosting keeps data off the provider's servers, but only if your network configuration actually enforces the boundary. A misconfigured Ollama instance bound to a public interface exposes your inference endpoint - and whatever documents you send to it - to anyone scanning the internet. Privacy is a property of your configuration, not of running locally.
Is Ollama safe to run for a team handling sensitive data?
Ollama is safe if configured correctly. It ships with no built-in authentication and its default Docker examples bind to all network interfaces. To run it safely: bind to localhost, put an authenticated reverse proxy in front of any external-facing port, and confirm your external IP does not respond to GET /api/tags. Check your version against known CVEs before deploying.
What is the difference between a managed AI API and self-hosted AI for data privacy?
With a managed API, data leaves your network encrypted in transit and is processed by the provider. With a well-configured self-hosted model, data stays on your infrastructure entirely. The gap is in who owns the security controls: a managed provider handles auth, patching, and network isolation; a self-hosted deployment puts all of that on your team. If your team skips any of those steps, a managed API with strong contractual data handling terms may be more private in practice.
What should I check if I already have a self-hosted AI endpoint running?
Run curl http://<external-ip>:11434/api/tags and curl http://<external-ip>:8000/v1/models from outside your network. If either returns data, your endpoint is reachable without authentication. Put it behind a VPN or authenticated proxy immediately, update to the latest Ollama version, and audit which documents or database connections the model can reach from its current network position.