---
title: Private LLM vs. a private AI service
description: What a private LLM is, what hosting one on-premises or in your own cloud takes in hardware and staff, and when a private AI service is the simpler path.
canonical: https://privatesuperintelligence.si/blog/private-llm-vs-private-ai-service/
last-updated: 2026-10-15
---

# Private LLM vs. a private AI service

A private LLM is a large language model that runs on hardware you control, on-premises or in your own cloud account, so prompts and documents never pass through a public AI provider. Running one yourself means owning the GPUs, the serving software, security and updates, plus the retrieval, permissions and interface that turn a model into something your team uses. A private AI service gives you the same isolation with a provider building and running the system for one client.

The model is the easy part. The question for a family office, a fund or a firm is whether you want to staff everything around it.

## What a private LLM is

"Private LLM" covers three setups, from most to least control:

- **On-premises.** The model runs on servers in your office or data center. Nothing leaves the building unless you allow it. This is also the starting point for an air-gapped system.
- **Your own cloud account.** You run an open-weight model on GPU instances you rent, or deploy one through a managed service such as Amazon SageMaker JumpStart. The cloud provider operates the hardware; you control the account.
- **A dedicated managed model.** You call a hosted model inside your cloud contract (Azure-hosted OpenAI models, Amazon Bedrock). This is private from the model's maker, though the cloud provider's terms apply. Our guide to [private ChatGPT for business](/blog/private-chatgpt-for-business/) covers this route.

## What it takes to run one yourself

Open-weight models have made the first step easier. OpenAI's gpt-oss-120b is published under the Apache 2.0 license and, by OpenAI's model card, runs on a single 80 GB GPU such as an NVIDIA H100 or AMD MI300X. The smaller gpt-oss-20b fits within 16 GB of memory. You can download either and run it today.

The model alone answers questions from what it learned in training. To be useful to your team, it has to read your documents and email, respect who may see what, and show where each answer came from. That is the system you build around it:

| Component | What it involves |
|---|---|
| Hardware | GPUs sized for the model and the number of users, plus redundancy |
| Serving | Software that loads the model, handles requests and scales |
| Retrieval | Indexing your documents and email so the model can cite them |
| Permissions | Making sure a junior analyst can't retrieve the principal's trust documents |
| Interface | A chat tool, integrations with mail and files, single sign-on |
| Security | Patching, monitoring, and defenses against prompt injection |
| Evaluation | Testing answers for accuracy each time you change a model |
| Availability | Backups, failover and disaster recovery |

The security row carries more weight than it looks. The OWASP Top 10 for LLM Applications (2025) starts with prompt injection (LLM01) and sensitive information disclosure (LLM02), and includes excessive agency (LLM06), where a model is given more power to act than it should have. A private LLM keeps the provider out; it doesn't keep a malicious email from instructing your model to forward a file. Our guide on [keeping company data private with AI](/blog/keep-company-data-private-with-ai/) covers these risks.

Even on a cloud platform, you own the operations. Microsoft's documentation for its confidential GPU virtual machines says you are responsible for building high availability and disaster recovery yourself.

## The staff you would need

The roles below are the usual minimum for a system people depend on daily. A small team may combine some, but each job has to belong to someone:

- **An ML or platform engineer** to choose, serve and update models, and to evaluate new versions before they replace old ones.
- **A data engineer** to connect mail, documents and systems of record, and to keep indexes current.
- **A security engineer** to own access control, logging and prompt-injection defenses, and to answer your auditors.
- **An application developer** for the interface and integrations your team uses.
- **On-call coverage**, because a tool the office relies on will break at 7 p.m. on a Friday.

For a family office with a lean team, this is often the deciding factor. Hiring a platform team to run one internal tool is a different business from running a family office.

## Private LLM vs. private AI service

| | Private LLM you run | Private AI service built for you |
|---|---|---|
| Who builds it | Your team or a contractor | The provider |
| Who runs it day to day | Your team | The provider |
| Where it runs | Where you choose | Where you choose, set per client |
| Model choice | Open-weight models you can host | Set per client, including leading providers |
| Retention and access | Whatever you build | Set by contract and design |
| Time to first use | Depends on your team and the scope you build | Set by the provider; ask for a date |
| Best fit | Firms with an engineering team and a reason to own the stack | Firms that want the result without the team |

The comparison isn't about which is more private. Both can keep data off public AI services. It's about who carries the work. Our comparison of [building versus buying private AI](/blog/build-vs-buy-private-ai/) goes deeper on the cost side.

## When each one makes sense

Run your own if you already have a platform team, if you need a specific open model fine-tuned on your data, or if a regulator or principal requires that no outside firm touch the system at all. Add [confidential computing](/blog/confidential-ai/) if the hardware sits in someone else's data center.

Choose a service if what you want is the outcome: an assistant that knows your documents, answers with sources and acts within limits you set, without hiring for it. Then hold the provider to the same questions you would ask your own team: where it runs, what it keeps, and who can reach it. Our guide on [private AI versus public AI](/blog/private-ai-vs-public-ai/) has a checklist.

## Private SuperIntelligence

Private SuperIntelligence from Mitosis Labs is the most capable AI, trusted with everything, delivered as a service. Each deployment is built for one client, from the hardware to the choice of model provider to where the data is stored, and delivery takes weeks. It runs on Mitosis, the knowledge graph Mitosis Labs builds for AI agents, and captures none of the client's data. [Register interest](/#waitlist) for the next intake.

## Frequently asked questions

### What is a private LLM?

A large language model that runs on infrastructure you control, on-premises or in your own cloud account, so your prompts and documents don't go to a public AI provider.

### Can I run an LLM on-premises?

Yes. OpenAI's open-weight gpt-oss-120b runs on a single 80 GB GPU, and gpt-oss-20b fits in 16 GB of memory. Running it for a whole team also takes serving software, retrieval, access control and support.

### Is a private LLM as good as ChatGPT?

Open-weight models are capable, but they are not the same models as the newest ChatGPT release. A private AI service can use leading model providers under private terms, which is one reason firms choose that route.

### Does a private LLM stop prompt injection?

No. Keeping the model in-house stops the provider from seeing your data, but a document or email can still carry instructions to the model. OWASP lists prompt injection as LLM01.

### How many people does it take to run a private LLM?

There's no fixed number, but the jobs are fixed: model serving, data connections, security, the interface and on-call support. A small team can combine roles; someone still has to own each one.


## Related reading

- [Is ChatGPT safe for confidential information?](/blog/is-chatgpt-safe-for-confidential-information/)
- [What is zero data retention AI?](/blog/zero-data-retention-ai/)
- [How to keep company data private when using AI](/blog/keep-company-data-private-with-ai/)
- [Private AI vs public AI](/blog/private-ai-vs-public-ai/)
- [Is Claude, Copilot or Gemini safe for confidential information?](/blog/is-claude-copilot-gemini-safe-for-confidential-information/)
- [Private ChatGPT for business: the options](/blog/private-chatgpt-for-business/)
- [What is private superintelligence?](/blog/what-is-private-superintelligence/)

## Sources

- [gpt-oss-120b model card (OpenAI on Hugging Face)](https://huggingface.co/openai/gpt-oss-120b), accessed 2026-10-15
- [OpenAI open weight models now in Amazon Bedrock and Amazon SageMaker JumpStart (AWS, August 2025)](https://aws.amazon.com/about-aws/whats-new/2025/08/openai-amazon-bedrock-sagemaker-jumpstart/), accessed 2026-10-15
- [OWASP Top 10 for LLM Applications 2025 (OWASP GenAI Security Project)](https://genai.owasp.org/llm-top-10/), accessed 2026-10-15
- [Azure Confidential GPU options (Microsoft Learn)](https://learn.microsoft.com/en-us/azure/confidential-computing/gpu-options), accessed 2026-10-15
- [Data, privacy, and security for Foundry Models sold by Azure (Microsoft Learn)](https://learn.microsoft.com/en-us/azure/ai-foundry/responsible-ai/openai/data-privacy), accessed 2026-10-15
- [Data protection (Amazon Bedrock User Guide, AWS)](https://docs.aws.amazon.com/bedrock/latest/userguide/data-protection.html), accessed 2026-10-15