Once a business decides it needs private AI, the next question is how private — a dedicated cloud tenant, a private cloud region, or on-premise hardware. The answer is usually private cloud for SMBs, on-premise for specific high-sensitivity workflows. This post is how to tell which applies.
The three tiers
- Dedicated cloud tenant. Azure OpenAI Service in your tenancy, Bedrock in your AWS account, or similar. Data stays inside the boundary, no training on your content, model runs on shared infrastructure behind the boundary.
- Private cloud region. Dedicated inference endpoints in a region or VPC the business controls. More expensive, more isolated, still cloud.
- On-premise. Model running on dedicated hardware inside the business's physical facility. Most isolated, most expensive, most operationally demanding.
When dedicated cloud tenant is enough
- HIPAA-regulated healthcare with BAA-covered cloud infrastructure.
- Attorney-client privileged material where the firm's counsel is comfortable with the BAA.
- IRS 7216-governed tax workflows inside the preparation exception.
- Most SMB regulated workloads. This is where most of our engagements land.
When private cloud region earns its cost
- Affiliations with larger systems (hospital groups, enterprise clients) that require stricter tenancy.
- Procurement requirements driven by enterprise clients.
- Workflows where even the shared-infrastructure-behind-a-boundary architecture is unacceptable to the compliance team.
When on-premise is the right call
- CUI-bearing government contracting workflows. See CUI and AI for DMV government contractors.
- Workflows requiring air-gapped operation.
- Regulatory environments where no cloud architecture is acceptable.
- Extreme data-sovereignty requirements.
Cost comparison
Dedicated cloud tenant is the cheapest private option and fits most SMB needs. Private cloud region costs meaningfully more in infrastructure. On-premise carries both hardware and operations cost — you're running a mini data center, even for a small model. For most SMBs, the right default is dedicated cloud tenant unless a specific regulatory or procurement requirement pushes harder.
Operational differences
- Cloud tenant: operated by the cloud provider. Business focuses on the application layer.
- Private cloud region: operated by the cloud provider but with stricter configuration. Business does more infrastructure work.
- On-premise: operated by the business. Real ops burden. Capacity planning, hardware refresh, security patching — the whole job.
Our default recommendation
For most DMV healthcare, legal, and tax engagements, we default to dedicated cloud tenant with BAA-equivalent contracts. For government contracting CUI workflows, we scope on-premise or GovCloud depending on the specific posture. Scope an engagement to talk through your specific requirements.