# Azure Landing Zone for AI Workloads — Reference Architecture **Publisher:** Earp Strategic Consulting, LLC **Scope:** Enterprise-scale Azure landing zone hardened for generative-AI and ML workloads in regulated industries. **Status:** Reference architecture. Adapt names, regions and address space to your tenant; every value below is an example. --- ## 1. Design principles 1. **Policy at the root, exceptions by pull request.** Guardrails are assigned at the highest management group that applies and inherited downward. An exemption is a reviewed code change with an expiry date, never a portal click. 2. **Private by default.** No PaaS service — including Azure AI Foundry, Azure OpenAI endpoints, Storage, Key Vault and AI Search — accepts traffic from the public internet. Access is through private endpoints resolved by private DNS zones in the hub. 3. **Identity is the perimeter.** Humans authenticate through Entra ID with phishing-resistant MFA and Conditional Access; workloads authenticate with managed identities. No shared keys, no long-lived secrets. 4. **Everything is code.** Hierarchy, policy, network, identity assignments and AI endpoints are deployed from Bicep and Terraform modules through CI with plan review. 5. **Every model call is on the record.** Prompts and responses pass through an inference gateway that writes a hash-chained ledger entry per call. --- ## 2. Management-group hierarchy ``` Tenant Root Group └── # intermediate root — all policy assigned here or below ├── Platform │ ├── Identity # domain controllers / identity services, if any │ ├── Management # Log Analytics, Automation, Sentinel │ └── Connectivity # Virtual WAN, firewalls, DNS, ExpressRoute ├── Landing Zones │ ├── Corp # internal workloads, no inbound internet │ ├── Online # internet-facing workloads behind Front Door │ └── AI # model hosting, inference gateways, vector stores ├── Sandbox # time-boxed experimentation, no hybrid connectivity └── Decommissioned # deny-all; subscriptions awaiting deletion ``` - The `AI` management group inherits everything from `Landing Zones` and adds AI-specific policy (section 7). - Subscriptions are the unit of workload isolation, billing and blast radius: one subscription per workload per environment (`-prod`, `-nonprod`). --- ## 3. Identity | Control | Implementation | |---|---| | Directory | Single Entra ID tenant; B2B for partners, no guest access to Platform subscriptions | | Human access | Entra ID groups mapped to Azure RBAC roles at management-group or subscription scope; no direct user assignments | | Privileged access | Privileged Identity Management (PIM) for Owner, Contributor and User Access Administrator; time-bound activation with justification and approval | | Conditional Access | Phishing-resistant MFA for all admins; compliant-device requirement for portal and CLI; block legacy authentication | | Workload identity | System- or user-assigned managed identities; workload identity federation for CI (GitHub Actions / Azure DevOps OIDC) — no client secrets | | Break-glass | Two cloud-only emergency accounts excluded from Conditional Access, with sign-in alerts routed to the SOC | --- ## 4. Network ### 4.1 Topology - **Azure Virtual WAN** with one secured virtual hub per region. Azure Firewall Premium in each hub, with routing intent sending private and internet traffic through the firewall. - **Spokes** are virtual networks in workload subscriptions, connected to the regional hub. Spoke-to-spoke traffic transits the hub firewall. - **Hybrid:** ExpressRoute (primary) and site-to-site VPN (backup) terminate on the hub. - **Ingress:** Azure Front Door Premium with WAF policy for internet-facing apps, reaching origins through Private Link. - **Private endpoints** for every PaaS dependency, with private DNS zones (`privatelink.*`) linked centrally in the connectivity subscription and assigned by policy. ### 4.2 Example CIDR plan | Scope | Region A (primary) | Region B (secondary) | |---|---|---| | Virtual WAN hub | 10.100.0.0/23 | 10.200.0.0/23 | | Shared services spoke | 10.101.0.0/22 | 10.201.0.0/22 | | Landing-zone spokes (pool) | 10.110.0.0/15 | 10.210.0.0/15 | | AI spokes (pool) | 10.120.0.0/16 | 10.220.0.0/16 | | Sandbox (isolated, not peered) | 172.20.0.0/16 | — | | Reserved for on-premises summarisation | 10.0.0.0/12 excluded | — | Standard AI spoke (/22 from the AI pool): | Subnet | Size | Purpose | |---|---|---| | `snet-app` | /24 | App Service / Container Apps VNet integration | | `snet-aks` | /24 | AKS nodes (Azure CNI Overlay; pod CIDR outside the plan) | | `snet-pe` | /25 | Private endpoints: AI Foundry, AI Search, Storage, Key Vault | | `snet-gateway` | /26 | Inference gateway (API Management internal mode or container) | | reserved | remainder | growth | --- ## 5. Security baseline - **Microsoft Defender for Cloud** plans enabled by policy on every subscription: Servers, Containers, Storage, Key Vault, Resource Manager, and Defender for AI services where available in the region. - **Microsoft Sentinel** on the central Log Analytics workspace in the Management subscription; analytics rules for privileged-role activation, policy exemption creation, private-endpoint deletion and anomalous model-API usage. - **Diagnostic settings** for every resource deployed by `deployIfNotExists` policy to the central workspace; activity logs exported at management-group scope. - **Baseline deny policies** (assigned at ``): - Allowed locations - Deny public network access on Storage, Key Vault, AI services, SQL and AI Search - Deny public IPs on NICs in Landing Zones - Require TLS 1.2+ and HTTPS-only - Require customer-managed keys on AI and data services in the `AI` management group - Deny local authentication (keys) on AI services — Entra ID only --- ## 6. Data - **Microsoft Purview** for data map, classification and lineage across Storage, SQL and Fabric/Synapse sources; sensitivity labels drive which data sets may be used for grounding or fine-tuning. - **Storage:** hierarchical-namespace accounts for data lakes; soft delete, versioning and immutable (WORM) containers for audit payloads. - **Retention:** prompt/response payloads kept in an immutable container for the period your regulator requires; the ledger stores only hashes and metadata, so payload deletion never breaks the chain. - **Encryption:** customer-managed keys in Key Vault Premium (HSM-backed), with key rotation policies and purge protection enabled. --- ## 7. AI services | Component | Configuration | |---|---| | Azure AI Foundry hub / project | Deployed per workload into the AI landing zone; managed virtual network with outbound rules restricted to approved FQDNs | | Model deployments | Private endpoint only; `disableLocalAuth = true`; access via managed identity and RBAC (`Cognitive Services OpenAI User` or least-privilege equivalent) | | Inference gateway | Internal API Management or a container service in `snet-gateway`: authenticates callers, applies quotas, runs guardrail checks, hashes prompt + response and writes the ledger entry | | Grounding | Azure AI Search with private endpoint and semantic ranker as needed; indexes built from Purview-classified sources only | | Content safety | Content filtering configured per deployment; blocked-category events logged to Sentinel | | Evaluation | Scheduled evaluation runs against versioned golden sets; results stored alongside the model deployment version | **AI-specific policy (assigned to the `AI` management group):** deny public network access on AI services; deny key-based auth; require diagnostic settings to the central workspace; restrict allowed model deployments to an approved list maintained as code. --- ## 8. IaC module layout ``` infra/ ├── modules/ │ ├── mg-hierarchy/ # Bicep: management groups + subscription placement │ ├── policy/ # Bicep: definitions, initiatives, assignments, exemptions (with expiry) │ ├── connectivity/ # Terraform: Virtual WAN, hubs, firewall policy, DNS zones │ ├── identity/ # Terraform: groups, PIM role settings, role assignments │ ├── subscription-vending/ # Terraform: subscription alias, MG placement, spoke VNet, budgets, tags │ ├── spoke-ai/ # Bicep: AI Foundry, private endpoints, AI Search, Key Vault, gateway │ └── monitoring/ # Bicep: Log Analytics, Sentinel, data collection rules ├── live/ │ ├── platform/ # one state per platform subscription │ └── workloads/// # one state per vended subscription └── tests/ ├── policy-fixtures/ # known-good / known-bad resources evaluated in CI └── what-if/ # Bicep what-if and terraform plan snapshots reviewed in PRs ``` CI rules: plans are posted to the PR; applies run only from the main branch through a federated (OIDC) deployment identity; policy changes run against fixtures before assignment. --- ## 9. Day-2 operations - **Subscription vending:** new workload = pull request to `live/workloads/`; the pipeline creates the subscription, places it in the right management group, connects the spoke, applies budget alerts and grants group-based access. - **Drift:** nightly plan against every state; drift opens an issue with the diff. Azure Policy compliance is exported daily to the ledger. - **Patching and upgrades:** AKS auto-upgrade channels per environment; model deployment versions pinned and upgraded through the eval gate. - **Key rotation:** automated rotation policies in Key Vault; quarterly test of disabling a key in a non-production subscription. - **Access reviews:** quarterly Entra ID access reviews on all privileged groups; PIM activation reports reviewed monthly. - **Incident response:** Sentinel playbooks to isolate a compromised identity, revoke tokens and disable a model deployment; runbooks versioned alongside the modules. - **Cost:** budgets per subscription, token-usage dashboards per model deployment, and anomaly alerts routed to the workload owner. --- *Earp Strategic Consulting — AI systems, governance and cloud architecture. This document describes a reference design; it is not a statement of compliance. Compliance frameworks are addressed through readiness mapping against your specific control set.*