Private cloud
Assistant on Microsoft Foundry (Azure OpenAI)
A model deployed in Microsoft Foundry (the platform that was Azure AI Foundry / Azure OpenAI), Azure AI Search for retrieval over your content, a web application you build, Microsoft Entra ID for sign-in, and private endpoints so the call never leaves your virtual network.
- Source
- Editorial recipe — no step evidence has been fetched yet
- Verified
- Evidence not verified
- Confidence
- Low
01Objective
Run a retrieval-grounded assistant on a frontier model inside your own Azure subscription, reachable over your own network, with no GPU fleet to operate.
- use caseDocument Q&AAsk questions of a specific document or a small set of them and get an answer with the passage it came from. Narrower than company-wide search: the user already knows which file matters and wants it read carefully.
- use caseInternal company searchOne search box over the documents a company already has — shared drives, wikis, ticket systems, email attachments — answered by a model that quotes the source. The value is finding the right paragraph in a corpus nobody has read end to end, not writing new text.
- use caseKnowledge managementCapturing what the organisation knows — process notes, decisions, answers given once already — and keeping it findable and current. AI helps by drafting entries, spotting duplicates and flagging pages that contradict each other.
- use casePrivate company ChatGPTA chat assistant for staff that behaves like a consumer chatbot but runs under company control: your accounts, your logging, your retention, your choice of model, and optionally your own documents attached.
02Recommended stack
| Role | Component |
|---|---|
| Authentication | Microsoft Entra ID |
| Gateway | Private endpoints and VNet integration |
| Inference | Azure OpenAI in Microsoft Foundry |
| Inference | GPT-5 (or another model sold by Azure) |
| Ui | Chat application (your own web app) |
| Vector store | Azure AI Search |
Architecture and data flow
Components
- Employees (office network) — people
- Private endpoint / VNet — network
- Chat application (your own web app) — application
- Microsoft Entra ID — identity
- Retrieval layer — retrieval
- Azure AI Search index — vector store
- Application store (chats, users) — database
- Azure OpenAI (managed endpoint) — inference server
- GPT-5 (model sold by Azure) — model
Connections
- Employees (office network) to Private endpoint / VNet — HTTPS (confidential data)
- Private endpoint / VNet to Chat application (your own web app) — reverse proxy (confidential data)
- Chat application (your own web app) to Microsoft Entra ID — OIDC sign-in (personal data)
- Chat application (your own web app) to Retrieval layer — question + user groups (confidential data)
- Retrieval layer to Azure AI Search index — embedding search (confidential data)
- Retrieval layer to Application store (chats, users) — documents + permissions (confidential data)
- Chat application (your own web app) to Application store (chats, users) — chats, users, settings (personal data)
- Retrieval layer to Azure OpenAI (managed endpoint) — prompt + retrieved passages (confidential data)
- Azure OpenAI (managed endpoint) to GPT-5 (model sold by Azure) — loaded weights
External data transfer · SOME
- confidential content leaves your premises for your own cloud tenancy ("Private endpoint / VNet"). You keep control of the account; the provider is a processor, so a DPA and a documented region apply.
- Everything sits in one Azure subscription and region you control; Microsoft is a processor for the model and the search service, not a recipient of your content for its own purposes.
- A model offered through the Marketplace rather than sold by Azure can fall under different terms — the tenancy is yours, the contracting path may not be.
03Suitable for
- Organisation size
- 50–20000 employees
- Data classes
- confidential, personal
- Constraints
- an Azure subscription with a landing zone, Entra ID and a signed agreement already in place; the model you want actually offered in the region your residency rule requires; a team, or an implementation partner, who can build a web application
- Industries
- Financial services, Insurance, Professional services, Technology, Public sector
- Jurisdictions
- any
04Hardware
No hardware profile was sized for this answer.
Indicative costUSD · one-off plus monthly
- Software
- No third-party licence: the application is yours and the managed services are billed on usage. The model bill sits on the model line.
- US$0
- Model usage
- Not estimated here: priced per token — see the cost section, which multiplies a fetched price by the usage band read from your brief. Nothing in this seed asserts a price.
- Not estimated
- Azure AI Search, application hosting and network
- Not estimated: the search tier, the application compute, private endpoints and egress are priced per tier and per region, and no price list was fetched. Read yours before budgeting.
- Not estimated
- Implementation (6–16 FDE-days)
- 6–16 FDE-days at US$760–1940 per day, converted from the HK$6,000–15,000 band at the HKMA Linked Exchange Rate band of HK$7.75–7.85 to one US dollar. One-off; excludes internal staff time.
- US$4,560 – US$31,040
- An Azure landing zone, Entra ID and a signed agreement are already in place — this adds a service, not a cloud programme.
- The model line is metered and left to the cost section to compute from a fetched price and the usage band; nothing here asserts a per-token figure.
- Search tier and application compute recur monthly; implementation is one-off.
05Difficulty
3 / 5
A project week with Linux, Docker and GPU experience
06Skills
- API integrationapi-integration
- development
- Azureazure
- infrastructure
- Compliance and governancecompliance-governance
- compliance
- Identity and SSOidentity-sso
- security
- Networkingnetworking
- networking
- Retrieval-augmented generationrag
- ml
- Security hardeningsecurity-hardening
- security
07Deployment steps
Commands are copied from each project’s own documentation, and the page they came from is linked under the step. 0 of 7 steps currently open an evidence record. The rest are linked to their source; §10 says which of those documents were fetched and which were fetched without their anchor being found — two different states, named differently there.
- 01
Choose the region, then confirm the model and the deployment type in itversion-sensitive
AssessmentResidency drives the region and the region constrains the model catalogue and the processing geography. Azure documents that for a Global deployment type "prompts and responses may be processed in any geography where the relevant model sold by Azure is deployed" — which can undo a residency rule your storage satisfies perfectly. Pick a Data Zone or regional deployment if that is what your rule needs, and check the model is offered there before designing around it.
- 02
Provision the Foundry resource and deploy the model
AssessmentCreate the Foundry resource in the chosen region and deploy the model. The catalogue is large — Microsoft states you can "access more than 10,000 models from Microsoft, OpenAI, Anthropic, Meta, and others" — so the decision is which model is both strong enough and available where you must stay, not which is best in the abstract.
- 03
Build the retrieval layer with Azure AI Search
AssessmentAzure AI Search is "a fully managed, cloud-hosted service that connects your data to AI". Index your corpus and enable vector search — "an information retrieval approach that supports indexing and querying over numeric representations of content" — so the assistant answers from your documents, with permissions carried into the index rather than bolted on after retrieval.
- 04
Close the network with private endpoints
AssessmentReach the model and search over private endpoints so the request never traverses the public internet. Foundry’s platform controls are the ones to turn on here: Microsoft describes applying "Microsoft Entra identity, role-based access control, content filters, network isolation, and Azure Policy". Add a policy that stops a developer deploying the model to a region you did not approve.
- 05
Build the application against the endpoints, with Entra sign-in
AssessmentThe web application, the conversation store and the audit log are yours. Wire sign-in to Entra ID and gate access by group, so the assistant inherits the directory you already run and leavers lose access automatically. Keep the retrieved passages and the model response in your own logs, under your own retention rule.
- 06
Write down the DPA scope, the no-training default and the routingversion-sensitive
AssessmentFor a model sold by Azure, Microsoft states prompts and completions "are NOT used to train any generative AI foundation models without your permission or instruction". Record which model you deployed, whether it is sold by Microsoft or offered through the Marketplace, the deployment type and its processing geography, and the abuse-monitoring setting — as a configuration you can show, not an assumption.
- 07
Evaluate on your own documents before launch
AssessmentA benchmark is not evidence about your corpus. Assemble a set of real questions with answers you agree on, run it against the assistant before launch and after every model version or prompt change, and keep the retrieval hits next to each answer so a wrong one can be traced to the document it came from.
08Compliance considerations
Structured issue-spotting to support your own review — not legal advice. Verify against the cited primary sources and your counsel.
Applies everywhere
Data residency · Cross-border transfershigh
Pin compute, search, storage and logs to one region, and choose the deployment type deliberately: Azure documents that a Global deployment may process prompts in any geography where the model is deployed. Where a model is offered in Hong Kong at all is the first thing to check for a Hong Kong residency rule.
Data processing agreement · Subprocessors · Terms-of-service restrictionshigh
Confirm which entity the model is sold by. A model offered through the Azure Marketplace from another provider can sit outside the standard Microsoft Products and Services DPA and under the provider’s own terms — verify the contracting path before putting real content through it rather than assuming the Azure agreement covers it.
Model training · Retentionhigh
For a model sold by Azure, Microsoft states inputs and outputs are not used to train foundation models without your instruction. Verify the current wording, the retention period, and whether abuse-monitoring stores prompts and whether an exemption applies to your subscription.
Security · Logginghigh
Private endpoints so nothing is reachable publicly, Entra role-based access on the Foundry and Search resources, keys in Key Vault, and account-level audit logging with a retention window that matches your policy.
09Alternatives
AWS Bedrock or Google Vertex, if you are on a different cloud
The same design on the cloud you actually run: a managed model, a managed retrieval service, your app and identity. Choose by the account you already have, not by the model — the models converge and the existing agreement does not.
- — Lands in an account, identity provider and agreement you already hold
- — Different model catalogue and different region availability per cloud
Your own model on GPU inside the same subscription
When a model you need is not offered, or a residency rule no endpoint satisfies applies, serve open weights on your own Azure GPU instances instead of calling the managed model.
- — No managed model service sees the prompt; the provider is only an infrastructure processor
- — You own GPU capacity, the serving stack and the evaluation after every upgrade
- Private cloudPrivate-cloud RAG in a single regionA GPU instance in one region running vLLM and Open WebUI, a managed PostgreSQL with pgvector for chats and embeddings, object storage for the original files, and your existing identity provider for sign-in. Same software as the on-premise recipe; different trust boundary.
- Private cloudAssistant on Amazon BedrockA model served through Amazon Bedrock, Bedrock Knowledge Bases for retrieval over your content (with OpenSearch Serverless if you manage the vector store yourself), a web application you build, IAM Identity Center for sign-in, and a VPC with PrivateLink so the call stays on your network.
10Evidence
Not yet in the ledger
not yet fetched
not yet fetched
not yet fetched
not yet fetched
not yet fetched
not yet fetched
11Community
Deployed this stack, or hit something this page does not cover? Corrections, sources and implementation reports are what keep a recipe worth reading.
Improve this page
Sign in to contribute
From the field
0 deployments · 0 questions
Nobody has reported deploying this here yet, and no question has been opened against this page. Both appear once a reviewer accepts them.
12Hire an FDE
If you would rather not build it, we can introduce a forward-deployed engineer who has deployed this stack before. The enquiry form starts from this recipe.