Assistant on Google Vertex AI

Architecture and data flow

Organisation size

50–20000 employees

Data classes

confidential, personal

Constraints

a Google Cloud organisation with an identity provider and a signed agreement already in place; the model you want offered in the region your residency rule requires; a team, or an implementation partner, who can build a web application

Industries

Technology, Professional services, Financial services, Retail, Media

Jurisdictions

any

No hardware profile was sized for this answer.

API integrationapi-integration

development

Compliance and governancecompliance-governance

compliance

Google Cloudgcp

infrastructure

Identity and SSOidentity-sso

security

Networkingnetworking

networking

Retrieval-augmented generationrag

ml

Security hardeningsecurity-hardening

security

Commands are copied from each project’s own documentation, and the page they came from is linked under the step. 0 of 7 steps currently open an evidence record. The rest are linked to their source; §10 says which of those documents were fetched and which were fetched without their anchor being found — two different states, named differently there.

1. 01

   Choose the region, then confirm the model is offered in itversion-sensitive

   Assessment

   The Agent Platform "provides access to broad classes of large language models (LLMs) and generative AI models, including Google first-party models, partner models, and open-weights models", but not every model in every region. Pick the region your residency rule requires and confirm the specific model is available there before the design depends on it.

   Source documentation

2. 02

   Choose first-party or managed partner model

   Assessment

   Two access routes matter for contracting. A managed API lets you "use managed APIs for partner and open models without managing infrastructure"; the catalogue includes "models from leading AI providers integrated into the platform, such as Anthropic Claude, Grok, and Mistral AI". A first-party Gemini model keeps one contract; a partner model adds the provider’s terms, so decide which before you build.

   Source documentation

3. 03

   Build the retrieval layer with Vertex AI Search

   Assessment

   Vertex AI Search (Agent Search) lets you "build secure, Google-quality enterprise-grade generative AI applications for search and recommendations" over your own content. Index the corpus with your source permissions carried into it, so retrieval cannot return a document the asker could not open.

   Source documentation

4. 04

   Put a service perimeter around the project

   Assessment

   Use VPC Service Controls to draw a perimeter around the project and Private Service Connect so the model endpoint is reached privately. Google’s data-governance page is where the processing commitments live: it states that "Google won't use your data to train or fine-tune any AI/ML models without your prior permission or instruction". Turn the network controls on before the first real document flows.

   Source documentation

5. 05

   Build the application, with Google identity sign-in

   Assessment

   The web application, the conversation store and the audit log are yours. Gate access by group so the assistant inherits the directory you run and leavers lose access automatically, and keep the retrieval hits alongside each answer for traceability.

   Source documentation

6. 06

   Record the no-training default, the model terms and the grounding retentionversion-sensitive

   Assessment

   Google states it "won't use your data to train or fine-tune any AI/ML models without your prior permission or instruction". Record that, the region every component is pinned to, whether you used a partner model (and therefore whose terms also apply), and how long any grounding or caching feature retains your content — verify the current retention wording rather than assuming it.

   Source documentation

7. 07

   Evaluate on your own documents before launch

   Assessment

   A leaderboard score is not evidence about your corpus. Assemble real questions with agreed answers, run them before launch and after every model or index change, and keep the retrieved passages beside each answer so a wrong one can be traced to its source.

   Source documentation

Structured issue-spotting to support your own review — not legal advice. Verify against the cited primary sources and your counsel.

• Azure or AWS, if you are on a different cloud

  The same design on the cloud you already run: a managed model, a managed retrieval service, your app and identity. Choose by the account you hold, not by the model.

  • — Lands in an account, identity provider and agreement you already have
  • — Different model catalogue and region availability per cloud

  azure-openaiaws-bedrock

• Your own model on GPU inside the same project

  When a needed model is not offered, or a residency rule no endpoint satisfies applies, serve open weights on your own GPU instances rather than calling the managed model.

  • — No managed model service sees the prompt; Google is only an infrastructure processor
  • — You own GPU capacity, the serving stack and the evaluation after every upgrade

  vllmopen-webuipgvector

• Private cloudPrivate-cloud RAG in a single regionA GPU instance in one region running vLLM and Open WebUI, a managed PostgreSQL with pgvector for chats and embeddings, object storage for the original files, and your existing identity provider for sign-in. Same software as the on-premise recipe; different trust boundary.
• Private cloudAssistant on Microsoft Foundry (Azure OpenAI)A model deployed in Microsoft Foundry (the platform that was Azure AI Foundry / Azure OpenAI), Azure AI Search for retrieval over your content, a web application you build, Microsoft Entra ID for sign-in, and private endpoints so the call never leaves your virtual network.

Not yet in the ledger

1. Google Cloud — Overview of models on Agent Platform

   Docs

   not yet fetched

2. Google Cloud — Partner models on Agent Platform

   Docs

   not yet fetched

3. Google Cloud — Model as a Service (managed APIs)

   Docs

   not yet fetched

4. Google Cloud — Generative AI data governance

   Vendor page

   not yet fetched

5. Google Cloud — Overview of Agent Search (Vertex AI Search)

   Docs

   not yet fetched

Deployed this stack, or hit something this page does not cover? Corrections, sources and implementation reports are what keep a recipe worth reading.

If you would rather not build it, we can introduce a forward-deployed engineer who has deployed this stack before. The enquiry form starts from this recipe.