Google Vertex AI — Inference on your account
With this integration, SecureAI’s large language model (LLM) calls run in your own Google Cloud project instead of on SecureAI-managed providers. Usage is billed directly to your Google account, so you can use your committed spend (CUDs) and negotiated discounts. SecureAI stays in the path of every request: DLP, model policies, data residency, and signed receipts apply exactly as they do with any other provider.Not to be confused with the other Google Cloud card. This page connects Vertex AI as an inference provider (Admin → Integrations → AI providers). The Google Cloud — Discovery card (Cloud category) only inventories the project in read-only mode and never sends a prompt; it is described in Google Cloud — Discovery. You can use both on the same project, but they are independent and ask for different permissions.
What runs on Vertex and what does not
Models available today: Gemini, the open models served as MaaS on Vertex (gpt-oss, DeepSeek, Llama, and Qwen), and Google’s partner models Claude and Mistral (and AI21), which SecureAI calls through Vertex’s partner-model endpoint (
rawPredict / streamRawPredict). Claude and Mistral run on your Vertex only if you choose their family; see Claude and Mistral, below.
How many cloud providers you can use
With more than one provider enabled, each one serves different model families: a family can be taken by only one enabled provider, and only one provider can be in Remap everything. When you save the integration turned on, SecureAI refuses it in two cases and explains why in the panel:
Prerequisites
- A Google Cloud project with billing enabled.
- The Agent Platform API (Vertex AI API) enabled on that project (the
aiplatform.googleapis.comservice). - A service account in your project with the Agent Platform User role (
roles/aiplatform.user), and one of these two ways for SecureAI to act as it:- Option A — a JSON key for that service account.
- Option B — keyless (impersonation): you grant the SecureAI service account shown in the panel the Service Account Token Creator role (
roles/iam.serviceAccountTokenCreator) on your service account, and no key is stored. This option is available only if your SecureAI deployment offers it; otherwise the panel shows it disabled with the notice Keyless access is not enabled on this deployment.
- Admin permission on the Integrations section of SecureAI. With read permission you can view the configuration but not change it.
Step 1. Prepare the project in Google Cloud
You can do it from the command line (quick) or from the console (with screenshots). Steps 1 to 3 are the same for both authentication options; step 4 (the JSON key) is for option A only.- gcloud (Cloud Shell)
- Google Cloud console
From Cloud Shell or with
gcloud installed. Replace MY_PROJECT with the real project ID:Option B: authorize SecureAI without a key
With option B no key is created. Instead:- In SecureAI, open the Vertex AI panel (see step 2), choose Keyless (impersonation), and enter your service account’s email in Your service account to impersonate.
-
The panel shows the command to run, already filled in with your deployment’s SecureAI service account. Copy it with Copy and run it in Cloud Shell. It looks like this:
The role is granted on your service account, not on the project: SecureAI can only act as that account, which in turn only holds
roles/aiplatform.user.
Open models (Llama, DeepSeek, Qwen, gpt-oss)
Gemini needs nothing more. To also use Llama, DeepSeek, Qwen, or gpt-oss, first accept each model’s terms in Vertex AI → Model Garden. These open models need a specific region (for exampleus-central1); with global they may not be available.
Claude and Mistral
To use Claude or Mistral, enable each model in Vertex AI → Model Garden and accept Anthropic’s or Mistral’s terms for that model. Then, in step 3, tick the Claude or Mistral family. SecureAI only sends these models to your Vertex when you choose their family, because it cannot know whether your project has them enabled: Remap everything alone never sends them there.- SecureAI uses the name Vertex gives them, with dashes: for example
anthropic/claude-sonnet-4-6. A model that needs its dated version (claude-…@20250929) is declared under Models from your cloud that SecureAI doesn’t list, below. - According to Google’s documentation, Claude on Vertex is available in specific regions and on the
globalendpoint. Check the regions in Google’s documentation on Claude in Vertex AI.
Step 2. Connect the project in SecureAI
- Go to Admin → Integrations, open the AI providers category, and click the Google Vertex AI — Inference on your account card.
-
Under Connection, fill in:

- Click Save. No traffic is routed yet: the integration is saved turned off so you can test it first.
Recommendation for day one: keep the Only these model families mode with Gemini only, the default models unchanged, and both fallback and the per-seat spend cap off. It is the most predictable setup. Leave Remap everything for when this has been validated.
With option A, the pasted key is never shown again. To replace it, paste a new one; if you leave the field empty when saving, the current key is kept.
Step 3. Choose what runs on Vertex
Under What runs on Vertex there are two modes:- Only these model families — the families you tick (Gemini, gpt-oss, DeepSeek, Llama, Qwen, Claude, Mistral) run on your Vertex; every other model keeps running on SecureAI’s providers. This is the most conservative way to start (default: Gemini only).
- Remap everything to Vertex — all LLM calls run on your Vertex. Models with no Vertex equivalent are served by a default model. Claude and Mistral go to Vertex only if you tick their family (see Claude and Mistral, in step 1).
There are also two options:

Available models depend on the location. The default models (several of them preview) may not exist in every region. If the connection test returns Model not available in this location, choose another region or replace the default models with ones your region serves. Open models (MaaS) require accepting their terms in Vertex AI Model Garden and a specific region. Claude and Mistral require the model to be enabled in Model Garden.
Step 4. Test the connection
- With the integration saved, under Connection test click Test connection.
-
SecureAI sends a minimal request to each default model, through the same path real traffic will use (gateway, policies, and receipts included), and shows each result with its latency.

- If a model fails, the panel shows the reason. See the Troubleshooting section at the end of this page.
What else the identity can do
The test also checks what else the connection’s identity can do in the project, withprojects:testIamPermissions and the lane’s own token: no extra role is needed and nothing changes. The panel shows one of these notices:
The test sends a few one-token requests and is billed to your Google account (cents).
Step 5. Review the routing and turn it on
- Open Routing preview → Show which lane serves each model. For every catalog model it shows whether your Vertex will serve it (and with which Vertex model) or SecureAI will, and which ones will be hidden from users. The preview simulates the integration being on, so you can review the effect before enabling it.
-
When the test passes, turn on Route traffic to Vertex AI and click Save again. If you do not save, the switch does not stay on. To turn it on, a project and a credential must be configured. If your plan does not allow another cloud provider or the routing overlaps another one, SecureAI refuses it and explains why (see How many cloud providers you can use, above).

- The card turns green: Routing to Vertex. If it is saved but off, it shows amber Configured · not routing to Vertex.
Step 6. Verify in the model selector
When the integration is active, models that run on your Vertex show the Google Cloud logo next to their name in the chat model selector. Hovering it says the model runs on your organization’s Vertex AI account.
Models from your cloud that SecureAI doesn’t list
Your project may have chat models enabled in Model Garden that SecureAI’s catalog does not include (for example a newer Claude, or Codestral). Once your SecureAI deployment enables them, the chat can offer them too:- Where they come from. Model Garden does not say which models your project enabled, so you name them in the Vertex AI panel as
publisher/model(for exampleanthropic/claude-opus-4-7ormistralai/codestral-2), add them with Add, click Save, and run Test connection to check that each one answers. A model that a SecureAI catalog model already reaches keeps its catalog name and is not repeated here. - Where they appear. In the chat model selector, in their own section: Your Google Vertex AI. Their ids start with
byo/gcp-vertex/. - Where they run. Only on your Vertex. They are never sent to SecureAI-managed providers, even if you allowed fallback.
- What they cost. SecureAI does not charge for them (they have no SecureAI list price), so the per-seat spend cap does not apply to them, and any seat can use them. Google bills the usage to your account, like the rest.
- Same governance. They go through the model policy, provider and vendor blocks, DLP, signed receipts, and audit. If your model policy lists the allowed models, add them in the policy editor, where they show up under your cloud’s section.
- What each one takes. Tools, images, and context size come from the facts the provider publishes about the model. If a model takes no tools or no images, the chat does not offer them for it, and images are never sent to another provider to be described.
What is billed, and to whom
- LLM usage is billed by Google to your account, not by SecureAI. You will see the charge in your Google Cloud billing, with your discounts and CUDs applied.
- SecureAI shows the cost of turns served on Vertex as a list-price estimate for analytics only. It is a reference: it does not include your discounts, so Google’s actual figure may be lower.
- Turns served on your Vertex do not count against each user’s seat spend cap, unless you turn on Apply each seat’s spend cap to turns served in your account. The seat level still applies: a user can only use models of their level or below, even when your Vertex serves them.
- A Govern BYO seat can only use models served on your organization’s own infrastructure: the ones your Vertex (or another connected cloud provider) serves and the self-hosted models on your servers. Any other model is refused, and the selector does not offer it Auto.
Security and compliance
- Encrypted credentials, or none. With option A, the service-account key is stored encrypted (AES-256-GCM) and is never returned to the browser, not even to an administrator; the screen only shows the account’s email. With option B no key is stored: SecureAI asks Google for short-lived tokens that act as your service account.
- The path stays protected. Every call goes through SecureAI’s SMLTP gateway: DLP, a signed per-request authorization token, and a verifiable signed receipt.
- Data residency. With a pinned region (for example
europe-west4), SecureAI sends each request to that region’s endpoint and applies your organization’s residency policies. Where Google processes the request depends on Google’s data-residency commitments. - Fails closed. If the configuration cannot be read or Google’s token cannot be obtained, the request is refused; it is not sent to another provider unless you allowed fallback.
- Audit. Saving, testing, or disconnecting the integration is recorded in the activity log, with the names of the fields changed and never the key.
- Least privilege, visible. The connection test checks what else the connection’s identity can do in the project and warns if it holds permissions outside AI (see What else the identity can do, in step 4).
- Minimal permissions.
roles/aiplatform.useris enough for inference; the integration does not need to read your IAM or billing.
Disconnect
Disconnect (panel footer) deletes the configuration and the stored key. From that moment, all traffic returns to SecureAI-managed providers. You can also turn off only the Route traffic to Vertex AI switch to pause routing while keeping the connection. Then remove the access in Google Cloud too:- Option A: if you stop using the key, revoke it (Service Accounts → Keys).
- Option B: remove the
roles/iam.serviceAccountTokenCreatorrole you granted the SecureAI service account on your service account (Service Accounts → your account → Permissions).
Troubleshooting
These are the reasons Test connection can show:
Errors users may see in the chat:
Current limitations
- Embeddings, re-ranking, OCR, the admin assistant, image generation, and realtime voice stay on SecureAI-managed providers.
- One Vertex project and location per organization.
- The number of inference providers you can enable at the same time depends on your plan (Scale: 1, Enterprise: up to 3). See Azure AI Foundry — Inference on your account and Amazon Bedrock — Inference on your account.
- The Claude Code proxy uses Anthropic directly and does not go through Vertex.
Related
- Google Cloud — Discovery — inventory of the project’s agents, models, identities, and costs.
- Azure AI Foundry — Inference on your account
- Amazon Bedrock — Inference on your account
- Cloud AI Providers Overview
- Signed receipts





