Skip to main content

Vercel AI Gateway

Vercel AI Gateway routes Toby’s requests to many upstream providers (OpenAI, Anthropic, Google, Amazon Bedrock, Meta, Mistral, DeepSeek, xAI, and more) through one API and one key.

Personas that use the vercel provider must use gateway model slugs in provider/model form, for example openai/gpt-5-mini or anthropic/claude-sonnet-4.6.

Get an API key​

  1. Sign in to the Vercel dashboard.
  2. Open your team’s AI Gateway settings (see Authentication in Vercel’s docs).
  3. Create an API key and copy it.

Alternatively, for local development on a linked Vercel project:

  • Run vercel env pull to provision VERCEL_OIDC_TOKEN so Toby can authenticate without a static key (see Vercel OIDC).

You can also set AI_GATEWAY_API_KEY in your environment instead of storing a key in ~/.toby/credentials.json.

Configure in Toby​

toby config
  1. Go to AI → Vercel AI Gateway → API Key and paste your key, or rely on AI_GATEWAY_API_KEY / VERCEL_OIDC_TOKEN.
  2. For a persona, set AI Provider to vercel.
  3. Pick a model from the list or enter a Custom model slug (any slug your gateway account exposes).

Toby validates slugs as provider/model (for example google/gemini-3-flash).

The configure UI lists slugs Toby tests against regularly. These work well for Toby’s chat, tool use, summarization, and schedules.

SlugRole
openai/gpt-5-miniBest default on the gateway—same family as the built-in Toby persona’s OpenAI direct default.
openai/gpt-4.1-nanoDefault pretreatment model on the vercel provider.
openai/gpt-4.1-miniUsed for planning and other explicit auxiliary calls.
anthropic/claude-sonnet-4.6Strong all-rounder if you prefer Anthropic for long context and careful tool use.
google/gemini-3-flashFast Google option for quick turns and high-volume schedules.

Faster or lower cost​

SlugRole
openai/gpt-5-nanoLightest OpenAI tier on the gateway.
anthropic/claude-haiku-4.5Fast Anthropic model for triage-style work.
google/gemini-2.5-flashPrevious-generation flash model; still useful where available.
amazon/nova-liteInexpensive option on Bedrock-backed routes.

Higher quality​

SlugRole
openai/gpt-5.4Top-tier OpenAI via gateway when quality matters more than cost.
anthropic/claude-opus-4.6Highest-capability Anthropic tier for difficult drafting or analysis.
google/gemini-2.5-proHeavier Google model for complex requests.
mistral/mistral-mediumSolid European-hosted option when it fits your policy needs.

Reasoning and alternatives​

SlugRole
deepseek/deepseek-v3.2Cost-effective reasoning-oriented model where enabled on your gateway.
xai/grok-4-fast-reasoningxAI reasoning route for experiments or comparison.
meta/llama-4-scoutOpen-weight style routing when you want Meta’s scout tier.

Start with openai/gpt-5-mini unless you have a reason to prefer another vendor. Use Custom model slug for any model listed in your gateway that Toby does not yet show in the picker.

Automatic caching​

Toby enables Vercel automatic caching on gateway requests (gateway.caching: auto) and adds provider-specific hints (for example OpenAI promptCacheKey and Anthropic ephemeral cache markers) so repeated system prompts cost less when the upstream supports it.

App attribution​

Toby sends optional app attribution headers on gateway requests so Vercel can list Toby on AI Gateway ecosystem pages:

HeaderDefault
http-refererhttps://github.com/kshehadeh/toby
x-titleToby

Override with:

VariablePurpose
TOBY_AI_GATEWAY_REFERER or AI_GATEWAY_HTTP_REFERERCustom referer URL
TOBY_AI_GATEWAY_APP_TITLE or AI_GATEWAY_X_TITLECustom app title
TOBY_PRETREAT_MODELPretreatment model (full slug, e.g. openai/gpt-4.1-nano)
TOBY_PRETREAT_DELTASet to 0 to disable follow-up delta pretreatment
AI_GATEWAY_API_KEYAPI key instead of configure-stored credentials

Switching from OpenAI direct​

If a persona used gpt-5-mini on OpenAI direct, set the vercel provider and model to openai/gpt-5-mini—the suffix matches; only the provider and slug format change. Toby can also normalize model ids when you change provider in configure.

See also​