Cost, usage & thinking levels

Two questions arrive the moment an agent reaches production: what did that call actually cost, and how do you ask a model to think harder without writing a branch per provider.

visvoai-ai answers both. usage_from and cost_of turn a reply into real dollars using the registry's per-deployment pricing, and a single off/low/medium/high scale maps onto Gemini's thinking level or budget, Claude's extended thinking, and OpenAI's reasoning_effort — so the calling code never names a provider.

Metering a call: usage_from + cost_of

Every LangChain message/stream chunk carries usage_metadata when the provider reports it. usage_from (visvoai/ai/usage.py) is the one place that reads it:

def usage_from(message_or_chunk: Any) -> dict:
    """{'input', 'output', 'total'} token counts — all 0 when absent."""

from visvoai.ai import build_chat_model, cost_of, usage_from

dep = "gemini:gemini-2.5-flash"
reply = build_chat_model(dep).invoke("Say hi in five words.")

u = usage_from(reply)                                  # {'input': .., 'output': .., 'total': ..}
usd = cost_of(dep, u["input"], u["output"])

cost_of(deployment_id, input_tokens, output_tokens) (visvoai/ai/resolve.py) looks up the deployment and applies its per-million rates:

(input_tokens / 1_000_000) * dep.input_cost_per_million +
(output_tokens / 1_000_000) * dep.output_cost_per_million

Cache-token detail is intentionally not surfaced by usage_from yet — per its docstring, provider reporting is inconsistent in the streaming path, and this is deferred alongside accurate cache pricing. cache_read_cost_per_million lives on the internal Deployment (reachable via get_deployment), not on the public DeploymentInfo projection — so cache-aware cost math you build yourself has to go through get_deployment for now, once you have the cached-token count from the raw provider response.

Gemini billing details worth knowing

The registry's module docstring documents the exact Gemini Developer API billing rules it was built against (Vertex AI prices differ):

• Cached tokens are a subset of prompt_token_count. Bill non-cached input as (prompt_token_count - cached_content_token_count) * input_rate, and cached input separately at cache_read_cost_per_million.
• Thinking tokens bill at the output rate. thoughts_token_count has no separate rate — total billable output is candidates_token_count + thoughts_token_count.
• Google Search grounding billing changed between generations. Gemini 2.5 and older bill per grounded request ($35 / 1,000 = $0.035/call, search_billed_per_request=True); Gemini 3+ bills per search query ($14 / 1,000 = $0.014/query, search_billed_per_request=False, and one API call can fire multiple sub-queries). ModelDefinition.search_query_cost and search_billed_per_request carry this per model.

Thinking levels, normalized

Every provider spells "reasoning" differently — token budgets, effort strings, adaptive on/off. visvoai-ai normalizes all of it to one stable, four-value scale (visvoai/ai/thinking.py):

class ThinkingLevel(str, Enum):
    OFF = "off"
    LOW = "low"
    MEDIUM = "medium"
    HIGH = "high"

This is the only vocabulary a consumer ever needs to know. Everything provider-specific is absorbed by ThinkingMechanism (internal — never surfaced to consumers) and the translation function thinking_kwargs(mechanism, level):

class ThinkingMechanism(str, Enum):
    NONE = "none"
    GEMINI_LEVEL = "gemini_level"                 # Gemini 3+: thinking_level enum
    GEMINI_BUDGET = "gemini_budget"                # Gemini 2.x: thinking_budget int
    ANTHROPIC_BUDGET = "anthropic_budget"          # Claude ≤4.5: legacy budget_tokens
    ANTHROPIC_ADAPTIVE = "anthropic_adaptive"      # Claude 4.6+: adaptive thinking
    OPENAI_EFFORT = "openai_effort"                # OpenAI o-series/gpt-5: reasoning_effort
    OPENAI_COMPAT_REASONING = "openai_compat_reasoning"  # Together/Groq compat reasoning
    OPENROUTER_REASONING = "openrouter_reasoning"        # OpenRouter `reasoning` field

Each Deployment in the registry declares which mechanism it uses (deployments.py::_mechanism), keyed off provider and, for Anthropic, the specific model generation (Claude 4.6+/Opus 4.6-4.8/Sonnet 4.6/Fable 5 get adaptive thinking; older Claude keeps the legacy budget mechanism). A new provider or API shape adds one enum member and one branch in thinking_kwargs — the consumer-facing level="medium" contract never changes.

Some notable per-mechanism quirks baked into thinking_kwargs:

• Gemini thinking_level: OFF must be sent as an explicit {"thinking_level": "minimal", "include_thoughts": False} — omitting the param entirely defaults Gemini to high-effort thinking (a real cost trap).
• Anthropic legacy budget: OFF omits the thinking param entirely (Claude thinking is opt-in). LOW/MEDIUM/HIGH map to budget_tokens 2000/8000/16000.
• Anthropic adaptive (4.6+): OFF also omits the param — an explicit {"type": "disabled"} 400s on some models in this family (e.g. Fable 5). ON is just {"thinking": {"type": "adaptive"}}; there's no separate depth control wired through yet.
• OpenRouter: reasoning rides extra_body={"reasoning": {"effort": level}}, never a top-level reasoning kwarg — a top-level kwarg flips langchain-openai to the OpenAI Responses API, which OpenRouter/Together don't support (see OpenAI-compatible providers).

Choosing a level

Three ways to set the level, in precedence order (resolve.py):

# 1. explicit level= argument (wins)
build_chat_model("gemini:gemini-3-flash-preview", level="high")

# 2. the deployment id's own @effort suffix
build_chat_model("gemini:gemini-3-flash-preview@high")

# 3. the deployment's own default_thinking (falls back to OFF)
build_chat_model("gemini:gemini-3-flash-preview")

resolve_level(value, default=ThinkingLevel.OFF) is drift-safe: None or any unrecognized string falls back to default rather than raising, so a stale stored preference (e.g. a level from a model that's since been deprecated) degrades gracefully instead of erroring.

For power users, thinking_raw={...} on build_chat_model bypasses the enum entirely and passes your dict straight through as provider kwargs.

d = get_deployment_info("gemini:gemini-3-flash-preview")
d.supports_thinking      # True
[l.value for l in d.thinking_levels]   # ['off', 'low', 'medium', 'high'] (or [] if unsupported)
d.default_thinking        # ThinkingLevel.MEDIUM (the registry's "Think" default label)

Next: Grounded search & URL fetch.