Tool retrieval at scale

Connect a handful of MCP servers and you can easily hold hundreds of tools. Binding all of them to the model every round wrecks it — context bloat, worse tool selection, higher cost per call. visvoai.core.retrieval (visvoai/core/retrieval.py) is the fix: index every tool once, then per request (or per agent round) bind only the tools relevant to the current intent.

ToolCatalog — BM25, optionally hybrid with cosine

from visvoai.core.retrieval import ToolCatalog

FLEET = [
    ("github__create_issue",   "Create a new issue in a GitHub repository"),
    ("slack__post_message",    "Post a message to a Slack channel"),
    ("postgres__run_query",    "Run a read-only SQL query against Postgres"),
    # ...imagine 300 of these
]

catalog = ToolCatalog(FLEET)
catalog.search("why is the checkout page slow?", k=4)
# → the 4 tool names BM25 ranks highest for that query

ToolCatalog(entries) builds a classic BM25 index (K1=1.5, B=0.75) over one document per tool: the tool's (name, description) — the name is tokenized into the document too, since tool names are highly informative lexically. Tokenization splits on non-alphanumerics and camelCase/ snake_case boundaries, so "kaggle__search_datasets" and "searchDatasets" both become ["kaggle", "search", "datasets"] / ["search", "datasets"] — names from very different naming conventions still retrieve correctly against natural-language queries.

No embeddings required — this works with zero ML infrastructure. Add an L2-normalized embedding as each entry's optional third element to enable hybrid retrieval:

entries = [("tool_name", "what it does", embedding_vector), ...]
catalog = ToolCatalog(entries)
picked = catalog.search(query, k=8, query_vec=my_query_embedding)

search(query, k, query_vec=None):

• No query_vec, or no entries carry embeddings → BM25 only, top-k.
• Otherwise → dual-pool union: top-k BM25 ∪ top-k cosine similarity (query vector · L2-normalized tool embedding, dot product). Union, not reciprocal-rank-fusion, by design — a strong semantic hit binds even when BM25 buries it under lexically similar names, and the result is never worse than BM25 alone, since BM25's own top-k is always included.

Building a catalog from MCP servers

from visvoai.core.retrieval import build_catalog_from_servers

catalog = build_catalog_from_servers(mcp_servers)
# entries are namespaced "{server_name}__{tool_name}"; works with any
# server object exposing .name and .tools[] (each with .name/.description,
# and optionally .embedding)

Wiring retrieval into the agent loop

make_per_round_retrieve(catalog, k=8, embed_query=None) builds the closure build_graph's per_round_retrieve argument expects:

from visvoai.core.retrieval import make_per_round_retrieve

retrieve = make_per_round_retrieve(catalog, k=4)
# retrieve("why is the checkout page slow?") -> ["postgres__run_query", ...]

graph = AgentRuntime().build_graph(
    model=model,
    core_tools=core_tools,                 # always bound, every round
    all_tools_map=all_tools_map,           # superset; non-core tools are deferrable
    system_prompt="...",
    per_round_retrieve=retrieve,
)

embed_query is the seam for semantic ranking: a callable turning a query string into an L2-normalized vector (or None, degrading that call to BM25-only). It's entirely optional — pass nothing and retrieval works with zero embedding infrastructure; supply an embedder (and a catalog whose entries carry matching embeddings) to enable the hybrid pool. The embedder owns its own API key/cache, if any — visvoai-core never calls out to an embedding provider itself.

Inside the loop (visvoai/core/graph.py::call_model), retrieval runs every round, on the most recent human message's text (_intent_query), and is additive to state["active_mcp_tools"] — a persistent, order-preserving set of previously discovered tools (see AgentState). The transient per-round result is not persisted back into state; it self-evicts each round, while anything you explicitly add to active_mcp_tools (e.g. a tool the model asked to "keep") stays bound. A retrieval failure is caught and logged, never fatal to the turn — the round proceeds with whatever tools were already active.

When to reach for this

• Don't bother below a few dozen tools — bind everything (per_round_retrieve=None, the default) and the model sees them all.
• Reach for it the moment tool count starts hurting selection quality or context cost — MCP fleets, plugin ecosystems, or any product where "how many tools does the user's workspace have connected" is unbounded.

(Runnable version: visvoai-core/examples/05_tool_retrieval.py.)

Next: Subagents.