reddit · SaaS · 09-28 19:50 65 分 ← 返回列表

Why are 33 tool schemas riding along on every turn?

the per request average was looking fine. the routing policy narrows 33 tools down to 4 candidates but all 33 schemas still land in the context window on every turn. I figured prefix caching was absorbing enough of that payload to make tool schema pruning something I could deal with later. not ideal. so I priced a typical 18 turn support session end to end and the schemas alone push roughly 72000 input tokens. the generation bill is lower than the schema block (I realized this while reviewi

五维评分

付费意愿

需求真实度

个人可落地

竞争空间

新鲜度

AI 分析

痛点：LLM工具调用场景中，大量tool schema每轮都进入上下文，导致token成本畸高且难以量化裁剪后路由质量是否下降。

目标用户：构建多工具LLM Agent的AI工程师、平台团队和SaaS开发者。

证据：the schemas alone push roughly 72000 input tokens. the generation bill is lower than the schema block... I'm evaluating Braintrust for span level token attribution, experiment diffs across schema sets and regression datasets around routing changes... has anyone found a good way to measure whether an experiment diff on smaller schema sets is preserving routing quality while cutting session cost?

变现：按量收费的schema token分析/裁剪代理，或团队订阅的Agent成本与路由质量评测平台；开源CLI引流企业版。

MVP：两周内做一个轻量代理/SDK：拦截OpenAI/Anthropic工具调用请求，统计每轮schema token占比，支持定义裁剪规则并跑固定路由测试集，输出成本节省与路由准确率对比报告。

竞品：Braintrust、LangSmith、Helicone、Langfuse、PromptLayer、W&B Weave

摘要：用户发现33个tool schema每轮都进入上下文，一个18轮支持会话仅schema就约72k input tokens，成本高于生成本身。他正在评估Braintrust做span级token归因和路由回归测试，寻找既能裁剪schema又不破坏工具选择质量的方法。这说明LLM Agent的tool schema成本优化与评测存在独立开发者可切入的产品机会。