📡 Idea Radar 每天 8:00 / 18:00 推送 · 18:00 生成精选
reddit · SideProject · 62 分 ← 返回列表

I built a proxy that learns the small decisions my app asks an LLM for, so it stops paying for them

Every app I build ends up asking an LLM the same closed questions thousands of times: which queue does this ticket go to, is this urgent, is this command safe to run. Paying a hosted model per call for that felt wrong, and the open local model (Laya) was not accurate enough on its own for longer label lists. So stuntd watches the traffic, trains a small head per decision, serves it in shadow mode first, and takes over only when it agrees with the teacher at the target I set. B

五维评分

付费意愿
55
需求真实度
60
个人可落地
88
竞争空间
35
新鲜度
65

AI 分析

痛点:AI 应用会反复向 LLM 询问相同类型的闭集决策,按次调用托管模型成本高,而本地小模型单独使用准确率又不够。

目标用户:独立 AI 应用开发者、LLM 应用工程团队、需要控制推理成本的 SaaS 团队。

证据:Paying a hosted model per call for that felt wrong, and the open local model (Laya) was not accurate enough on its own for longer label lists. ... it stops paying for them

变现:开源核心 + 托管代理/企业版:按节省的 token 抽成、月订阅、私有部署、团队策略管理、监控告警和 SLA 支持。

MVP:两周内做一个 Python 代理/SDK,拦截重复的闭集 LLM 调用,记录输入输出,为每个决策训练轻量分类头;先以 shadow 模式验证,达到设定一致率后自动接管,低置信度回退原 LLM;支持 pip 安装和本地无外部调用模式。

竞品:LiteLLM、Portkey、Helicone、GPTCache、RouteLLM、Martian、OpenRouter、Not Diamond

摘要:独立开发者发布 stuntd,一个学习 App 对 LLM 重复闭集决策的代理,先在影子模式训练小模型头,达到阈值后接管,低置信度仍调用原 provider。Snake demo 中训练 138 秒后自主处理 99.6% 动作,得分 11.4,教师为 21。它切中 LLM 按次调用成本痛点,但帖子没有明确付费意愿或收入数据,且 LLM 网关/缓存/路由竞品较多。