Now vibe coding, so learning hammer FE ?https://channel.meo-ai.comhttps://channel.meo-ai.com/posts/663https://channel.meo-ai.com/posts/663Wed, 17 Jun 2026 01:11:36 GMT<span class="post-title-source">《永不浪费一个Token：AI推理流的可恢复性设计》</span><br /><br />标签：<a href="/search/result?q=%23%E5%90%8E%E7%AB%AF" title="#后端">#后端</a> <a href="/search/result?q=%23AI%E6%8E%A8%E7%90%86" title="#AI推理">#AI推理</a> <a href="/search/result?q=%23Cloudflare" title="#Cloudflare">#Cloudflare</a> <a href="/search/result?q=%23DurableObject" title="#DurableObject">#DurableObject</a> <a href="/search/result?q=%23%E6%B5%81%E5%BC%8F%E6%81%A2%E5%A4%8D" title="#流式恢复">#流式恢复</a> <a href="/search/result?q=%23Token%E8%AE%A1%E8%B4%B9%E4%BC%98%E5%8C%96" title="#Token计费优化">#Token计费优化</a><br /><br />总结：<br />本文探讨了AI Agent在推理过程中因进程崩溃或重新部署导致流式连接中断、已付费Token被重复计费的痛点。核心方案是在Agent与LLM提供商之间引入一个持久化缓冲区（Durable Object），将流式数据实时写入SQLite，使连接与Agent进程解耦。Agent重启后可通过游标恢复，避免重复调用和重复付费。该机制同时解决了浏览器断网重连和进程崩溃恢复两种场景，且即将集成到Cloudflare AI Gateway中，实现一键开启的持久化推理。<br /><br />文章要点：<br />1. 隐藏的成本黑洞：Agent进程崩溃或重新部署时，正在进行的LLM流式请求会中断，已付费的生成Token全部丢失，恢复后必须重新调用并重新付费，且旗舰模型（如GPT-5.5）的重复成本是小模型的15倍。<br />2. 解耦连接与进程：将LLM提供商连接移出Agent进程，部署为独立的持久化缓冲区（Durable Object），在后台持续将流式数据写入SQLite。即使Agent进程被替换，缓冲区仍继续接收数据，Token不再浪费。<br />3. 一份日志，两种用途：持久化缓冲区存储的原始字节流既支持浏览器断网重连（追赶实时游标），也支持进程崩溃恢复（回放已存储的片段）。两者共用同一套SQLite日志机制，仅需判断生产者是否仍在运行。<br />4. 零自定义解析：存储原始字节而非解析后的SSE事件，恢复时通过各提供商的官方插件（如OpenAI、Anthropic）重新解析，避免维护多套格式解析器，且能自动适配格式变更。<br />5. 行业现状对比：OpenAI Responses API已原生支持后台模式恢复；Anthropic和Google Gemini均不支持服务端续流，只能重新提示并重复计费；Vercel的<code>resumable-stream</code>为应用层方案，无法承受部署替换。<br />6. 即将落地AI Gateway：Cloudflare AI Gateway正在集成该持久化恢复能力，未来只需在Agent基类中设置<code>durableBuffer = true</code>即可一键开启，实现跨提供商的Token零浪费。<br /><br />URL：<a href="https://sunilpai.dev/posts/never-waste-a-token/" target="_blank" rel="noopener" title="https://sunilpai.dev/posts/never-waste-a-token/">https://sunilpai.dev/posts/never-waste-a-token/</a><a class="tgme_widget_message_link_preview" href="https://sunilpai.dev/posts/never-waste-a-token" target="_blank" rel="noopener" title="durable inference: resumable streams, crash recovery, and why the LLM request shouldn't die with your process."> <div class="link_preview_site_name accent_color">Solving the decision problem</div> <img class="link_preview_image" alt="never waste a token" src="/static/https://cdn4.telesco.pe/file/uHdH2ydGrBpImS6yyNDlEDCpezqDJHVi4VJ787KA-5JzVHqpKDz2skyD1IkcJl-JWsJXS_cL7Zz5eZcup8qUKZL5AS2eud86C9zlfexF4zE4pKORoLpxrwFniNgkRm274h_6ALna2B8dBYm-1UZzlEERSxMj7-Vd46jSd_hDv5FwfMuCiylul6PiW4JVwmI3P_nxlaQ7ODor-w0loTOvNlHj9lAIcyLwJsYhEFnVijYrzoQTy0roEE84c0H55IEsXmrIjMY9i9ps8ghJj-Vcdfb4328m17gE6qUmsj-az1_ilYzSwhf-_OGEe9v4NMe7dbWgUQZ5Kfe-nlj1dw9LuA.jpg" width="1200" height="630" loading="eager" /> <div class="link_preview_title">never waste a token</div> <div class="link_preview_description">durable inference: resumable streams, crash recovery, and why the LLM request shouldn't die with your process.</div> </a>