Baseten adds deferred tool loading to Model APIs

baseten.co5 min read

Baseten has added deferred tool loading for open models on its Model APIs to reduce token bloat and latency in coding agents. By retrieving tool schemas just-in-time rather than preloading them into the system prompt, benchmarks showed coding agents were up to 62.5% cheaper and 26.2% faster. Open article →