Data Systems for AI
Cache Saver
Efficient and reproducible LLM inference
Active ResearchOverview
Cache Saver caches LLM calls in multi-step pipelines. Its namespace-aware, list-valued cache keeps sampled outputs statistically valid and makes experiments reproducible. It cuts inference cost by about 25% and CO2 emissions by about 35% on average.