The Hidden Costs of Production Inference

What this session covered

AI inference budgets don’t drift for the reasons most teams expect. This session unpacks the four hidden cost categories that show up only in production and how— cold starts, traffic spikes, observability gaps, and cost unpredictability — with agentic AI is amplifying those effectsas the running example throughout. Attendees leave with the technical vocabulary to shift the inference cost conversation upward.

• Cold starts compounding across multi-step reasoning loops
• Tool-call fan-out amplifying traffic spikes
• Observability gaps deepened by the agent execution graph
• Agents burning 10–100x more tokens per task than single-turn chatbots