CoreWeave Model Distillation, part of CoreWeave Forge, turns a proven production task into a smaller model, judged head-to-head against the model you run today.
Most production AI calls repeat the same prompt and the same tools millions of times a day. Yet, they still run on a frontier model built to do far more than that one job. In this demo, we distill a customer support agent from start to finish. We capture its traffic through the Model Distillation proxy, build a dataset from logged calls, and use a stronger model to relabel weak answers. Then we train open-weight models on serverless infrastructure, score them head-to-head on held-out production traffic, and roll the best one out to a slice of live traffic.
You pay for the stronger model once, when relabeling, not on every request. Every evaluation row can be opened, so you can see exactly why the judge preferred one answer over another. The rollout is gradual; the model you’re replacing stays live as a reference, and the weights are yours to export. Because traffic is logged the whole time, each version you ship provides training data for the next one.
