Training optimizes model weights from data; inference runs the trained model on new inputs—different costs, failure modes, and update cadences.
Training is batch/offline and compute-heavy. Inference is online serving: latency, throughput, and token economics dominate. “Model updates” change behavior—teams pin versions or run regression suites when hallucination or quality shifts appear.
Mental model
Changing prompts or tools is fast iteration; changing weights (fine-tuning) is slower and needs evaluation discipline.