Models that jointly handle more than one modality—commonly text plus images, audio, or documents—so prompts can include screenshots, diagrams, or clips.
Multimodal systems expand context windows with richer inputs: vision for UI screenshots, audio for meetings via speech models, and document PDFs. Costs and failure modes differ from text-only: OCR errors, compression artifacts, and privacy of camera/mic data.
Pipelines
Many products chain specialists (speech-to-text → LLM) instead of one giant model—design for evaluation at each hop.