Microsoft · AI-300
Observability
AI-300 · Lesson 13 of 15 · 1:32:24
YouTube loads only after you press play. This chapter opens at its timestamp in the full course. Watch on YouTube ↗
In this lesson
Prepare for the AI-300 exam (Microsoft Certified: Machine Learning Operations Engineer Associate). In this lesson: latency, response time, throughput, tokens, and errors, OpenTelemetry tracing with Application Insights, continuous evaluation and alerts, and tracking and reducing token cost. Includes three original practice questions.
Preview the topics
- Users report slow answers. One trace shows a request of six seconds, where the span that generates a structured profile takes five. What should the team optimize first?
- Groundedness was high at release, but a month later users complain about invented answers, while latency and errors look normal. What would have caught this early?
- Traces show 6,000 input tokens per request, mostly a long preamble repeated in every prompt, while the answers are short. What should the team do first?