AWS · AIF-C01
Evaluating FMs and AI applications
AIF-C01 · Lesson 13 of 17 · 1:34:20
YouTube loads only after you press play. This chapter opens at its timestamp in the full course. Watch on YouTube ↗
In this lesson
Prepare for the AIF-C01 exam (AWS Certified AI Practitioner). In this lesson: evaluation approaches, from human evaluation and benchmarks to Amazon Bedrock Evaluations, the metrics ROUGE, BLEU, BERTScore, and LLM-as-a-judge, how to evaluate RAG, agents, and workflows, and the business objectives that prove an AI application works. Includes three original practice questions.
Preview the topics
- AnyCompany compares the claim summaries of two models with summaries written by adjusters, checking above all how much of the reference content each one captures. Which metric fits best?
- AnyCompany’s RAG assistant gives wrong answers, and the team first wants to know whether the knowledge base retrieves the right passages. Which evaluation fits?
- Which metric shows whether AnyCompany’s customer assistant actually resolves requests without a human agent?