Databricks · Databricks Generative AI Engineer Associate
Retrieval Evaluation
Databricks Generative AI Engineer Associate · Lesson 5 of 17 · 27:03
YouTube loads only after you press play. This chapter opens at its timestamp in the full course. Watch on YouTube ↗
In this lesson
Prepare for the Databricks Certified Generative AI Engineer Associate exam. In this lesson: a repeatable evaluation set before any tuning, recall at k and precision at k, the retrieval relevance and sufficiency judges, improving retrieval step by step (hybrid search, filters, reranker, data, query, prompt, then fine-tuning), and what a reranker costs and brings. Includes three original practice questions.
Preview the topics
- Copperline’s warranty answers must list every condition that applies, and an answer that misses one condition is wrong. Which retrieval metric should the team watch first?
- Customers type product codes such as CX-450, and Copperline’s vector search often returns manuals of other models with similar names. What should the team try first?
- Copperline’s agent retrieves fifty candidate chunks, and the right one is usually among them, but rarely in the top three that go into the prompt. What should the team add?