Databricks · Databricks Data Engineer Associate
Liquid clustering and compute failures
Databricks Data Engineer Associate · Lesson 19 of 22 · 2:22:23
YouTube loads only after you press play. This chapter opens at its timestamp in the full course. Watch on YouTube ↗
In this lesson
Prepare for the Databricks Certified Data Engineer Associate exam. In this lesson: liquid clustering, predictive optimization, compute startup failures and library conflicts, and out-of-memory errors. Includes three original practice questions.
Preview the topics
- Hollowbrook is creating a new sales table. Most queries filter by customer, a column with millions of distinct values, and the filters are expected to change over time. How should the data engineer lay out the table?
- A job cluster stops during startup, and its event log shows an init script failure. The script installs a package from an internal repository. Where should the data engineer look first?
- A notebook loads 300 million sales rows into pandas with toPandas(), to compute the revenue for each day, and the driver runs out of memory. What should the data engineer do?