AWS · MLA-C02 · BETA
Inference options and endpoints
MLA-C02 · Lesson 12 of 23 · 1:39:11
YouTube loads only after you press play. This chapter opens at its timestamp in the full course. Watch on YouTube ↗
In this lesson
Lesson 12 of the free MLA-C02 beta series (AWS Certified Machine Learning Engineer – Associate, domain 3, deployment and orchestration). In this lesson: real-time, serverless, asynchronous, and batch inference, multi-model and multi-container endpoints, inference pipelines and inference components, deployment targets, orchestrators, and deploying with code, and auto scaling metrics with Inference Recommender. With three quick checks and a recap, and AnyCompany Mobility, a car sharing service, as the running example.
Preview the topics
- The damage model receives photo sets of several hundred megabytes, each taking about ten minutes. Requests come in bursts, and the endpoint should cost nothing between bursts. Which option fits?
- The 300 demand models all use XGBoost, and most cities call their model only a few times each hour. The team wants the lowest hosting cost, and a short delay on the first call is fine. What should it use?
- A real-time endpoint scales on invocations per instance, but sudden spikes at rush hour cause slow responses for a minute or two before new instances arrive. What should the ML engineer change?