AWS · MLA-C02 · BETA
Hosting foundation models and imported models
MLA-C02 · Lesson 13 of 23 · 1:46:21
YouTube loads only after you press play. This chapter opens at its timestamp in the full course. Watch on YouTube ↗
In this lesson
Lesson 13 of the free MLA-C02 beta series (AWS Certified Machine Learning Engineer – Associate, domain 3, deployment and orchestration). In this lesson: Amazon Bedrock on-demand inference and Provisioned Throughput, Custom Model Import and models built elsewhere, LMI containers, inference components, and LoRA adapters on SageMaker AI, and GPUs, Inferentia, Trainium, and scaling. With three quick checks and a recap, and AnyCompany Mobility, a car sharing service, as the running example.
Preview the topics
- The assistant serves a steady, high volume around the clock, and on-demand calls are sometimes throttled at peaks. It will run for at least a year, and the team wants the lowest hourly price for guaranteed throughput. What should it buy?
- The partner's model is a fine-tuned Llama model in the Hugging Face format. The team wants to call it through the Bedrock API, like its other models, and without managing any endpoint. What should the ML engineer do?
- A transformer model serves millions of short predictions every day on a SageMaker AI endpoint. The team wants the lowest cost per inference, and is open to accelerators designed by AWS. Which instances first?