INSIGHT AI

Source-linked text edition ·

AWS: SageMaker Inference now supports Multi-Instance GPU to maximize GPU utilization for model endpoints

The written, source-linked counterpart to this video.

Spoken scripts · Report and scripts (PDF)

Published edition1 story

AWS: SageMaker Inference now supports Multi-Instance GPU to maximize GPU utilization for model endpoints

Amazon SageMaker Inference now supports GPU partitioning with NVIDIA Multi Instance GPU (MIG), which divides one GPU into hardware isolated partitions so you can run a different model in each or take a right sized fraction of a newer accelerator.

The AWS Machine Learning Blog page documents the AWS update.

Video edition · Markdown edition

Source ledger