This repositories contains examples and best practices for AI workloads on Azure

Agent Baker is aiming to provide a centralized, portable k8s agent node provisioning lib as well as rich support on different OS image with optimized k8s binaries.

Access large language models from the command-line

A high-throughput and memory-efficient inference and serving engine for LLMs

vLLM’s reference system for K8S-native cluster-wide deployment with community-driven performance optimization

Health checks for Azure N- and H-series VMs.

llm-d is a Kubernetes-native high-performance distributed LLM inference framework

Redis for LLMs

The source for REST API specifications for Microsoft Azure.

Self-hosted huggingface mirror service. 自建huggingface镜像服务。

A handy shell script that enables you to write repeatable demos in a bash environment.

Public sync with private repo azure-aks-docs-pr

Azure HPC/AI VM Images

Setup and configure nodes to support GPUs on k8s, with a focus on AKS nodes. This repo contains steps to build a container image with the Nvidia driver, and dependencies for integration.

NVIDIA GPU metrics exporter for Prometheus leveraging DCGM

Kubernetes AI Toolchain Operator

Container Image for Azure Cloud Shell (https://azure.microsoft.com/en-us/features/cloud-shell/)

NVIDIA GPU Operator creates, configures, and manages GPUs in Kubernetes

NVIDIA NCCL Tests for Distributed Training

Kubernetes Operator for MPI-based applications (distributed training, HPC, etc.)

Run Kubernetes locally

NVIDIA Network Operator