Build and Deploy Secure AI Workloads with Docker

Course details

Unlock the power of large language models (LLMs) on your local machine using Docker Model Runner. In this course, join Docker Captain and DevOps Institute Ambassador Pawel Piwosz as he shows you how to set up and configure Docker Model Runner to run LLMs locally—making them accessible from both containers and the host system. Explore how to pull and manage prebuilt models from Docker Hub, as well as package and distribute your own models. Along the way, learn how to integrate local LLMs into your applications using the familiar OpenAI API interface. Whether you're building prototypes, safeguarding privacy, or reducing inference costs, this course covers the tools you need to know to run powerful models without relying on the cloud.

Learning objectives
Set up and configure Docker Model Runner to run LLMs locally, accessible from containers and from the host.
Pull, run, and manage prebuilt models from Docker Hub using Docker Model Runner.
Package and distribute custom models on Docker Hub using Docker Model Runner.
Integrate local LLMs into local applications via the OpenAI API provided by Docker Model Runner.

Concepts

Introduction

AI s role in containerization and Docker s response
Quick start - a real-world AI app running locally with Docker

Docker Model Runner

Docker Model Runner - core capabilities
Enable Docker Model Runner
Manage Model Runner with Docker Desktop
Run models with Docker Model Runner (CLI)
Push models to Docker hub
Integrate Model Runner with Docker Compose
Connect containers to LLMs with Model Runner

Docker Offload

Explore Docker Offload solutions
Enable Docker Offload in Docker Desktop
Enable Docker Offload with the CLI
Manage Docker Offload costs

Docker MCP Servers

Explore Docker MCP Servers
Agentic architecture explained
Explore the Docker MCP catalog
Use images from the Docker MCP catalog
Use MCP servers from the CLI

Docker MCP Toolkit in Action

Docker MCP Gateway explained
Using Docker Compose with MCP
Build an MCP-powered app with Docker Compose

Another Approach - docker-agent

What is docker-agent and why use it
docker-agent CLI essentials
Integrate docker-agent with LLM platforms
Build multiagent workflows with docker-agent

Applied AI Real-World Scenarios

Build an AI technical writer
Build an unbiased AI code assistant
Build a custom MCP server with the MCP toolkit
Build a local RAG with Docker Agent

Build and Deploy Secure AI Workloads with Docker

Authors

Pawel Piwosz