Skip to content

Containers & Docker

What you’ll learn. What a container actually is at the kernel level; how images and layers work; Dockerfile best practices and multi-stage builds with real examples; Docker Compose for local stacks; registries and tagging; and image security (scanning, minimal bases, non-root, rootless). Builds on Linux & Networking; leads into Kubernetes.

A container is an ordinary Linux process (tree) with a restricted view of the system and restricted resources, plus a packaged filesystem. There is no “container” object in the kernel; the runtime assembles one from:

  • Namespaces (what the process can see): pid (own process tree, it is PID 1), net (own interfaces, routes, ports), mnt (own mount table/root filesystem), uts (hostname), ipc, user (map UID 0 inside to an unprivileged UID outside), cgroup.
  • cgroups (what it can use): limits and accounting for CPU, memory, I/O, and number of PIDs. Exceeding the memory limit triggers the OOM killer for that cgroup.
  • Union/overlay filesystem: the image’s read-only layers stacked with a thin writable layer on top (OverlayFS).
  • Security layers: Linux capabilities (dropped by default to a small set), seccomp (syscall filtering), AppArmor/SELinux profiles.
Host kernel (shared by all containers)
├─ container A: [pid ns][net ns][mnt ns] + cgroup(512MiB, 1 CPU) + overlay(image A)
└─ container B: [pid ns][net ns][mnt ns] + cgroup(1GiB, 2 CPU) + overlay(image B)

Consequences you should internalize:

  • Containers share the host kernel. They start in milliseconds and are light, but isolation is weaker than a VM; a kernel exploit can escape. Strong multi-tenant isolation uses gVisor, Kata Containers, or Firecracker microVMs.
  • A Linux container needs a Linux kernel; on macOS/Windows, Docker Desktop runs a hidden Linux VM.
  • “Works on my machine” is largely fixed because the userland (libraries, interpreter) ships in the image, but the kernel, CPU architecture (amd64 vs arm64), and resource limits still matter.

Hands-on peek:

Terminal window
docker run -d --name web --memory=256m --cpus=0.5 nginx:1.27
docker inspect -f '{{.State.Pid}}' web # host PID of the container's init
sudo ls -l /proc/$(docker inspect -f '{{.State.Pid}}' web)/ns # its namespaces
cat /sys/fs/cgroup/system.slice/docker-*/memory.max 2>/dev/null | head # cgroup v2 limit

The ecosystem is standardized under the Open Container Initiative (OCI): the image spec (layer format, manifest, config), the runtime spec (how to run a bundle), and the distribution spec (registry API). Docker is one implementation; others include containerd and CRI-O (what Kubernetes uses), runc (the reference low-level runtime), Podman and Buildah (daemonless, rootless-first), and nerdctl. Dockerfiles build OCI images that run anywhere OCI images run. Kubernetes removed its direct Docker (dockershim) integration in 1.24; images built with Docker still run fine on containerd.

An image is a manifest pointing to a config (entrypoint, env, user, exposed ports) and an ordered list of layers, each a compressed tarball of filesystem changes, identified by the hash of its content. Layers are shared and cached: ten images based on the same debian layers store them once, and a pull downloads only missing layers.

Each Dockerfile instruction that changes the filesystem (RUN, COPY, ADD) creates a layer. Deleting a file in a later layer does not shrink the image; it only masks the file (the bytes stay in the earlier layer). So clean up in the same RUN.

Terminal window
docker history myapp:1.0 # layers and sizes
docker image inspect myapp:1.0 | jq '.[0].RootFS.Layers'
docker buildx imagetools inspect nginx:1.27 # manifest list (multi-arch), digests

A tag (nginx:1.27) is a mutable pointer; a digest (nginx@sha256:...) is the immutable content hash. For reproducible deploys, deploy by digest (or pin and automate updates via Renovate/Dependabot). Avoid latest in anything that matters.

A Dockerfile is a recipe evaluated top to bottom. The build cache reuses a layer if the instruction and its inputs (for COPY, the file contents’ hashes) are unchanged; once one layer is invalidated, all later layers rebuild. So order instructions from least to most frequently changing.

A naive Python Dockerfile and its problems

Section titled “A naive Python Dockerfile and its problems”
FROM python:3.12
COPY . /app
WORKDIR /app
RUN pip install -r requirements.txt
CMD python main.py

Problems: any source change busts the cache for the (slow) pip install; the image is large (full Debian + build tools); runs as root; CMD shell form means signals are not forwarded to Python; no .dockerignore so .git and secrets get copied; unpinned base.

1.7
FROM python:3.12-slim AS base
ENV PYTHONDONTWRITEBYTECODE=1 \
PYTHONUNBUFFERED=1 \
PIP_NO_CACHE_DIR=1
WORKDIR /app
# 1) dependencies first: cached until requirements.txt changes
COPY requirements.txt .
RUN --mount=type=cache,target=/root/.cache/pip \
pip install -r requirements.txt
# 2) then the source: changes often, cheap layer
COPY . .
# 3) run as non-root
RUN useradd --system --uid 10001 --no-create-home appuser
USER 10001
EXPOSE 8080
HEALTHCHECK --interval=30s --timeout=3s CMD ["python", "-c", "import urllib.request; urllib.request.urlopen('http://localhost:8080/healthz')"]
# exec form: python is PID 1 and receives SIGTERM
ENTRYPOINT ["python", "-m", "gunicorn", "-b", "0.0.0.0:8080", "app:app"]

And .dockerignore:

.git
.venv
__pycache__
*.pyc
.env
node_modules
Dockerfile
tests/

Checklist of practices:

  1. Small, specific base: -slim, Alpine (musl libc, can surprise with glibc-compiled wheels), or distroless/Chainguard for runtime-only images.
  2. Order for cache: dependency manifests first, source last.
  3. Combine RUN steps with cleanup: apt-get update && apt-get install -y --no-install-recommends pkg && rm -rf /var/lib/apt/lists/*.
  4. Pin versions (base image tag or digest; package versions where reproducibility matters).
  5. Non-root USER with a numeric UID (Kubernetes runAsNonRoot can verify numeric IDs).
  6. Exec-form ENTRYPOINT/CMD (JSON array) for correct signal handling. ENTRYPOINT is the executable, CMD is default arguments.
  7. One process per container is the guideline (not a law); log to stdout/stderr, not files (Observability).
  8. No secrets in layers. ENV, ARG, and COPY’d files persist in image history. Use BuildKit secret mounts:
RUN --mount=type=secret,id=npmrc,target=/root/.npmrc npm ci
Terminal window
docker build --secret id=npmrc,src=$HOME/.npmrc -t myapp .
  1. Prefer COPY over ADD (ADD has implicit tar-extraction and URL fetching behaviors).
  2. Handle PID 1: use --init/tini if your process does not reap children or handle signals.
  3. Build reproducibly with docker buildx/BuildKit (default in modern Docker), and add OCI labels (org.opencontainers.image.source, .revision) for traceability.

Compile in a fat “builder” stage; copy only the artifact to a thin runtime stage. Build tools, compilers, and source never reach production, shrinking the image and attack surface.

FROM golang:1.23 AS build
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 go build -trimpath -ldflags="-s -w" -o /out/server ./cmd/server
FROM gcr.io/distroless/static-debian12:nonroot
COPY --from=build /out/server /server
USER nonroot:nonroot
ENTRYPOINT ["/server"]

The final image is just a few megabytes with no shell or package manager, so there is little for an attacker to use.

Node (build then runtime with prod deps only)

Section titled “Node (build then runtime with prod deps only)”
FROM node:22-slim AS deps
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci
FROM deps AS build
COPY . .
RUN npm run build
FROM node:22-slim AS runtime
ENV NODE_ENV=production
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci --omit=dev && npm cache clean --force
COPY --from=build /app/dist ./dist
USER node
CMD ["node", "dist/server.js"]

You can target a stage (docker build --target build .) for running tests in CI, and BuildKit builds independent stages in parallel and skips unused ones.

Terminal window
docker run --rm -it -p 8080:80 -e LOG_LEVEL=debug -v "$PWD/data:/data" myapp:1.0
docker ps -a
docker logs -f --tail 100 web
docker exec -it web sh
docker stats
docker system df && docker system prune # reclaim space
  • Ports (-p host:container) publish container ports via NAT; EXPOSE is documentation only.
  • Volumes persist data beyond the container’s lifetime: named volumes (managed by Docker), bind mounts (host paths, handy in development). The container’s writable layer is ephemeral; databases must use volumes.
  • Restart policy: --restart unless-stopped.
  • Resource flags: --memory, --cpus, --pids-limit; hardening flags: --read-only, --cap-drop ALL, --security-opt no-new-privileges.

The default bridge network gives each container a private IP behind NAT; user-defined bridge networks add DNS-based name resolution between containers (by container or service name). --network host shares the host’s network namespace (no isolation, no port mapping); none has no network.

Compose declares a multi-container stack in one YAML file, ideal for local development and simple single-host deployments.

compose.yaml
services:
api:
build:
context: .
target: runtime
ports:
- "8080:8080"
environment:
DATABASE_URL: postgres://app:app@db:5432/app
REDIS_URL: redis://cache:6379/0
depends_on:
db:
condition: service_healthy
cache:
condition: service_started
develop:
watch:
- action: sync
path: ./src
target: /app/src
db:
image: postgres:16
environment:
POSTGRES_USER: app
POSTGRES_PASSWORD: app
POSTGRES_DB: app
volumes:
- pgdata:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U app"]
interval: 5s
timeout: 3s
retries: 10
cache:
image: redis:7-alpine
volumes:
pgdata:
Terminal window
docker compose up -d --build
docker compose logs -f api
docker compose exec db psql -U app
docker compose down -v # -v also removes named volumes

Services reach each other by service name (db, cache) on the Compose default network. depends_on with condition: service_healthy waits for health, not merely start; a real app should also retry connections. Use profiles: for optional services, and compose.override.yaml for dev-only tweaks. Compose is a development tool and a fine fit for one host; at multi-node scale, move to Kubernetes (or a managed container service).

A registry stores and serves OCI images: Docker Hub, GitHub Container Registry (ghcr.io), AWS ECR, Google Artifact Registry, Azure ACR, Harbor (self-hosted), Quay. Image names: registry/namespace/repo:tag.

Terminal window
docker login ghcr.io -u USER --password-stdin <<< "$TOKEN"
docker build -t ghcr.io/me/myapp:1.4.2 -t ghcr.io/me/myapp:1.4 .
docker push ghcr.io/me/myapp:1.4.2
docker buildx build --platform linux/amd64,linux/arm64 -t ghcr.io/me/myapp:1.4.2 --push .

Tagging strategy: immutable tags per build (git sha or semver), optionally moving tags (1.4) for convenience, and deploy by digest. Registries should have retention policies (prune old CI images), pull-through caches to avoid Docker Hub rate limits, and private access control. Mirror critical base images into your own registry so an upstream outage or deletion does not block your builds.

Containers are a security boundary only with care. Layers of defense:

  1. Minimal images: fewer packages mean fewer CVEs and fewer tools for attackers. Distroless, Chainguard/Wolfi, scratch for static binaries.
  2. Scan images for known vulnerabilities in OS packages and language dependencies: Trivy, Grype, Docker Scout, Snyk. Run in CI and continuously against the registry (new CVEs appear for old images).
Terminal window
trivy image --severity HIGH,CRITICAL --exit-code 1 ghcr.io/me/myapp:1.4.2
trivy config . # lint Dockerfiles/Kubernetes manifests for misconfigurations
syft ghcr.io/me/myapp:1.4.2 -o spdx-json > sbom.json # generate an SBOM

A CVE count is a noisy signal: prioritize by exploitability, whether the vulnerable code is reachable, and whether a fix exists (VEX statements help). 3. Non-root user; drop capabilities (--cap-drop ALL and add back only what is needed); read-only root filesystem; no-new-privileges; default seccomp profile left on. 4. Never mount the Docker socket (/var/run/docker.sock) into a container, and avoid --privileged; both are effectively root on the host. 5. Sign and verify images (cosign) and generate provenance and SBOMs (DevSecOps). 6. Update base images regularly; automate with Renovate/Dependabot plus rebuild pipelines. Images rot even if your code does not change. 7. Secrets at runtime from the orchestrator or a secrets manager, not baked into images.

Traditionally the Docker daemon runs as root, so anyone who can talk to it is effectively root. Rootless mode runs the daemon and containers as an unprivileged user using user namespaces (UID 0 inside maps to your UID outside), so even a container escape lands as an unprivileged user. Podman is daemonless and rootless by default (CLI mostly compatible: alias docker=podman). Trade-offs: some features need extra setup (binding ports below 1024, certain storage drivers, performance of rootless networking via slirp4netns/pasta). In Kubernetes, the analogous controls are runAsNonRoot, user-namespace support for pods, and Pod Security Standards.

Terminal window
podman run --rm -it -p 8080:80 docker.io/library/nginx:1.27 # no daemon, no root
Terminal window
docker logs --tail 200 -f web
docker inspect web | jq '.[0].State, .[0].HostConfig.Memory'
docker exec -it web sh # may not exist in distroless
docker run --rm -it --pid=container:web --net=container:web nicolaka/netshoot # debug sidecar sharing namespaces
docker events
docker diff web # files changed in writable layer

Exit code 137 usually means SIGKILL (often the OOM killer, check docker inspect ... .State.OOMKilled); 143 is SIGTERM. A container that exits immediately usually has no foreground process or crashed at startup; read docker logs.

  • Copying .git, node_modules, or .env into the image (use .dockerignore).
  • Cache-busting layer order (COPY . . before dependency install).
  • Running as root and with a writable filesystem by default.
  • Unpinned FROM leading to non-reproducible builds, or pinned-forever images that never get security updates.
  • Using containers as lightweight VMs (SSH servers, many services in one container).
  • Storing state in the container filesystem; treat containers as disposable cattle.
  • Ignoring architecture: building on an Apple Silicon laptop (arm64) and deploying to amd64 nodes; use --platform or multi-arch builds.
  • A container is a process with namespaces (isolated views), cgroups (resource limits), and an overlay filesystem; the kernel is shared.
  • Images are stacks of content-addressed layers; order Dockerfile steps by change frequency, clean up within the same layer, and pin by tag or digest.
  • Multi-stage builds separate build tooling from runtime and can yield tiny, shell-less images.
  • Run as non-root with dropped capabilities; never bake secrets into layers; never expose the Docker socket.
  • Scan, sign, and rebuild images regularly; generate SBOMs.
  • Compose is ideal for local multi-service stacks; orchestrators take over at scale.
  1. Hands-on. Containerize a small web app with a naive Dockerfile, then optimize it (layer ordering, slim base, non-root, multi-stage). Record image size and rebuild time after a one-line source change for both.
  2. Hands-on. Run docker run -d --memory=64m a process that allocates 200 MB and observe the exit code and OOMKilled flag.
  3. Conceptual. Why does RUN rm bigfile in a later layer not shrink the image? How would you fix it?
  4. Hands-on. Use unshare --pid --fork --mount-proc and chroot on an extracted Alpine rootfs to build a “container” by hand (no Docker).
  5. Security. List five hardening settings for a production container and what attack each mitigates.
Solutions (3, 5)
  1. Layers are additive; the file’s bytes remain in the earlier layer and are merely hidden in the union view. Download and store cost stays. Fix by downloading, using, and deleting in the same RUN, or by a multi-stage build that never copies the file.
  2. Non-root user (limits damage of app compromise); --cap-drop ALL (removes privileged kernel operations); read-only root filesystem (prevents persistence/tampering); no-new-privileges (blocks setuid escalation); seccomp default profile (blocks dangerous syscalls); distroless base (no shell or tools to pivot with); no Docker socket mount (prevents host takeover).
  • Liz Rice, Container Security (2020).
  • Nigel Poulton, Docker Deep Dive (regularly updated).
  • Docker documentation: “Best practices for writing Dockerfiles”, BuildKit and Compose references.
  • Open Container Initiative specifications (image, runtime, distribution).
  • Julia Evans, “How containers work” zine; Michael Kerrisk’s namespaces man pages (namespaces(7), cgroups(7)).