Multi Arch Docker Buildx in CI/CD

August 14, 2020

Automated Multi Arch Docker Image Builds

I've recently been working on some projects with some heavy dependencies (FFmpeg, scipy, NumPy, etc).

Python Libraries, in particular, take forever to install if a wheel is not available for your OS/Arch This explains why it takes so long to build on Alpine (not even a different arch).

Side note: Future Mac's being ARM-based makes the issue of multi-arch builds even more poignant.

For less common platforms eg linux/arm/v6 (pi zero and original pis) or linux/arm64 some of these builds can take forever and require a dedicated PI (or a emulated VM)

The Difference is in the logs

To demonstrate this we can look at the buildx logs of a image based on the one we will be building in this post.

If we later build the image on an arm64 based on our prebuilt multi arch image we can see it's building for

, we can also see it

  • already has Pythom FFmpeg
  • Installing Pip packages
    • aionotify
      does have a pre built wheel for this arch.
    • asyncpg
      doesn't have a wheel to download for this arch


not having a wheel is what takes so long for different distributions/architectures.

 => CACHED [linux/arm64 production 4/5] COPY requirements.txt /app/requirements.txt                                                                                                                                                                                                                                                 0.0s
 => [linux/arm64 production 5/5] RUN apt-get update     && apt-get install -y build-essential     && pip install --no-cache-dir -r requirements.txt && apt-get remove -y build-essential && apt-get auto-remove -y && rm -rf /var/lib/apt/lists/*                                                                                  75.5s
 => => # Requirement already satisfied: python-ffmpeg==1.0.11 in /usr/local/lib/python3.8/site-packages (from -r requirements.txt (line 3)) (1.0.11)
 => => # Collecting aionotify==0.2.0
 => => #   Downloading aionotify-0.2.0-py3-none-any.whl (6.6 kB)
 => => # Requirement already satisfied: pyee in /usr/local/lib/python3.8/site-packages (from python-ffmpeg==1.0.11->-r requirements.txt (line 3)) (7.0.2)
 => => # Building wheels for collected packages: asyncpg
 => => #   Building wheel for asyncpg ( started

Some notes from the Docker Docs.

Docker Buildx is a CLI plugin that extends the docker command.... It provides the same user experience as docker build with many new features like creating scoped builder instances and building against multiple nodes concurrently.


Build multi-platform images BuildKit is designed to work well for building for multiple platforms and not only for the architecture and operating system that the user invoking the build happens to run. When you invoke a build, you can set the --platform flag to specify the target platform for the build output, (for example, linux/amd64, linux/arm64, darwin/amd64).

Now to show your how to install BuildX (they say it comes with 19.03 and up but does not..) we can look at the dockerfile I wrote for a buildx enabled docker image (to run docker in docker) .

To install locally just run the last two


FROM docker:19.03.10

RUN wget

RUN mkdir -p ~/.docker/cli-plugins && mv buildx-v0.4.1.linux-amd64  ~/.docker/cli-plugins/docker-buildx && chmod a+x ~/.docker/cli-plugins/docker-buildx

Source for this can be found on Github the Docker Image is available here

Automated Builds and Deployment (CI/CD)

The Dockerfile

Let's take a super simple example. We want to create an image based on the standard python image, that comes preinstalled with FFMPEG. More so we want to build this for multiple CPU Architectures, eg amd64 and ARM.

The Dockerfile looks like this an automatically handles cleaning up dependencies. This file could have a few layers, but it's the same dockerfile I use for all my Debian based python docker images.

ARG BASE_IMAGE=python:3.8.5-slim-buster

# Install any runtime depenencies here

RUN apt-get update \
    && apt-get install -y $RUNTIME_DEPENDENCIES \
&& rm -rf /var/lib/apt/lists/*

ENV BUILD_DEPENDENCIES="build-essential"

COPY requirements.txt /app/requirements.txt

# Install any build dpendencies depenencies here
RUN apt-get update \
    && apt-get install -y $BUILD_DEPENDENCIES \
    && pip install --no-cache-dir -r requirements.txt \
&& apt-get remove -y $BUILD_DEPENDENCIES \
&& apt-get auto-remove -y \
&& rm -rf /var/lib/apt/lists/*

The CI config file.

I'm using a self-hosted Drone CI server and runners. This example could easily be switched to anywhere you can enable an experimental docker server. Note at the bottom we use docker in docker.

The .drone.yml file looks like this (a few changes to only show the important parts).

kind: pipeline
name: default

  DOCKER_HOST: tcp://docker-in-docker:2375

- name: Docker Information
  image: ncrmro/docker-buildx:19.03.10
      from_secret: dockerhub_access_token
  - docker version
  - docker buildx version
  - echo "$DOCKER_PASSWORD" | docker login -u "$DOCKER_USERNAME" --password-stdin
  - docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
  - docker buildx create --name multiarch --use
  - docker buildx build --platform linux/amd64,linux/arm64,linux/arm/v7,linux/arm/v6 -t ncrmro/py-ffmpeg-docker-images:latest -t ncrmro/py-ffmpeg-docker-images:1.0.11 -t ncrmro/py-ffmpeg-docker-images:$DRONE_COMMIT_SHA --push .

- name: docker-in-docker
  image: docker:19.03.12-dind
  command: ["dockerd", "--host", "", "--experimental"]
  privileged: true

Also note we

  • started
    This is what emulates our different cpu architectures.
  • created a builder and seet it as the active builder
  • created three image's,
    python FFmpeg ver
    git sha

Hope you enjoyed the post