Fixing Spin Apps: Containerd-shim-spin Hangs On Docker V28+
Unraveling the Mystery: Spin Apps Stuck on Docker v28+
Hey everyone! If you're diving deep into the world of WebAssembly (WASM) and trying to run your awesome Spin applications directly on Docker, you might have hit a bit of a snag, especially if you've updated to Docker v28 or newer. We're talking about a pretty significant hiccup where your Spin apps are failing to run on Docker v28+, just completely hanging instead of springing to life. This isn't just a minor annoyance; it's a critical containerd-shim-spin hanging issue that prevents your WASM workloads from executing as expected. Historically, things worked smoothly with Docker v27, allowing developers to seamlessly deploy their Spin framework applications. However, something fundamental shifted with Docker v28, and now we're seeing these applications get stuck during startup. This is a huge deal for folks who rely on Docker for their development and deployment workflows, especially when embracing the lightweight, high-performance nature of WebAssembly. The core of the problem lies within the containerd-shim-spin component, which is designed to bridge the gap between containerd (Docker's underlying container runtime) and Spin applications. When you attempt to docker run a Spin app, this shim is supposed to orchestrate its execution. But with Docker v28+, that orchestration is hitting a wall, leaving your containers in an idle, unresponsive state. This issue has highlighted a crucial gap in our testing strategy, as the containerd-shim-spin has traditionally been tested predominantly within Kubernetes environments, specifically k3d, but not extensively against direct Docker invocations, especially with newer versions. This oversight means that changes in Docker's internal workings or containerd's API interactions can go unnoticed until they hit users in production or development. Understanding why this shift occurred and how to diagnose it is paramount for the entire Spin community and anyone pushing the boundaries of WASM in containers. The goal here is to get your WASM applications back up and running reliably on Docker, no matter the version. It’s all about ensuring that the promise of efficient, portable WebAssembly applications within the familiar Docker ecosystem remains intact. We'll explore this Docker v28 compatibility issue in detail, look at how to reproduce it, and discuss the diagnostic steps taken so far.
Understanding the Core Problem: What's Happening with Docker v28+?
Alright, let's get down to the nitty-gritty of this containerd-shim-spin hanging issue on Docker v28+. The central complaint, as many of you have probably experienced, is that when you try to launch a Spin application using Docker versions 28 or newer, the command simply hangs. It doesn't throw an error, it doesn't crash, it just... stops. Imagine you're trying to fire up your shiny new Spin app, maybe something like ghcr.io/spinframework/containerd-shim-spin/examples/spin-rust-hello:v0.22.0, using the docker run command with the --runtime io.containerd.spin.v2 flag. On Docker v27, this command would output something similar to the following, happily indicating that your app is serving and its routes are available:
$ docker run --name spin --runtime io.containerd.spin.v2 --platform wasi/wasm --publish 8080:80 ghcr.io/spinframework/containerd-shim-spin/examples/spin-rust-hello:v0.20.0 /
Unable to find image 'ghcr.io/spinframework/containerd-shim-spin/examples/spin-rust-hello:v0.20.0' locally
v0.20.0: Pulling from spinframework/containerd-shim-spin/examples/spin-rust-hello
a8350dc9cb8c: Download complete
53dcbf4e5934: Download complete
f012b1ca8a19: Download complete
Digest: sha256:1c3bcefb614e433c0c0548aa6b36b6d70eb2c27ae438bc60b088d1b7d1440b34
Status: Downloaded newer image for ghcr.io/spinframework/containerd-shim-spin/examples/spin-rust-hello:v0.20.0
Serving [http://0.0.0.0:80](http://0.0.0.0/)
Available Routes:
hello: [http://0.0.0.0:80/hello](http://0.0.0.0/hello)
go-hello: [http://0.0.0.0:80/go-hello](http://0.0.0.0/go-hello)
This is the expected behavior – your Spin app starts, listens on its specified port, and is ready to handle requests. It's beautiful, right? But then you try the exact same command on Docker v28+, and you get this:
$ docker run --name spin --runtime io.containerd.spin.v2 --platform wasi/wasm --publish 8080:80 ghcr.io/spinframework/containerd-shim-spin/examples/spin-rust-hello:v0.22.0 /
Unable to find image 'ghcr.io/spinframework/containerd-shim-spin/examples/spin-rust-hello:v0.22.0' locally
v0.22.0: Pulling from spinframework/containerd-shim-spin/examples/spin-rust-hello
25d3662ed73c: Pull complete
891ca7c43ac7: Pull complete
25b36397191d: Pull complete
Digest: sha256:ad8874b8560cc5e3bf17367bb534e404b16dc7fe097975f34cf43b1f96b9e015
Status: Downloaded newer image for ghcr.io/spinframework/containerd-shim-spin/examples/spin-rust-hello:v0.22.0
# HANGING HERE
Notice that # HANGING HERE? That's the frustrating part. The Docker command pulls the image, it seems to process things up to a certain point, but then the application itself never starts. The prompt just sits there, waiting, indefinitely. This means your WASM applications, which are designed for quick startup and execution, are being held hostage. The containerd-shim-spin process, which is responsible for initializing and managing the WASM runtime for your Spin app, seems to be getting stuck in some sort of loop or waiting state that never resolves. This Spin app not starting on Docker v28 issue is critical because it breaks the fundamental promise of portability and ease of use that Docker offers. Developers expecting their Spin apps to just run are finding themselves blocked. The problem isn't with the Spin application code itself, nor is it necessarily with the WASM module, but rather with the interaction layer between Docker, containerd, and the specific io.containerd.spin.v2 shim. This highlights a subtle but significant change in how Docker (and by extension, containerd) handles runtime interactions with custom shims in its newer versions. Our previous testing primarily focused on k3d (Kubernetes in Docker), which might abstract away some of these lower-level containerd interactions that are now exposed when running directly with docker run on the host. This disparity between testing environments and real-world Docker usage is precisely what we're aiming to address with more robust Docker-in-Docker testing, which will help us pinpoint the exact point of failure within this intricate system.
Reproducing the Issue: Hands-On Steps for Debugging
To really get a handle on this containerd-shim-spin hangs on Docker v28+ problem, the best thing we can do is roll up our sleeves and try to reproduce the Spin Docker hang ourselves. Luckily, there's a specific branch and a testing script that's been set up to make this process as straightforward as possible. This approach, using Docker-in-Docker testing, is absolutely crucial for debugging these types of environment-specific issues, as it allows us to test different Docker versions without mucking up our primary dev environment. So, let's dive into the steps. First things first, you'll need to clone the specific GitHub pull request (PR) that includes the necessary test infrastructure. This PR (https://github.com/spinframework/containerd-shim-spin/pull/377) introduces a dedicated script for testing on Docker, which is super helpful for isolating this problem. Open up your terminal, folks, and let's get cloning:
git clone -b docker-integration-test git@github.com:kate-goldenring/containerd-shim-spin.git
Once that's done, you'll want to navigate into the specific test directory within the cloned repository. This is where all the magic happens for our Docker integration tests:
cd tests/docker-integration-test
Now, here's where we demonstrate the difference between working and non-working Docker versions. We're going to use environment variables to control both the containerd-shim-spin version we're testing and the Docker version within our Docker-in-Docker setup. This setup is key for getting reliable and repeatable results for this Docker compatibility issue. First, let's try running a Spin app with a shim version (v0.22.0) on what should be a working Docker version, say v27 (even though the example below uses 29 for the failed attempt, let's imagine we're stepping back to 27 for success – the key is the behavior):
# Example for successful execution on a compatible Docker version (like v27)
CONTAINERD_SPIN_SHIM_VERSION=v0.22.0 DOCKER_VERSION=27 ./docker-integration-test.sh
When you run this command with DOCKER_VERSION=27 (or any version known to work), you should see the Spin application start up successfully, outputting the