Running Docker in Incus Containers

This guide demonstrates how to properly set up and run Docker inside an Incus container on Debian 12. It builds upon our previous article “Installing Incus on Debian 12 with ZFS Storage” and focuses on the secure deployment of Docker in a containerized environment.

Prerequisites

• Incus installed and configured with ZFS storage (refer to our previous article)
• Debian 12 (Bookworm) host system
• Root or sudo access
• ZFS 2.2 or later (for proper overlay support)

Security Considerations

Running containers within containers (nesting) requires specific security configurations. This guide follows the principle of least privilege, enabling only the necessary capabilities while maintaining a secure environment.

Step-by-Step Configuration

1. Creating the Incus Container

Launch a Debian 12 container with the required security settings:

incus launch images:debian/12 docker \
  -c security.nesting=true \
  -c security.syscalls.intercept.mknod=true \
  -c security.syscalls.intercept.setxattr=true

These settings enable:

• security.nesting: Required for running containers inside the Incus container
• security.syscalls.intercept.mknod: Allows creation of device nodes
• security.syscalls.intercept.setxattr: Enables setting extended attributes

2. Network Configuration

Set a static IP address for the container (adjust the IP according to your network):

incus config device override docker eth0 ipv4.address=10.0.10.10
incus restart docker

3. Installing Docker

Access the container shell:

incus shell docker

Inside the container, install Docker and its dependencies:

# Update package lists
apt update

# Install prerequisites
apt install -y \
    apt-transport-https \
    ca-certificates \
    curl \
    gnupg

# Add Docker's official GPG key
curl -fsSL https://download.docker.com/linux/debian/gpg | gpg --dearmor -o /etc/apt/keyrings/docker.gpg

# Add Docker repository
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] \
    https://download.docker.com/linux/debian bookworm stable" | \
    tee /etc/apt/sources.list.d/docker.list > /dev/null

# Update package lists again
apt update

# Install Docker packages
apt install -y \
    docker-ce \
    docker-ce-cli \
    containerd.io \
    docker-buildx-plugin \
    docker-compose-plugin

4. Verifying the Installation

Still inside the container, verify that Docker is running correctly:

# Check Docker status
systemctl status docker

# Verify Docker is functional
docker run hello-world

5. Storage Configuration

Docker automatically uses the overlay2 storage driver, which is optimal for our setup with ZFS 2.2. You can verify this with:

docker info | grep "Storage Driver"

Resource Management (Optional)

You can set resource limits for your Docker container if needed:

# Set memory limit
incus config set docker limits.memory=4GB

# Set CPU limit
incus config set docker limits.cpu=2

# Set storage quota
incus config set docker limits.disk.priority=10

Best Practices and Security Considerations

1. Unprivileged Containers: Keep your containers unprivileged. Never use security.privileged=true unless you have a specific requirement that absolutely necessitates it.
2. Network Security: Consider implementing network policies to restrict container communication if running in a production environment.
3. Regular Updates: Keep both the Incus container and Docker installation updated:

   apt update && apt upgrade

4. Resource Monitoring: Regularly monitor resource usage, especially in production environments:

   incus info docker
   docker stats

Troubleshooting

Common Issues and Solutions

1. Docker daemon fails to start

• Check system logs: journalctl -xe
• Verify required kernel modules: lsmod | grep overlay

1. Network connectivity issues

• Verify container’s network settings: incus config device show docker eth0
• Check host firewall rules

1. Storage problems

• Verify ZFS pool status: zpool status
• Check Docker daemon logs: journalctl -u docker.service

Conclusion

This setup provides a secure and efficient way to run Docker containers within Incus containers. By following the principle of least privilege and using only necessary security configurations, we maintain a robust security posture while ensuring full Docker functionality.

Remember to regularly check for updates to both Incus and Docker, and always test your configurations in a development environment before deploying to production.