Devbox: Turn a 1000+ Container Script into <10 Lines

Devbox: Turn a 1000+ Container Script into <10 Lines

TLDR; At Jetpack.io, our development container initialization script used to be like this:

image: ubuntu:22.04
...
# Install Go.
wget --progress=bar:force:noscroll https://golang.org/dl/go1.19.linux-amd64.tar.gz -O go.tar.gz
sudo tar -C /usr/local -xzf go.tar.gz
echo "export PATH=$PATH:/usr/local/go/bin" >> ~/.zshrc
rm go.tar.gz
# Add symlinks for the binaries to be accessible in non-login sessions.
sudo ln -s /usr/local/go/bin/go /usr/local/bin/go
sudo ln -s /usr/local/go/bin/gofmt /usr/local/bin/gofmt

View 516 more lines...

Now with Devbox, it looks like this:

{
  "packages": ["go_1_19", "nodejs-16_x", "nodePackages.yalc", "poetry", "python310", "buf", "sops", "kubectl"],
}

Our struggles with code onboarding at Jetpack.io

At Jetpack.io, our main repository is a monorepo with 10+ different projects in it. We build tools and frameworks that make deploying to Kubernetes easy and accessible for all developers, and building these frameworks requires us to support many different languages and technologies. Together they need C, Go 1.19, Node.js 16, Python3.10, Protobuf, Kubectl, Sops, AWS, GCP, Terraform, and diverse toolings and credentials. Setting this up correctly from scratch can be very challenging.

Solution 1: Mount our monorepo in a local Docker container

To make this setup more manageable, and to ensure all our team members have the same experience, we turned to Docker for rescue. We created a single, unified dev environment in a Docker image and mounted the entire monorepo to the locally running container. In the process, for each language toolchain we had to:

  1. Get the right Ubuntu base image
  2. Install the proper packages with apt-get
  3. Curl the tar files, untar them, and install them in the right location
  4. Setup variables and symlinks
  5. Clean up the garbage

Creating this container wasn’t easy, but it did succeed in simplifying our initial setup.

Result: Our laptops can't handle such a massive Docker container.

Unfortunately, our development Docker container quickly bloated up to 9 GB, and spinning up a new environment, or upgrading a core package, could take hours. CPU usage sometimes goes over 100%, and on a hot day, our laptops just quit. Without proper clean-ups, memory usage can quickly bump to 30+ GB. The complaints from our team keep coming, but we struggled to find better alternatives.

Solution 2: Set up containerless shell environments with Devbox.

This pain is what inspired us to build Devbox. Devbox gives us isolated development shell environments without the headache of a Docker container. With a simple devbox.json like the following, our team can do code onboarding in no time:

{
  "packages": ["buf", "go_1_19", "nodejs-16_x", "nodePackages.yalc", "poetry", "python310", "sops", "kubectl"],
  "shell": {
    "init_hook": [
      "source ./devtools/scripts/shell.sh"
    ]
  }
}

With packages, all team members have the same tools, languages, and dependencies. You no longer need to deal with apt-get, tarball files, symlinks, or garbage clean-ups. With shell init_hook, each Devbox shell setup the proper credentials and environment variables for a session:

#!/bin/bash
 
# Script of commands to run when an interactive devbox shell is started.
 
if [ -z ${AWS_SECRET_ACCESS_KEY+1} ]; then
 echo $AWS_SECRET_ACCESS_KEY
 echo "You must provide AWS credentials in AWS_SECRET_ACCESS_KEY."
fi

Result: Lightweight, repeatable dev environments (and cooler laptops)


After moving from a Docker container to the containerless Devbox shell, we freed up CPU usage and multiple Gigabytes of memory on our computers. The start-up time went from a couple of hours to a few seconds. Our init script went from 1000+ lines down to 9. Now we had the best of both worlds: developers could get a reliable dev environment up and running in one step, without the performance costs and complexity of using Docker containers.

Try Devbox Today!

To help other teams that may be struggling with their dev environments, we open-sourced Devbox and released it for free! You can head to GitHub for the installation guide and check the source code. Visit jetpack.io/devbox to learn more. Have questions? Join the Discord community