This Week #2: Templates, Secrets, OS Packages

This week was planning week: every 3 weeks we take some time to plan what are we going to work on next. I focused on two things this week.

Daniel

This week I focused on planning around two areas:

1. Authentication: How should authentication of our products evolve to support teams? How can we structure our authentication so that it can be re-used across our products?
2. Secrets: With devbox we are trying to make it possible to recreate development environments. Some times those environments require secrets. What's our approach to secrets?

We have some projects coming up that address both of these. Hoping to ship them soon!

Lucille

I spent this week heads down on building devbox templates - it is finally done and live! Check it out at https://jetpack.io/devbox/templates . By making the templates more discoverable, we hope developers can have an easier time bootstrapping and setting up their own devbox projects.

In addition to finishing up devbox templates, I spent some time researching auth0 alternatives such as workOS and Stytch. It’s time to level up our login flow.

Mohsen

This week my focus was split between two things:

1. Finishing up the Devbox cloud templates changes so that we can have user-submitted templates and
2. Planning the details of some exciting new features for Devbox.

On the Devbox cloud templates side, we’re pretty much finished. Just smaller UI fixes that’s remained.

On the planning side, without spoiling too much, we achieved consensus on what to work on and how to implement the integration on the architectural side of things. This week we’ll get to the implementation and hash out the details there.

Aside from this, I tried to keep up with questions and issues that was posted in our Discord server. Also, we added a new flag to Devbox CLI called --pure a while back but embarrassingly, I forgot to add it to our docs page. I updated the docs page in a PR so hopefully that won’t be remaining as a problem for long. By the time anyone’s reading this, chances are the docs page is already updated.

Rodrigo

This week was a bit slow for me. I continued work on adding metrics for user metering in Devbox Cloud, which is almost done.

I had some issues when upgrading nix in order to continue work on the remove nixpkgs feature. Once done, I found two more bugs in the way that Devbox manages the nix profile when installing packages from store paths (note: normally, packages are installed with a flake reference, but in order to break the dependency on nixpkgs, we will install from cached store paths in the future). I haven't yet implemented fixes for these bugs, but I understand the root causes and will work on fixes next week.

Savil

Last week, my work was very focussed on a single project: enabling per-OS packages. This week's work was scattered over a number of areas.

To start with: for a couple of months now., we've been working on a perf optimization that reduces the likelihood of users needing to download a copy of the nixpkgs registry. This should save users about 30-40 seconds, whenever they encounter this. I'll write more details when the feature is ready (hopefully next week's update). This week's work involved adding some concurrency code to determine if users can enter this "fast path". We don't write much concurrency code in the Devbox CLI, so I was glad to have Greg review my code carefully and point out my overlooking some shared data-structures that needed mutex locks.

This feature also will only work for users on nix 2.17+ (which is the latest nix as of this writing). So, I modified our CICD tests to work for nix < 2.17 and 2.17 so that both fast-paths and slow-paths will be tested going forward.

Strangely, this led to our tests timing out for nix 2.17. The cause wasn't the feature code because that is turned off via a feature-flag. After some painful test bisecting, I narrowed it down to a yarn package rebuilding nodejs. This would take a long time (over 20 minutes) and tests would time out. I'm still not sure why this was only happening on nix 2.17. Pinning yarn in the test's devbox.lock file to a nixpkgs commit hash that we know has a pre-built binary seemed to fix it. Incidentally, ensuring that devbox update pins packages to only those nixpkgs commit hashes that have prebuilt binaries is what Greg is about start work on. Hopefully, he can solve it before our users encounter this issue!

Lastly, I was feeling a bit disheartened mid-week due to the poor ergonomics of the concurrency code we needed to check in, and these tests timing out. So, I spent about 30 minutes on a fun task to modernize the devbox info <package> command. This had been added prior to Devbox implementing Versioned Packages , and its output was minimal: package name and a version that was a bit inscrutable. I changed the internals to query the search API and it now displays the latest version available about this package, as well as a description:

> devbox info go
go 1.20.6
The Go Programming language

Daniel had a good suggestion during our Demo Day to change it to something like:

> devbox info go
go
The Go Programming Language
installed version: 1.20.3         // inspect the devbox.lock file
latest available version: 1.20.6  // query Devbox's Search API

I'm hoping to make this change next week.