Boot 2.5: Slow is smooth, smooth is fast

If you like Clojure, you'll like Boot. It's a functional build tool, written in Clojure, and 2.5 is our finest release yet. Learn more and get started

Boot logo

There is an old adage: slow is smooth, smooth is fast. The idea is, if you want to be really fast at anything, don't be reckless. Instead, be careful and deliberate. Reckless gains are quickly lost. Real speed comes from mastery, and mastery is a slow process.

Since we released Boot 2.0 back in May we've proceeded carefully and deliberately. As a community, we've been helping eachother use it every day to build, deploy, and operate Clojure and ClojureScript applications.

We've also put careful thought into ways we could improve Boot's quality. We've had enough experience with it now to know which parts were ready to be made simpler and faster without conceding to any other concern. With this 2.5 release, we are delighted to share these improvements.

For a concise list of changes in 2.5, see CHANGES.md

Speedier sift

Boot's execution model is roughly: thread an immutable value representing a filesystem through a stack of task functions, each of which takes and returns a FileSet.

Tasks are easy to write — they're just Clojure functions — and are the primary means of extending Boot to meet a project's automation needs. However, Boot ships with a number of builtin tasks for very common scenarios.

One such task, sift, can be used to move and filter files in the FileSet by path or regex.

You can see all of Boot's tasks by running boot -h, and learn more about sift specifically by running boot sift -h.

sift has been completely rewritten and is now orders of magnitude faster. If your build uses it, your build probably just got much faster.

Lightning uber

Another builtin, uber, also got significantly faster in Boot 2.5.

"Uber jars/wars" or "fat jars" are a convention for packaging application code with all dependencies, typically to create standalone executable jars or to deploy web applications to application containers like Tomcat or WildFly.

Historically, in both Boot and Leiningen creating an uber jar or war can be painfully slow. This was usually for two reasons:

  1. Clojure files needed to be AOT-compiled, as containers require a concrete .class file to instantiate the application
  2. Dependency jars need to all be exploded, merged, and zipped with application code on every build

Boot has solved reason #1 since 2.0 with the web task. Instead of AOT-ing Clojure code, one need only specify a namespaced symbol of a Ring-compliant handler to the web task. A web.xml is automatically generated, and a pre-compiled Java proxy class is added to the FileSet. This class, a Servlet subclass, then uses Clojure's Java API to resolve and invoke the Ring application.

Reason #2 can be mitigated somewhat, at least when deploying to servlet containers, by bundling dependency jars into the application jar without exploding them. Then, at runtime, the container adds the "jars in the jar" at WEB-INF/lib to the application's classpath.

The --as-jars option of the uber task bundles dependencies in WEB-INF/lib

Unfortunately, not every deployment destination supports the WEB-INF/lib jar-smuggling scheme. Standalone jars and various map-reduce frameworks are common examples.

These particular scenarios are what Boot 2.5's uber task improvements address. Not only is Boot's uber faster than it was before, it's on par with Leiningen, and maybe a little faster. But the weirdest and coolest part? If you watch a pipeline with uber and run the build a few times, you'll notice that after each run, uber gets faster.

The trick to it is immutability via structural sharing and the inherent effectiveness of caching in systems oriented around immutable values and defined by transitions between them.

Under the hood, Boot maintains immutable file values by managing a system of hard links in a content-addressed structural sharing scheme. Boot uses this system, and caching, to save exploded dependency jars and only do the work once. It also tracks source changes and calculates the difference these changes will make in the uberjar. Finally, it constructs a patch and modifies an existing copy of the jar in place, finally presenting it as a new value.

Again, no changes are necessary in user code to take advantage of this speed gain. Just upgrade to Boot 2.5 and enjoy!

Transparent pods

Pods are Boot's practical "unit of classpath" and are used to mitigate dependency hell by providing an easy-to-use means of classpath isolation. Pods are Clojure runtimes, in which any Clojure code can be run, and to which any dependencies or sources can be added.

Pods improve the composability of code built on Boot, because such code can be freely combined in a JVM without concern for how its respective dependencies will interact.

As of 2.5, pod environments have a new degree of visibility. Every pod has a name, and all pods can be listed with the show task. show can also be used to diagnose dependency problems in individual pods. In addition, a REPL can be started in any pod, with the new --pod option to repl

Target is a task

We often joke that "Boot is not a build tool", because we try actively not to do many of the things build tools are expected to do. We'd rather reserve these activities to the user, who is likely a perfectly competent programmer his or herself, and who has a much deeper understanding of the problem at hand than us.

One problem we definitely didn't need to solve unilaterally, and shouldn't have even tried to, was what to do with the FileSet at the end of the build.

When we first made Boot, it was hard to imagine much about what a build tool would do if it didn't eventually dump files in some directory. So, that's what we made Boot do, and built in the logic that synchronizes the last FileSet value to target/ when a task pipeline completes.

Now we know that this is assuming too much. What if the user wants to synchronize the last value to S3? Or to multiple places? Or not at all?

Boot 2.5 corrects us by introducing a target task, which can be used to synchronize the FileSet to disk anywhere in the task pipeline (or not at all).

Boot will still synchronize to target/ on pipeline completion, but this behavior can be disabled by setting BOOT_EMIT_TARGET to no. If automatic synchronization is on, the target directory can still be configured with set-env! like (set-env! :target "other-target" ... )

Big thanks to community & contributors

Boot wouldn't have gotten this far without its amazing community of users, developers, and supporters, and we'd like to thank in particular those who have contributed code so far. They are:

  • Alexander Solovyov
  • Bozhidar Batsov
  • Christian Romney
  • Chris Truter
  • Craig McDaniel
  • Daemian Mack
  • Daniel Szmulewicz
  • Dave Dixon
  • Devin Walters
  • Dylan Butman
  • Elango Cheran
  • Fabio
  • Jeremy Heiler
  • John Guidry
  • Josh Johnson
  • Joshua Davey
  • Jozef Wagner
  • Juho Teperi
  • jumblerg
  • kul
  • Lake Denman
  • Léon Talbot
  • Lucas Leblow
  • Martin Klepsch
  • Murphy McMahon
  • Nick Ogden
  • Norbert Wójtowicz
  • Paul deGrandis
  • Piotr Krawiec
  • Ragnar Dahlén
  • Ralf Schmitt
  • Ryan Neufeld
  • Sergey Lymar
  • serzh
  • Steven Degutis
  • Toby Crawley

Like the article?

Get notified of future blog posts. Don't worry - we won't make it hard to get to inbox zero: no more than 2 e-mails a month. We promise.