Franklin Webber[1] went on a quest to find out whether
utility code was better or worse than core code in open source
projects. Utility code is defined as anything necessary but not directly related to the application’s
mission. Franklin found that overall quality of utility code was higher, perhaps because this code changes less. He also found a lot of repetition among projects, many
people doing the same thing. A function to find the home directory
in several possible env vars, for instance. HashWithIndifferentAccess,
which doesn’t care whether you pass the key as string or symbol (keyword
in Clojure). He asks, why not break these out into tiny little gems?
Dependency management is an unsolved problem in our industry. We want
single-responsibility, and we want re-use. Reuse means separately
released units. But then when we compile those into one unit, transitive
dependencies conflict. Only one version of a particular library can make it into the final executable. In the example below, the Pasta library might not be compatible with the v2.1.0 of Stir, but that’s what it’ll be calling after it’s compiled into Dinner.
Programs like WebLogic went to crazy ends to provide different
classloaders, so that they could run apps in the same process without
this dependency conflict. That’s a mess!
We could say all libraries should all be backwards compatible. Compatibility is great thing, but
it’s also limiting. It prevents API improvements, and makes code ugly.
There’s more than API concerns as well: if the home-directory-finding
function adds a new place to look, its behavior can change, surprising
existing systems. We want stability and progress, in different places.
With these itty-bitty libraries that Franklin proposes we break out,
compatibility is a problem. So instead of dependency hell, we have chosen
duplication, copying them into various projects.
What if, for little ones, we could inline the dependency? Like
#include in C: copy in the version we want, and link whatever calls
it to that specific implementation. Then other code could depend on
other versions, inlining those separately. These transitive dependencies
would then not be available at the top level.
Seems like this could help with dependency hell. We would then be free
to incorporate tiny utility libraries, individually, without concern
about conflicting versions later.
I wonder if it’s something that could be built into bundler, or leinengen?
This isn’t the right solution for all our dependency problems: not large libraries, and not external-connection clients. Yet if we had this choice, if a library could flag itself as suitable for inlining, then microlibraries become practical, and that could be good practice.
——-
[1] Talk from Steel City Ruby 2014
[2] Angry face: