The State of Javascript Package Management

One notable thing about JavaScript in the browser compared to other popular languages is the almost lack of a standard library. Even some of the most common tasks can require a fair chunk of code. You can write it all yourself, but crafting cross-browser compatible code remains a minefield. If you don’t want to do it yourself, what can you do?

The “traditional” way in JavaScript was to copy dubious fragments of code from forums and Q and A sites into your project and hope for the best. Then, later, came the big libraries. Now if you want a cross-browser method of changing the colour of all of your links you can download an almost 10,000 line library file that included, somewhere in the middle, the functionality that you wanted. And you could manually download a fresh copy of a probably slightly different version each time you started a new project.

Clearly, this situation is not ideal, which is why we have seen several package managers spring up.

What is a package manager anyway?

I guess before we go any further we should talk about what a package manager is meant to achieve in the general case. Most package management systems are trying to solve a set of related problems.

The first, and most obvious, is to provide a reliable, standard mechanism for installing software created by third-parties. By specifying a convention (or possibly some very strict rules) about the structure of the packages the end-user only has to learn how the package manager works, not the specific installation procedure of the package they are trying to install. Things like where the software is installed, how it is built or compiled, or how it integrates with the rest of the system are all taken care of by the package manager.

Secondly, package managers provide dependency management. If package foo requires libraries bar and baz in order to function then the package manager will know to install bar and baz as well when asked to install foo. Most  can also deal with specific version requirements for dependencies, attempting to satisfy the tree of dependencies in such a way that compatible versions of packages are installed together.

Another common feature is to allow you to keep up to date with new versions of packages as they are released. In some cases you don’t want to upgrade – you don’t need those new features, so why risk breaking your project? Other times you want the latest and greatest so that you have all of the exciting features or essential bug fixes. A good package manager allows you to choose your own path.

A package manager should also allow you to discover new packages. If you want to solve a problem that you are pretty sure has been solved before, you can jump into your package manager’s search and see a list of possible solutions to check out.

The last features we’ll talk about here are for the people who create the packages in the first place. Some, but certainly not all, package managers will come with tools to help you create the package ready for distribution, and all of these systems need a way for the package author to put their work somewhere it can be discovered and used by the community.

A JavaScript package management tour

There are currently six contenders in the JavaScript package management arena – npm, bower, Ender, volo, component and jam. npm comes bundled with recent versions of Node.js, while the others are installable via npm with npm install <package> -g where the package name is bower, ender, volo, component or jamjs.

npm

According to the FAQ, npm does not stand for Node Package Manager, though it is certainly a good description. The npm repository contains many JavaScript libraries and quite a few command line tools. There are packages available through npm to be used with browser based projects but most are specific to Node on the server. npm is important for browser development anyway, though, as all of the other browser-oriented tools mentioned below are themselves npm packages.

There are two ways to install packages with npm – global and local. Usually you would install tools you want to use from the command line globally and libraries you want to require() locally. Packages are installed with the npm install command. When you locally install a package npm will place the package files in a node_modules/<package> folder in your project route. Packages installed this way will be accessible in your node programs with require(). Globally installed command line tools will usually be put somewhere in your PATH (like /usr/local/bin on Linux).

The npm repository contains almost 20,000 packages which can be searched either on the command line with npm search or via the website. An npm module is defined by a file named package.json in the project root. This file has details like the name, description and version, which file is loaded when you require() the package, and dependencies. Dependencies are split into “dependencies” and “devDependencies” so that things that are only needed to build or test your package when developing it wont be downloaded by your end users.

When your package is ready for public consumption you can use npm publish to push it to the repository. Names are taken on a first-come, first-served basis. Only someone registered as an owner of a package can upload new versions but this is easily managed with the command line tools if you wish to add someone or transfer ownership. You need to have an npm account to publish packages.

bower

bower was created by Twitter for use on their internal web projects. A bower package can contain any assets a website might need, such as CSS, JavaScript or images.

This is the simplest of the tools discussed here as it provides a method for downloading packages and their dependencies and no more. This also makes it the most liberal, as any existing package can be put into the bower repository without modification and without adding constraints on the user.

The bower repository has 698 components which can be searched (by name only) using bower search or on a third-party search site. With bower a ‘package’ is actually a git repository, so the bower repository is only a list mapping names to public git endpoint urls. bower install <name> fetches the git repository into a components/<name> directory within the project root. bower has zero security or identity management. Anyone can publish a package on bower with bower register <name> <git endpoint>. Repository management is manual, which is to say that you have to message one of the admins if you want to remove, rename or re-register a component.

The package itself is defined by a file called component.json in the root of the git repository. Only the name, version and dependencies fields actually do anything. When retrieving packages from git endpoints bower uses tag names in the repository as valid version identifiers, so you must tag at least one revision before your package can be consumed.

Ender

Not just a package manager, Ender actually builds you a customised library to use from small, modular packages. The philosophy behind Ender is to give you only the code that you actually need for your project, and to do so in such a way that a module could be replaced by something roughly equivalent with the minimum of fuss.

To start a project using Ender you use ender build <name>[, <name>, …] to create a custom library using the given packages. The created library exposes a global variable $ with properties defined by the component packages. Once your library has been started you use ender add and ender remove to manage the components that it provides. The library is written into ender.js and ender.min.js.

Ender piggybacks off of the npm registry, with each Ender package being an npm package. To signify that an npm module is compatible you add the ‘ender’ keyword to package.json. ender search will search npm, giving higher weight to packages with the keyword. You can see all packages marked for Ender using the npm keyword search on the website. There are currently 212 registered.

As well as adding ‘ender’ to your keyword list you can also optionally add an ender property to your package.json file that specifies a ‘bridge’ file which allows you to tightly integrate with the Ender API. Otherwise, creating and registering packages is the same as for npm.

volo

volo can be used much like bower to simply install the contents of a package, but it also includes a project automation/build system and can automatically turn installed packages into AMD modules, amongst other things.

You can bootstrap a project by using volo create <project name> which will create a directory called ‘<project name>’ containing the default template, though you can specify another template to use. You don’t have to use a template to use volo, however. If you use volo add <name> to add a package to your project then volo will use a set of rules to determine where to extract the package.

volo actually uses Github as its backend – a package is a Github repository and volo add will use the Github search API to find packages if you only provide a name or keyword. You can specify a full repository name with volo add <username>/<project name>, e.g. volo add jquery/jquery.

There is no special file for defining a package for volo, and publishing is simply a matter of creating the repository on Github. There is, however, a set of guidelines that will make your project easier to find and consume. Because every browser-oriented JavaScript repository on Github is technically a volo package, volo almost certainly has the highest available package count though exactly how many is unknown.

component

component is based on CommonJS style modules (the module style that Node uses) so like with Ender you must commit your whole project to this style. You start a project with component create <project name>  which will ask you several questions before creating a new directory under the current path with the given name. From inside the project root you can use component install <name> to install dependencies. You then need to use component build to create the file that you will load in the browser.

Like volo, component packages are Github repositories so dependency names are in the format <username>/<project name>. You can “publish” your package by adding the details of your repository to the wiki. You specify the details of your package in a component.json which is not compatible with the bower config file with the same name.

You can search the available components with component search or view the list on the wiki. The list currently contains 431 entries.

Jam

Jam is a tool for managing AMD packages only, and comes with a modified version of require.js, including a “compiler” based on the r.js tool.

Packages are installed with jam install <name> and get put in jam/<name> by default. To actually consume the packages, though, you use the modified require.js that comes with Jam. This file is updated when you add and remove packages to your application.

Jam has its own repository and you need an account to manage packages. The repository contains 279 packages which can be searched with the web interface.or with jam search. The sort order of the results is a little strange though; when searching for ‘jquery’, jQuery itself is the very last result.

Jam uses a package.json file which is compatible with npm, with some jam specific options in the jam property. You can then share your package by typing jam publish from the root of your project.

Enough facts, here are some opinions

So, six solutions to the same problem. One for node modules and command line tools, and five for the browser. Its definitely a great thing that these tools exist, and I highly recommend using one of them, even if you only use them as a convenient way of downloading jQuery when you start a project.

On the other hand, I can’t imagine that five different browser script managers will survive for long if they continue to overlap so much – both with each other and with other available tools. The command line interfaces are all very similar, as are the configuration options. If you want project automation then grunt has a much bigger community around it than Ender, Jam or volo, and Yeoman looks like it is going to clean up in the bootstrapping race. The repositories contain mostly the same packages and repository management is not likely to be the killer feature.

In an ideal world there would be two or maybe three options but they would be providing very different things. Instead, Ender and component are solving the same problem – small, decoupled components to replace the monolithic libraries. And grunt plus bower can do everything that Jam and volo do.

Package authors either have to put in the extra effort to maintain compatibility with all of the different systems or you end up with no system having access to every package. If a package author decides to create a component module, well then tough. You can’t use that with the other systems; especially not with bower as the component.json files conflict with each other.

Of course, it’s not up to me to declare a winner, and there isn’t an obvious candidate. I don’t get to say “you should all just use bower” or whatever, so it will probably take a while to shake out. But if I may make a prediction for 2013 it is that a lot of time will be wasted making each tool a little bit more like the others, rather than trying to differentiate themselves.