Ruben Verborgh

The devil is in the details,
but the demons are in the semantics.

JavaScript module loaders: necessary evil?

The fear of global state makes us perhaps unnecessarily run towards AMD.

Modules—we need them when projects go large, and to reuse work from others. Every programming language offers a way to partition code in reusable chunks. Some of them, such as C and Ruby, provide explicit mechanisms for this. JavaScript, on the other hand, leaves the modularization to the programmer, and the Asynchronous Module Definition (AMD) API is one way to achieve this. However, does AMD really offer the final solution to JavaScript modularization?

JavaScript doesn’t come with an out-of-the-box way to modularize code, except for, distributing code over different files. Some server-side JavaScript environments, such as Node, provide a modular system, which is often an implementation of CommonJS Modules. Its paradigm consists of a require function to load modules and an export function to define them. This mechanism is quite flexible, since even circular dependencies are possible. However, it makes one important assumption: that modules can be loaded synchronously.

On the client-side, in the browser, this assumption no longer holds: every individual JavaScript file has to be fetched over the network, and this takes too much time to wait. If we require a module, we want it immediately. However, somehow, modules must be able to depend on one another. Traditionally, this is done by attaching modules to the root object (window), equivalent to global variables in other programming languages. For example, you first load the jQuery script, and then a script that depends on the window.jQuery (or $) module.

Unfortunately, when you are working on a large project with many modules, this approach becomes cumbersome. The Asynchronous Module Definition (AMD) API is a mechanism that allows client scripts to indicate their dependencies. You can then use a module loader that asynchronously downloads the required modules in the right order and binds them together. The question is: does our project need AMD?

Should we attach modules directly to the window root object, or should we use an AMD loader?

How AMD module loaders work

Suppose we’re building a Markdown visualizer, a module which depends on the jQuery and markdown modules. To create this module In AMD, we call the define method, provided by the module loader, with three parameters:

  1. the identifier of the module (optional)
  2. the dependencies of the module (optional)
  3. the factory function that returns the module

For example, our markdownViz would be defined like this:

define('markdownViz', ['jQuery', 'markdown'], function ($, markdown) {
  var markdownViz;
  /* add functionality to markdownViz here */
  return markdownViz;
}

To then use this module in a script file main.js, we use the require method that takes the dependencies of the script and the script itself:

require(['markdownViz', 'jQuery'], function (mdViz, $) {
  mdViz.init($('#textinput'));
}

This main script then gets started by a loader such as RequireJS.

<script data-main="scripts/main" src="scripts/require.js"></script>

Then the following process happens:

  1. The loader fetches and executes main.js.
  2. require stores the main function and the identifiers of its dependencies.
  3. The loader fetches the scripts in its dependency list: markdownViz and jQuery.
  4. The jQuery module has no dependencies, so the loader executes and stores it.
  5. The markdownViz module depends on markdown, so the loader fetches it.
  6. The markdown module has no dependencies, so the loader executes and stores it.
  7. The markdownViz module is executed with jQuery and markdown arguments.
  8. The main function is executed with markdownViz and jQuery.

As you can see, the loader only executes the factory function of a module if all of its dependencies have been loaded, passing them in to the module’s factory function.

How the window approach would have worked

If we had used the used the window approach and thus defined markdownViz as a global variable, it would look like this:

window.markdownViz = {
  /* add functionality to markdownViz here */
};

The script and its dependencies would be loaded in the HTML source:

<script src="scripts/jQuery.js"></script>
<script src="scripts/markdown.js"></script>
<script src="scripts/markdownViz.js"></script>
<script src="scripts/main.js"></script>

Wow, this is remarkably more… simple? In fact, since the loader script itself isn’t necessary anymore, the window approach only needs 4 scripts instead of 5.
So where’s the gain? Why would we want an AMD loader instead of global variables?

The impact on testability appears minimal

The AMD philosophy seems to be partly founded on the fear of introducing global state. We of course all agree that global state is bad. It makes it difficult to test code in isolation and to reuse a module in a different context. The Law of Demeter, also known as the Principle of Least Knowledge, tells us to ask—not look—for things. According to Miško Hevery, failure to comply is a top-3 testability issue—global state itself is on the 4th spot. However, JavaScript is a dynamic language and therefore, the concept of global state is different from that of statically typed languages in which its consequences are likely more severe. Interestingly, people have argued that AMD trades global variables for global identifiers. Does that smell like global state, too?

Let’s therefore look at AMD and global variables from a testability perspective. Suppose we want to test markdownViz in isolation, thus stubbing or mocking jQuery and markdown. With AMD, we test like this:

function testA() {
  require.define('jQuery',   { /* mock jQuery   */ });
  require.define('markdown', { /* mock markdown */ });
  require(['markdownViz'], function (markdownViz) {
    /* perform test A on markdownViz here */
  };
}

With global state, we have to test like as follows. Here, fetch gets the source code of markdownViz with XMLHttpRequest. Also, note the assumption that markdownViz only alters one global variable.

function testA() {
  window.jQuery =   { /* mock jQuery   */ };
  window.markdown = { /* mock markdown */ };
  fetch('scripts/markdownViz.js', function (source) {
    eval(source);
    /* perform test A on markdownViz here */
    delete window.markdownViz;
  });
}

Of course, there are various (maybe shorter) ways to write the above functions. However, both situations can easily be abstracted with a generic test function:

test({ jQuery:   { /* mock jQuery   */ },
       markdown: { /* mock markdown */ }},
       function (markdownViz) {
         /* perform test A on markdownViz here */
       });

So, as far as testability is concerned, the difference seems minimal, and can be hidden, since we want to avoid repetition anyway.

It all comes down to your build process

For both approaches, we’ve not discussed the build process yet. After all, it wouldn’t be a good idea to deploy a Web application that needs to fetch 4 or 5 scripts. In the AMD case, we can run the optimizer, which combines the scripts in one file and minifies them. A tiny replacement loader such as almond can then be used at runtime, making the AMD footprint minimal (but not zero). In the window case, you have to concatenate your ordered scripts, minify them, and change the 4 script tags into one that points to your final script. This script doesn’t have any overhead.

Both cases can easily be incorporated in your build process. For the AMD case, the tool already exists. For the window case, you’ll have to write your own, but that shouldn’t be too hard: just read the script tags in order and process them.

So this makes me think. If you need a build process anyway, how about this one?

  • Place a scripts.json configuration file in your project. It can either just contain an array where the script files are in the right order (manual dependency tracking) or an array with script files and their dependencies (automatic dependency tracking).

  • Your HTML generator calls a script generator that reads this configuration file (and works out the right order, in case of automatic dependency tracking).

  • In development mode, the script generator returns a series of script tags. For deployment, it returns a minified script.

AMD is not necessary anymore to track dependencies here, since they are implicitly or explicitly indicated in the configuration file. It’s then up to you whether you use global variables, or wrap the whole script in one big anonymous function. The only impossibility seems circular dependencies, but they’re a bad idea anyway. With a little effort, and only minor overhead, we could even write our scripts as CommonJS Modules, achieving Node-compatible modularization.

To AMD or not to AMD?

In the end, both AMD and the window approach seem to introduce some form of global state. But is this really bad? First of all, the dynamic nature of JavaScript allows us to test our module with an equal effort in both cases. Secondly, we can still comply to the Principle of Least Knowledge by using constructor arguments (if a module returns a constructor-like object):

var myMdViz = new markdownViz({ markdown: { /* mock markdown */ }});
myMdViz.init($('#textinput'));

Here, module dependencies can be passed as optional arguments. If not present, the constructor defaults to the global modules.

That’s definitely testable, and it’s also really easy. So with the right build process, I don’t think your project needs AMD. You still have to know the dependencies between modules, but they’re just specified in a different way, and your build script can easily figure that our for you.