Migrating Gumroad from RequireJS to webpack

BigBinary has been working with Gumroad for a while. Following blog post has been posted with permission from Gumroad and we are very grateful to Sahil for allowing us to discuss the work in such an open environment.

This application is a JavaScript-heavy application as most consumer-oriented applications are these days. We recently changed the JavaScript build system for Gumroad from RequireJS to webpack. We'd like to talk about how we went about doing this.

Gumroad's web application is built using Ruby on Rails. The project was started way back in 2011 as this hacker news post suggests. When we began working on the code it was building JavaScript assets through two systems Sprockets and RequireJS. From what we could tell, all the code which was using a new(at the time) frontend framework was processed by RequireJS first and then sprockets whereas the JavaScript files which are usually present under app/javascrips/assets and vendor/assets/javascripts in a typical Rails application were present as well but they were not being processed by RequireJS. Also, there were some libraries which were sourced using Bower.

We were tasked with the work of migrating the RequireJS build system over to webpack and replacing Bower with NPM. The reason behind this was that we wanted to use newer tools with wider community support. Another reason was that we wanted to be able to take advantage of all the goodies that webpack comes with though that was not a strong motivation at that point.

We decided to break down the task into small pieces which could be worked on in iterations and, more importantly, could be shipped in iterations. This would enable us to work on other tasks in the application in parallel and not be blocked on a big chunk of work. Keeping that in mind we split the task in three different steps.

Step 1: Migrate from RequireJS to webpack with the minimal amount of changes in the actual code.

Step 2: Use NPM packages in place of Bower components.

Step 3: Use NPM packages in place of libraries present under vendor/assets/javascripts.

Step 1: Migrate from RequireJS to webpack with the minimal amount of changes in the actual code

The first thing we did here was create a new webpack.config.js configuration file which would be used by webpack. We did our best to accurately translate the configuration from the RequireJS configuration file using multiple resources available online.

Here is how most JavaScript files which were to be processed by RequireJS looked like.

"use strict";

define(["braintree", "$app/ui/product/edit", "$app/data/product"], function (
  Braintree,
  ProductEditUI,
  ProductData
) {
  // Do something with Braintree, ProductEditUI, and ProductData
});

As you can see, the code did not use the newer import statements which you'd see in comparatively newer JavaScript code. As we've mentioned earlier, our goal was to have minimal code changes so we did not want to change to import just yet. Luckily for us, webpack supports the define API for specifying dependencies. This meant that we would not need to change how dependencies were specified in any of the JavaScript files.

In this step we also changed the build system configuration (The webpack.config.js file in this case) to use NPM packages where possible instead of using libraries from the vendor/ directory. This meant that we would need to have aliases in place for instances where the package name was different from the names we had aliased the libraries to.

For example, this is how the 'braintree' alias was set earlier in order to refer to the Braintree SDK. Now all the code had to do was to mention that braintree was a dependency.

require.config({
  paths: {
    braintree: "/vendor/assets/javascripts/braintree-2.16.0",
  },
});

With the change to use the NPM package in place of the JavaScript file the dependency sourcing did not work as expected because the NPM package name was 'braintree-web' and the source code was trying to load 'braintree' which was not known to the build system(webpack). In order to avoid making changes to source code we used the "alias" feature provided by webpack as shown below.

module.exports = {
  resolve: {
    alias: {
      braintree: "braintree-web",
    },
  },
};

We did this for all the dependencies which had been given an alias in the RequireJS configuration and we got dependency resolution to work as expected.

As a part of this step, we also created a new common chunk and used it to improve caching. You can read more about this feature here. Note that we would tweak this iteratively later but we thought it would be good to get started with the basic configuration right away.

Step 2: Use NPM packages in place of Bower components

Another goal of the migration was to remove Bower so as to make the build system simpler. The first reason behind this was that all Bower packages which we were using were available as NPM packages. The second reason was that Bower itself is recommending users to migrate to Yarn/webpack for a while now.

What we did here was simple. We removed Bower and the Bower configuration file. Then, we sourced the required Bower components as NPM packages instead by adding them to package.json. We also removed the aliases added to source them from the webpack configuration.

For example, here's the change required to the configuration file after sourcing clipboard as an NPM package instead of a Bower component.

resolve: {
  alias: {
    // Other Code

    $app:           path.resolve(__dirname, '../../app/javascript'),
    $lib:           path.resolve(__dirname, '../../lib/assets/javascripts')
-   clipboard:      path.resolve(__dirname, '../../vendor/assets/javascripts/clipboard.min.js')
  }
}

Step 3: Use NPM packages in place of libraries present under vendor/assets/javascripts

We had a lot of javascript libraries present under vendor/assets/javascripts which were sourced in the required javascript files. We deleted those files from the project and sourced them as NPM packages instead. This way we could have better visibility and control over the versions of these packages.

As part of this migration we also did some asset-related cleanups. These included removing unused JavaScript files, including JavaScript files only where required instead of sourcing them into the global scope, etc.

We were continuously measuring the performance of the application before and after applying changes to make sure that we were not worsening the performance during the migration. In the end, we found that we had improved the page load speeds by an average of 2%. Note that this task was not undertaken to improve the performance of the application. We are now planning to leverage webpack features and try to improve on this metric further.