Helping Babel move to ES Modules

The Babel project recently moved to running builds on Node.js 14, which means that Babel can now use ES Modules (ESM) instead of CommonJS (CJS) to import/export modules for internal scripts.

Also, with the upcoming Babel 8.0.0 release, the team is aiming to ship Babel as native ES Modules. With this goal in mind, the team is shifting all CommonJS imports/exports to ESM ones. This is where I got the opportunity to contribute to Babel recently.

Why ES Modules though?

For a very long time, JS (or ECMAScript) did not have a standardized module import/export syntax. Various independent packages introduced formats to help work with modules in JS. Most browsers used the AMD API (Asynchronous Module Definition) implemented in the Require.js package, which had its own syntax and quirks.

CommonJS on the other hand was the standard used by Node.js, and it was no less quirky. Inconsistent formatting and poor interoperability between packages irked JS developers enough to demand a standard format.

Lately, the ECMAScript Standardization body (TC39) has adopted ESM (ECMAScript modules) as the standard format for Javascript. Most web browsers already support this format and Node.js 14 now provides stable support for it.

The task at hand

The next task was to convert all internal top-level scripts from using CommonJS to ESM. The finer details of the implementation, along with interoperability issues with non-ESM files, would trouble CommonJS for some time though.

The simplest of changes was to replace require() statements in each file with import statements. For example, files starting like:

1"use strict";
2
3const plumber = require("gulp-plumber");
4const through = require("through2");
5const chalk = require("chalk");

would be modified like here:

1import plumber from "gulp-plumber";
2import through from "through2";
3import chalk from "chalk";

to allow modules to be imported as ES modules.

In the above example also note that because ES modules are in strict mode by default, so "use strict"; declarations were removed from the beginning of these top-level scripts.

Almost all current NPM packages are CommonJS packages, exposing their functionalities using the module.exports syntax.

In case a file/package exports more than one value, we need to use named imports instead:

1import { chalk } from "chalk";

Where the default export object from a CommonJS module was named differently, it had to be aliased during import to avoid breaking pre-existing variables' names in the files being converted to ESM. For example,

1const rollupBabel = require("@rollup/plugin-babel").default;

had to be replaced with:

1import { babel as rollupBabel } from "@rollup/plugin-babel";

so we could keep using the variable rollupBabel in the file.

For instances where require() statements needed to be replaced by the dynamic import() statements

1const getPackageJson = (pkg) => require(join(packageDir, pkg, "package.json"));
2
3// replaced by
4const getPackageJson = (pkg) => import(join(packageDir, pkg, "package.json"));

the subsequent calls everywhere now needed to be awaited:

1   .forEach(id => {
2      const { name, description } = getPackageJson(id);
3   })
4
5   //await added
6   .forEach(id => {
7      const { name, description } = await getPackageJson(id);
8   })

Other things like importing JSON modules are currently only supported in CommonJS mode. Those imports were left as-is.

Blockers

With all the changes made and committed, we bumped into the next big roadblock: package dependencies. Babel uses Yarn 2 internally, and particularly the PnP feature of Yarn 2. Unfortunately, the ESM loader API was experimental at the time and not being used by PnP. The Babel and Yarn teams coordinated to implement it soon after.

Similarly, Jest has its own custom loader for ESM, which meant it could not support testing ESM modules with Babel. That issue was side-stepped for the time being.

Network effects

The good thing about the whole grind of shifting from CommonJS to ESM is that a lot of other major packages are also considering and implementing ESM support. The shift to ESM-only by Babel is already building confidence in others to do the same. Special thanks to the Babel maintainers for setting a great example and encouraging others to move to ESM.

Conclusion

All told, it was a great experience adding a new feature into a well-maintained and widely-used package. The biggest lesson from this has to be how changes made in Babel affect and influence other major packages, and how maintainers of various major open source packages work in tandem to avoid breaking each other's code. It is a very open and collaborative ecosystem with people discussing and working through github issues, comments, and even twitter threads.

Check out the pull request for more details.

Failing Gracefully: Error Boundaries in React

React 16 introduced the concept of "Error Boundaries" within component trees. Web developers are often confused on its proper application; Should the entire app be wrapped in a single error boundary? Or should each component be wrapped in its own error boundary so that individual breakages don’t affect the whole app?

Below is my talk from React Day Bangalore that aims at figuring out some common patterns and design decisions on when and where to use React error boundaries for a fault tolerant React application.

Useful links

• Talk Slides
• New lifecycle method: getDerivedStateFromError [reddit]
• react-error-boundary

Ruby 3.1 adds Array#intersect?

Ruby 3.1 introduces the Array#intersect? method which returns boolean value true or false based on the given input arrays have common elements in it.

We already know Array#intersection or Array#& methods which are used to find the common elements between arrays.

1=> x = [1, 2, 5, 8]
2=> y = [2, 4, 5, 9]
3=> z = [3, 7]
4
5=> x.intersection(y) # x & y
6=> [2, 5]
7
8=> x.intersection(z) # x & z
9=> []

The intersection or & methods return an empty array or array having the common elements in it as result. We have to further call empty?, any? or blank? like methods to check whether two arrays intersect each other or not.

Before Ruby 3.1

1=> x.intersection(y).empty?
2=> false
3
4=> (x & z).empty?
5=> true
6
7=> (y & z).any?
8=> false

After Ruby 3.1

1=> x.intersect?(y)
2=> true
3
4=> y.intersect?(z)
5=> false

The Array#intersect? method accepts only single array as argument, but Array#intersection method can accept multiple arrays as arguments.

1=> x.intersection(y, z) # x & y & z
2=> []

The newly introduced intersect? method is faster than the above described checks using intersection or & since the new method avoids creating an intermediate array while evaluating for common elements. Also new method returns true as soon as it finds a common element between arrays.

Here's the relevant pull request and feature discussion for this change.

Rails 7.0 adds encryption to Active Record models

Before Rails 7.0, to add encryption on attributes of ActiveRecord models, we had to use third-party gems like lockbox which served the purpose but at the cost of an additional dependency.

Before we delve deeper, let's take a look at some terms related to encryption:

1. Encrypt: to scramble a message in a way that only the intended person can extract the original message.
2. Decrypt: to extract the original message from an encrypted one.
3. Key: a string of characters used to encrypt/decrypt a message.
4. Cipher: an algorithm used to encrypt and decrypt a message; RSA, Blowfish, and AES are some well-known examples.
5. Deterministic: a process with guaranteed results; the sum of a set of numbers never changes.
6. Non-deterministic: a process with unpredictable results; a roll of the dice can never be predicted.

Rails 7.0

Rails 7.0 adds encryption to attributes at the model level. By default, it supports encrypting serialized attribute types using the non-deterministic AES-GCM cipher.

To use this feature, we have to set the key_derivation_salt, primary_key, and deterministic_key variables in our environment file. These keys can be generated by running bin/rails db:encryption:init.

Let's enable encryption on the passport_number attribute of the User model.

1# app/models/user.rb
2class User < ApplicationRecord
3  encrypts :passport_number
4end

In the above code, we have asked Rails to encrypt the passport_number attribute of User model when writing it to the database. Now, whenever we create a User, we will see the encrypted value of passport_number in the table, and the unencrypted value when we query using ActiveRecord.

1# rails console
2>> User.create name: "Akhil", passport_number: "BKLPG564"
3  TRANSACTION (0.1ms)  begin transaction
4  User Create (0.6ms)  INSERT INTO "users" ("name", "passport_number", "created_at", "updated_at") VALUES (?, ?, ?, ?)  [["name", "Akhil"], ["passport_number", "{\"p\":\"iOUC3ESsyxY=\",\"h\":{\"iv\":\"lDdCxI3LeoPv0hxZ\",\"at\":\"D50hElso0YvI6d8Li+l+lw==\"}}"], ["created_at", "2021-04-15 10:27:50.800729"], ["updated_at", "2021-04-15 10:27:50.800729"]]
5  TRANSACTION (1.5ms)  commit transaction
6
7>> User.last
8=> #<User id: 1, name: "Akhil", adhar: "BKLPG564", created_at: "2021-04-15 10:27:50.800729000 +0000", updated_at: "2021-04-15 10:27:50.800729000 +0000">

Under the hood, Rails 7 uses the EncryptableRecord concern to perform encryption and decryption when saving and retrieving values from the database.

Check out the pull request for more details of this encryption system. It is also documented in the Ruby on Rails guides.

Rails 6.1 adds invert_where method

Rails 6.1 adds an invert_where method that will invert all scope conditions.

Let's see an example.

1class User
2  scope :active, -> { where(accepted: true, locked: false) }
3end
4
5>> User.all
6=> #<ActiveRecord::Relation [
7#<User id: 1, name: 'Rob', accepted: true, locked: true>
8#<User id: 2, name: 'Jack', accepted: false, locked: false>
9#<User id: 3, name: 'Nina', accepted: true, locked: false>
10#<User id: 4, name: 'Oliver', accepted: false, locked: true>

Now let's query for active and inactive users

1>> User.active
2# SELECT * FROM Users WHERE `accepted` = 1 AND `locked` = 0
3=> #<ActiveRecord::Relation [
4#<User id: 3, name: 'Nina', accepted: true, locked: false>]>
5
6>> User.active.invert_where
7# SELECT * FROM Users WHERE NOT (`accepted` = 1 AND `locked` = 0)
8=> #<ActiveRecord::Relation [
9#<User id: 1, name: 'Rob', accepted: true, locked: true>
10#<User id: 2, name: 'Jack', accepted: false, locked: false>
11#<User id: 4, name: 'Oliver', accepted: false, locked: true>]>

As we can see above, if we use invert_where with multiple attributes, it applies logical NOR, which is NOT a OR NOT b, to the WHERE clause of the query. Using DeMorgan's Law, it can also be written as NOT (a AND b) to match the second output.

Check out the pull request for more details.