Streamlining translation resource loading in React apps with babel-plugin-preval

At Neeto, our product development involves reusing common components, utilities, and initializers across various projects. To maintain a cohesive and standardized codebase, we've created specialized packages, or "nanos" such as neeto-commons-frontend, neeto-fields-nano, and neeto-team-members-nano.

neeto-commons-frontend houses utility functions, components, hooks, configuration settings etc. neeto-fields-nano manages dynamic field components, while neeto-team-members-nano handles team member management functionalities.

These nanos, along with others, reduce redundancy and promote consistency across our products.

Translation Challenges

Many of our packages export components with text that requires internalization, maintaining their own translation files. We encountered an issue with the withT higher-order component (HOC) using react-i18next inside neeto-commons-frontend. Upon investigation, we found discrepancies in how packages handled dependencies.

withT is an HOC which provides the t function from react-i18next to the wrapped component as a prop.

import { withTranslation } from "react-i18next";

const withT = (Component, options, namespace = undefined) =>
  withTranslation(namespace, options)(Component);

export default withT;

// Example usage of withT:
const ComponentWithTranslation = withT(({ t }) => <div>{t("some.key")}</div>);

Let us first understand the difference between dependencies and peerDependencies. dependencies are external packages a library relies on, automatically installed with the library. peerDependencies suggest that users should explicitly install these dependencies in their application if they want to use this library. If not installed, we will get warnings during installation of this library to prompt us to install the peer dependencies.

react-i18next and i18next were listed as peerDependencies in neeto-commons-frontend's package.json. So, it will be using the instances of these libraries from the host application.

Examining neeto-fields-nano, we found that it listed react-i18next and i18next as dependencies rather than peerDependencies. This meant it had its own instances of these libraries, leading to initialization discrepancies.

Contrastingly, neeto-team-members-frontend listed react-i18next and i18next as peerDependencies, relying on the host application's initialization of these libraries.

The initialization logic, which is common among all the products is placed inside neeto-commons-frontend. To ensure translations from all packages, including neeto-commons-frontend are merged with that of the host application, we crafted a custom initializeI18n function:

import DOMPurify from "dompurify";
import i18n from "i18next";
import { curry, mergeAll, mergeDeepLeft } from "ramda";
import { initReactI18next } from "react-i18next";

import commonsEn from "../translations/en.json";

const packageNames = [
  "neeto-molecules",
  "neeto-integrations-frontend",
  "neeto-team-members-frontend",
  "neeto-tags-frontend",
];

const getPackageTranslations = (language, packageNames) => {
  const loadTranslations = curry((language, packageName) => {
    try {
      return require(`../${packageName}/src/translations/${language}.json`);
    } catch {
      return {};
    }
  });
  const allTranslations = packageNames.map(loadTranslations(language));

  return mergeAll(allTranslations);
};

const packageEnTranslations = getPackageTranslations("en", packageNames);

const en = mergeDeepLeft(commonsEn, packageEnTranslations);

const initializeI18n = resources => {
  i18n.use(initReactI18next).init({
    resources: mergeDeepLeft(resources, { en: { translation: en } }),
    lng: "en",
    fallbackLng: "en",
    interpolation: { escapeValue: false, skipOnVariables: false },
  });
};

export default initializeI18n;

Here we are looping through all the packages mentioned in packageNames and merging with the translation keys inside neeto-commons-frontend, along with the translation keys from the host app passed as an argument to initializeI18n function.

While this approach successfully merges translations, it introduced complexity. As our project expanded with the inclusion of more packages, we found the need to regularly update the neeto-commons-frontend code, manually adding new packages to the packageNames array. This prompted us to seek an automated solution to streamline this process.

Given that all our packages are under the @bigbinary namespace in npm, we explored the possibility of dynamically handling this. An initial thought was to iterate through packages under node_modules/@bigbinary and merge their translation keys. However, executing this in the browser was not possible since the browser does not have access to its build environment's file system.

Enter babel-plugin-preval:

To automate our translation aggregation process, we turned to babel-plugin-preval. This plugin allows us to execute dynamic tasks during build time.

babel-plugin-preval allows us to specify some code that runs in Node and whatever we module.exports in there will be swapped.

Let us look at an example:

const x = preval`module.exports = 1`;

will be transpiled to:

const x = 1;

With preval.require, the following code:

const fileLastModifiedDate = preval.require("./get-last-modified-date");

will be transpiled to:

const fileLastModifiedDate = "2018-07-05";

Here is the content of get-last-modified-date.js:

module.exports = "2018-07-05";

Here, the 2018-07-05 date is read from the file and replaced in the code.

In order to use this plugin we just need to install it and add preval to the plugins array in .babelrc or .babel.config.js

Streamlining Translations with preval:

We revamped the initializeI18n function using preval.require to dynamically fetch translations from all @bigbinary-namespaced packages. This eliminated the need for manual updates in neeto-commons-frontend whenever a new package was added.

With preval, our initializeI18n function was refactored as follows:

const initializeI18n = hostTranslations => {
  // eslint-disable-next-line no-undef
  const packageTranslations = preval.require(
    "../configs/scripts/getPkgTranslations.js"
  );

  const commonsTranslations = { en: { translation: commonsEn } };

  const resources = [
    hostTranslations,
    commonsTranslations,
    packageTranslations,
  ].reduce(mergeDeepLeft);
};

The code for getPackageTranslations.js:

const fs = require("fs");
const path = require("path");

const { mergeDeepLeft } = require("ramda");

const packageDir = path.join(__dirname, "../../");

const getPkgTransPath = pkg => {
  const basePath = path.join(packageDir, pkg);

  const transPath1 = path.join(basePath, "app/javascript/src/translations");
  const transPath2 = path.join(basePath, "src/translations");

  return fs.existsSync(transPath1) ? transPath1 : transPath2;
};

const packages = fs.readdirSync(packageDir);

const loadTranslations = translationsDir => {
  try {
    const jsonFiles = fs
      .readdirSync(translationsDir)
      .filter(file => file.endsWith(".json"))
      .map(file => path.join(translationsDir, file));

    const translations = {};

    jsonFiles.forEach(jsonFile => {
      const content = fs.readFileSync(jsonFile, "utf8");
      const basename = path.basename(jsonFile, ".json");

      translations[basename] = { translation: JSON.parse(content) };
    });

    return translations;
  } catch {
    return {};
  }
};

const packageTranslations = packages
  .map(pkg => loadTranslations(getPkgTransPath(pkg)))
  .reduce(mergeDeepLeft);

module.exports = packageTranslations;

In this workflow, we iterate through all the packages to retrieve their translation files and subsequently merge them. We are able to access the translation files of our packages since we have exposed those files in the package.json of all our packages.

files property in package.json is an allowlist of all files that should be included in an npm release.

Inside package.json of our nanos, we have added the translations folder to the files property:

{
  // other properties
  files: ["app/javascript/src/translations"];
}

It's worth noting that we won't run preval at the time of bundling neeto-commons-frontend. Our objective is to merge the translation keys of all installed dependencies of the host project with those of the host project itself. Since neeto-commons-frontend is one of the dependencies of the host projects, executing preval within neeto-commons-frontend is not what we needed.

Consequently, we've manually excluded the preval plugin from the Babel configuration specific to neeto-commons-frontend:

module.exports = function (api) {
  const config = defaultConfigurations(api);
  config.plugins = config.plugins.filter(plugin => plugin !== "preval");
  config.sourceMaps = true;
};

With this change, the Babel compiler simply skips the code for preval during build time and the preval related code will be kept as it is after compilation for neeto-commons-frontend.

Another challenge arises from the default behavior of webpack, which does not transpile the node_modules folder by default. However, it's necessary for our host application to perform this transpilation. To address this, we wrote a custom rule for webpack. The webpack rules are also placed in neeto-commons-frontend and shared across all the products.

  {
    test: /\.js$/,
    include:
      /node_modules\/@bigbinary\/neeto-commons-frontend\/initializers\/i18n/,
    use: { loader: "babel-loader", options: { plugins: ["preval"] } },
  },

This configuration ensures that Babel applies the necessary transformations to the code located in node_modules/@bigbinary/neeto-commons-frontend/initializers/i18n/ within the host application.

Upon transpilation, our system consolidates all translations from each package, including those from the neeto-commons-frontend package, and incorporates them into the host application.

To mitigate potential conflicts arising from overlapping keys, we've implemented a namespacing strategy for translations originating from various packages. This ensures that translations from our packages carry a distinctive key, uniquely identifying their source.

As an illustration, consider the neeto-filters-nano package. In its English translation file (en.json), the translations are organized within a dedicated namespace:

neetoFilters: {
    "common": { }
}

Conclusion:

Leveraging babel-plugin-preval significantly simplified our translation resource loading process. The automation introduced not only streamlined our workflow but also ensured that our applications stay consistent and easily adaptable to future package additions.