---
title: "Challenges faced while building Neeto commons frontend"
description: "Challenges faced while building Neeto commons frontend"
canonical_url: "https://www.bigbinary.com/blog/neeto-commons-frontend"
markdown_url: "https://www.bigbinary.com/blog/neeto-commons-frontend.md"
---

# Challenges faced while building Neeto commons frontend

Challenges faced while building Neeto commons frontend

- Author: Amaljith K
- Published: July 11, 2023
- Categories: JavaScript, ReactJS

At [neeto](https://www.neeto.com/), we are building
[a lot of products](https://blog.neeto.com/p/neeto-products-and-people) to
simplify how we work. Many of these products share similar features, such as a
404 page, team member invitations, a sidebar, app switcher, Slack integration,
etc. For consistency in both UI and functionality, these common business
requirements must remain uniform across all Neeto products.

To bring a new Neeto product to market, we used to copy the whole repo of an
already existing product and then we used to delete the previous
product-specific code from the new repo. This ensured that the visual design,
application initialization logic, code quality enforcement rules, etc. are the
same in all Neeto products. However, during active development, we noticed three
big problems with the consistency of Neeto products.

- Different teams implemented the same business requirements in different ways,
  causing products to go out of sync.
- Bringing an update to the common functionality required manual changes to
  every repository.
- Some teams made quick and dirty changes to the common copied logic to fix
  coding inconveniences.

To address these challenges, we needed a way to share the common code. Simply
copying the common code to each repository was not scalable. We built a ruby gem
named `neeto-commons-backend` and an NPM package named `neeto-commons-frontend`
to hold all our common code.

The implementation of the `neeto-commons-backend` gem was relatively easy, but
the implementation of the frontend package, `neeto-commons-frontend`, posed
several challenges. Let's discuss some of the challenges we faced while building
`neeto-commons-frontend`.

### public or private?

Both `neeto-commons-backend` and `neeto-commons-frontend` contained a lot of
business logic specific to neeto. So having these two repos as "private" in
GitHub was an easy call.

When it comes to using `neeto-commons-backend` gem, we can directly use the gem
from GitHub if we configure the access tokens correctly in the host
applications. Recently, we have deployed a private gem server to host our gems.
However, for `neeto-commons-frontend`, things aren't that straightforward. If we
decide to serve the package directly from Github private repository, we will
have the following problems:

- Apart from `neeto-commons-frontend`, we have multiple other frontend packages.
  To use `neeto-commons-frontend` as a dependency in them, we will need to
  hardcode the GitHub access token in their `package.json` file. But,
  `package.json` is considered to be a public file. So it is not safe to add any
  secret keys or tokens to it. If we unknowingly publish any of those packages
  to npm, our tokens would leak to the public.
- We cannot directly use the ES6 source code in the host application. We need to
  transpile the JS files before serving. If we were serving
  `neeto-commons-frontend` directly from GitHub, we are limited to these
  options:

  - Add a `post_install` hook to the package: `post_install` command is said to
    be executed at the time of running `yarn install` or `yarn add` on the host
    project. We can add a command to transpile `neeto-commons-frontend` from
    that hook. But the `post_install` hook isn't guaranteed to always run. So it
    isn't a reliable strategy.
  - Another option is to maintain a copy of transpiled JS output in our GitHub
    repo by using pre-commit and prepush hooks. But, keeping generated code in
    version control isn't a good practice. Moreover, can't trust pre-commit and
    prepush hooks because they can be skipped or can fail to run.

So, we decided to bundle `neeto-commons-frontend`'s JS code using
[rollup](https://rollupjs.org) and release it to NPM as a public package. Even
though the source code will remain private in GitHub, it would make our bundle
available to the public. Anyone can do
`yarn add @bigbinary/neeto-commons-frontend` to obtain our JS bundle.

However, minified JavaScript bundles are nearly impossible to comprehend. Hence,
we think it's reasonable to make them public. We anyway need the JavaScript
bundle to be served publicly in the browsers while loading Neeto products. We
cannot keep the frontend JS code completely private.

### Release management

In the initial stages of building `neeto-commons-frontend`, we used to split a
large feature into several small sub-issues. So, we raise many small PRs to
accomplish a single feature. For this reason, we didn't want to publish a new
version of `neeto-commons-frontend` after merging every PR. We needed manual
control over the publishing process.

Also, whenever we do decide to publish a new version, we wish to have an
automated mechanism to generate release notes explaining the changes from the
previous revision.

To satisfy these requirements, we decided to use
[GitHub releases](https://docs.github.com/en/repositories/releasing-projects-on-github/managing-releases-in-a-repository).
It offered the following benefits:

- GitHub can automatically generate release notes using the titles of the merged
  PRs since the last release.
- The generated release notes will contain a link to visualize code diff between
  the current and previous releases.
- We can run a GitHub action to automatically publish the package to npm when we
  create a new release.

We successfully followed that process for a long time. Later on,
`neeto-commons-frontend` became stable. Now, each PR is comprehensive and
requires an NPM publish. So, we changed our GitHub action to create a GitHub
release and publish the package to NPM on every PR merge.

### common initializers

The `neeto-commons-backend` gem consists of various code components necessary
for initializing the Rails backend, such as configuring CORS, establishing
cache, and more.

Likewise, in the frontend, certain initialization tasks must be completed before
the React components can begin rendering. These tasks include configuring Axios
interceptors and headers, initializing Honeybadger and Mixpanel integrations,
setting up translation resources, and more.

Just like `neeto-commons-backend`, we wanted `neeto-commons-frontend` to perform
all these frontend initialization tasks on its own. But it wasn't as
straightforward as we anticipated. Here are some challenges we faced while
initializing the host application from `neeto-commons-frontend`:

#### Modifying axios instance of the host application

Axios lets us customize its default instance at runtime by adding custom
headers. With that, all network requests from our app will have those headers
set implicitly. Also, Axios lets us register request and response interceptors
to view and edit requests and responses before it is sent or received.

All Neeto products use this feature to set the Auth token and CSRF token in the
headers. They also register interceptors for different use cases like showing
toaster messages, handling authorization errors, etc.

Since this logic is the same in all products, we decided to move it to
`neeto-commons-frontend`. Our requirement was to customize the host project's
Axios instance from `neeto-commons-frontend`, without having to write any code
from the host project.

For better clarity, let us assume that we are trying to initialize Axios in
[NeetoCal](https://www.neeto.com/neetocal) using `neeto-commons-frontend`.

To use Axios in `neeto-commons-frontend`, just like any other JS project, we
need to add `axios` to its package.json. But if we were to add `axios` as a
dependency, rollup will pull out the source code from `axios` and include it in
the package's published bundle.

Similarly, since NeetoCal has both `neeto-commons-frontend` and `axios` as its
dependencies, [webpack](https://webpack.js.org) will pull out both of their
source code and add it to the JS bundle of NeetoCal. It will cause NeetoCal's JS
bundle to have two copies of `axios` code. One from NeetoCal's dependencies and
another one from `neeto-commons-frontend`'s bundle.

When a browser loads NeetoCal's bundle, both those codes will get initialized
and we will have two separate instances of Axios. Any customizations done from
`neeto-commons-frontend` will be applicable only to its own Axios instance. We
won't be able to touch the NeetoCal's Axios instance from
`neeto-commons-frontend`.

As a solution for this, we defined `axios` as a `peerDependency` in
`neeto-commons-frontend`'s package.json. Since we use
[rollup-plugin-peer-deps-external](https://www.npmjs.com/package/rollup-plugin-peer-deps-external)
plugin, rollup will consider `axios` as an external dependency while bundling.
That means rollup will not pull code from `axios` and add it to
`neeto-commons-frontend` bundle. Instead, it will keep the
`import axios from "axios"` statement as it is and assume that NeetoCal will
have this dependency installed and available at runtime.

Since both NeetoCal and `neeto-commons-frontend` are now importing Axios from
the same source, both of them will share the same instance. Any modifications
done from `neeto-commons-frontend` will reflect on the Axios instance used in
the project as well.

#### Placement of initialization logic

We were unsure of where to initialize the application from. We first tried
initializing the app from `useEffect` hook of the top-most component, `App.jsx`.

But, as per React's life cycle, the app will run one complete render cycle of
all nested components before `useEffect` gets called. Also, a parent component's
`useEffect` will be executed only after all the `useEffect`s registered in the
child components are completed.

This won't work for us due to the following reasons:

- We were using `i18next.t()` function in several constants to render locale
  translations. For the translations to be available, we need to initialize
  `i18next` before using it. But since constants get initialized immediately
  after the bundle is loaded, all those calls will result in
  `Translation not found` errors.
- Some nested components were performing API calls from their `useEffect` hooks.
  Since initialization is not completed by that time, the requests will fail due
  to missing authentication keys.

To avoid the problem with the delay in `useEffect` hook, we could have invoked
the initialization step directly from the rendering code of `App.jsx`, by
inlining it with the function definition. But, it is not a good practice to
introduce side effects from outside `useEffect` hooks. So, we didn't go with
that.

After some trials and errors, we finally decided to place the initialization
function call in `app/javascript/packs/application.js`. It is the file that gets
executed before React gets mounted. So our app will be fully initialized before
it starts to render.

### utility functions

We created `neeto-commons-frontend` by copying common code from all neeto
products to it. We identified these categories of common code: application
initialization logic, react components and hooks, and general utility functions.

When copying utility functions, we realized that we could implement a new set of
utility functions to minimize boilerplate code in all Neeto products. There are
a lot of operations done using array functions like `map`, `filter`, `find`,
etc. We were using arrow functions to compare nested properties, which was the
most common boilerplate.

We decided to introduce a function `matches`, which checks whether the given
pattern is partially equal to the given object. It works like this: the pattern
`{ name: "Oliver" }` matches the object `{ name: "Oliver", phone: 000000 }`
because the object contains the key `name` and its value is the same in both the
pattern and the object.

With this function as the foundation, we built several array functions like
`findBy`, `removeBy`, `replaceBy`, etc. All these functions check for the
element that `match`es the given pattern from an array and performs the required
operation on that element.

We also took inspiration from Ramda and implemented currying for such utility
functions. This shortened the JS code and made it more declarative.

```js
// before
setUsers(users =>
  users.map(user => (user.address.pincode === 600213 ? newUser : user))
);

// after
setUsers(replaceBy({ address: { pincode: 600213 } }, newUser));
```

```js
// before
const defaultOrg = organizations.find(({ users }) =>
  users.includes(DEFAULT_USER)
);

// after
const defaultOrg = findBy({ users: includes(DEFAULT_USER) }, organizations);
```

You can read our blog
[Extending pure utility functions of Ramda.js](https://www.bigbinary.com/blog/extending-pure-utility-functions-of-ramda)
to learn more about how we built these utility functions.

### dependency management

The utility functions exported by `neeto-commons-frontend` are like an extension
to Ramda. They can be used outside Neeto web applications as well. Several
frontend packages and even React Native team can use these utility functions.

Since we import multiple packages as external modules (to use the host project's
modules, for example, Axios), we cannot export `neeto-commons-frontend` as a
single bundle. This will force the host applications to have those packages in
their dependencies.

To avoid this problem, we decided to create separate bundles for each category.
We now have four independent bundles: `pure`, `utils`, `react-utils`, and
`initializers`.

`pure` bundle contains all the pure functions we have discussed earlier. It
needs only Ramda as an external dependency. `utils` bundle encompasses general
utility functions which has dependencies on packages other than Ramda. An
example for that is `copyToClipboard` function. It shows a toaster message if
copying is successful. So it depends on `@bigbinary/neetoui` package as well.
`react-utils` and `initializers` contains several neeto-specific external
dependencies. They are designed to work only on Neeto web apps.

### IDE support & types

Initially, all frontend packages at BigBinary were serving UMD bundles. They are
compatible with every environment. So there is not much headache of having to
publish multiple bundles for different environments.

But UMD bundles do not assist IDEs well. That is, IDE can't provide auto-import,
autocompletion, and type support if we distribute UMD packages alone. IDEs even
give false positive errors when importing items from the package since they
can't detect such an export in the bundle.

The first workaround we tried is to serve ESM or CJS bundles instead of UMD.
Both ESM and CJS work well with imports. But imports from the bundle are
implicitly typed as `any` by the IDE. We won't get any predictions for function
parameters or component props. We were OK with this setup for a few weeks. This
at least does not give false positive errors.

But the problem with this setup is that the developer continuously needs to
refer to the docs to understand the parameters a function accepts. This
significantly degrades the development experience. To avoid this hassle, some
developers preferred not to use our functions and instead wrote lengthy vanilla
JS code.

Later, we found a solution to this problem. We added explicit type definition
using `.d.ts` files in `neeto-commons-frontend` package. They contain the type
definition of all our exports, written in typescript. We won't copy the JS
implementation code to it. It will only contain function declarations.

Since we were exporting four different bundles, we had to add four different
`.d.ts` files with the same name as the bundle. That is, we have `pure.d.ts`,
`utils.d.ts`, `react-utils.d.ts`, and `initializers.d.ts`. The IDE automatically
picks up the correct type definition file for the bundle we are importing and
uses it to give predictions.

The introduction of type declaration helped us improve the IDE support
significantly. It also allows us to add JSDoc comments and deprecation notices
for the exported items. Using these, the IDE can show documentation for the
functions while the developer types.

### People not knowing the available functions

Even though we were exporting tons of functions from `neeto-commons-frontend`,
people were not aware of the existence of many of them. So we found that many
were reinventing the wheel or wasting their time writing the boilerplate.

As a solution for this, we decided to add some custom ESLint rules to
`eslint-plugin-neeto`, which shows warnings to the user about a possible Ramda
or `neeto-commons-frontend` alternative when we detect a corresponding
boilerplate code.

You can find the story behind `eslint-plugin-neeto` and the challenges faced
during its development on another blog here.

## Links

- [Human page](https://www.bigbinary.com/blog/neeto-commons-frontend)