Simplifying code with standardized pagination, sorting, and search

Neeto is a collection of products. Here, we standardize boilerplate and repetitive code into Nanos. The search, sorting, and pagination functionalities are essential for every listing page. We realized that each product had its own custom implementations for these operations, resulting in a lot of duplicated code. We wanted this logic to be abstracted and handled uniformly.

Sorting and pagination

If you have worked with tables, you may already be familiar with the following code. In this example, we are using the Table component from NeetoUI. It is the developer's responsibility to handle the sorting and pagination logic, make the API calls with an updated payload, and ensure that the URL is updated accordingly.

import React, { useState } from "react";
import { Table } from "@bigbinary/neetoui";

const Teams = () => {
  const [page, setPage] = useState(1);
  const [sortBy, setSortBy] = useState(null);
  const [orderBy, setOrderBy] = useState(null);

  const handleSort = ({ sortBy, orderBy }) => {
    setSortBy(sortBy);
    setOrderBy(orderBy);
    // Custom sort logic
  };

  const handlePageChange = page => {
    setPage(page);
    // Custom pagination logic.
  };

  const { data: teams, isLoading } = useFetchTeams({ page, sortBy, orderBy });

  return (
    <Table
      currentPageNumber={page}
      handlePageChange={handlePageChange}
      onChange={(_, __, sorter) => handleSort(sorter)}
      {...{ totalCount, rowData, columnData, ...otherProps }}
    />
  );
};

export default Teams;

To make both the frontend and backend more standardized and reusable, we established some conventions. The NeetoUI Table will handle both sorting and pagination internally. When the user navigates to a different page, NeetoUI will update the page and page_size query parameters in the URL.

Similarly, when a column is sorted, the sort_by and order_by query parameters are updated. We no longer need a dedicated state to store these values. When the query parameter changes, the API will be called with the modified payload.

Here is what the simplified code looks like:

import React from "react";
import { Table } from "@bigbinary/neetoui";
import { useQueryParams } = "@bigbinary/neeto-commons-frontend/react-utils"

const Teams = () => {
  const { page, sortBy, orderBy } = useQueryParams();

  const { data: teams, isLoading } = useFetchTeams({ page,
                                                     sortBy,
                                                     orderBy });

  return (
    <Table {...{ totalCount, rowData, columnData, ...otherProps }} />
  );
};
export default Teams;

Here, we utilize a useQueryParams utility helper. It parses the URL and returns the query parameters after converting them to camel case.

We noticed that a similar cleanup can be done on the backend side. After establishing a consistent variable naming convention for page, page_size, order_by, and sort_by in every listing page, it became easier to create common utility functions for the backend. Here's what it looks like:

module Filterable
  extend ActiveSupport::Concern
  include Pagy::Backend

  def sort_and_paginate(records)
    sorted_records = apply_sort(records)
    apply_pagination(sorted_records)
  end

  def apply_sort(records)
    records.order(sort_by => order_by)
  end

  def sort_by
    params[:sort_by].presence || "created_at"
  end

  def order_by
    case params[:order_by]&.downcase
    when "asc", "ascend"
      "ASC"
    when "desc", "descend"
      "DESC"
    else
      "ASC"
    end
  end

  def apply_pagination(records)
    pagination_method = records.is_a?(ActiveRecord::Relation) ? :pagy : :pagy_array
    pagy, paginated_records = send(pagination_method, records, page: params[:page], items: params[:page_size])

    [pagy, paginated_records]
  end
end

The apply_pagination helper method can be used to paginate the results. It uses the Pagy gem internally to perform the pagination. It provides a generic method pagy that works with ActiveRecord out of the box. It also provides a pagy_array method which paginates an array of records. Both these methods returns a Pagy instance along with the paginated records.

The apply_sort method sorts the given records based on a specified column name and direction. By default, records are sorted in ascending order of created_at timestamps, but this can be customized by specifying values for sort_by and order_by params. As sorting and pagination are common requirements of the listing pages, we also added a sort_and_paginate method which performs both these operations on provided records.

Searching

Searching is a common functionality in any web application. Developers need to ensure that the results are refetched whenever the search term changes. The debouncing logic is added to minimize API requests and improve user experience. As you can see in the below snippet, we had to maintain a dedicated state for search term as well.

import React, { useState } from "react";
import { useDebounce } from "@bigbinary/neeto-commons-frontend/react-utils";
import { useFetchTeams } from "hooks/reactQuery/useFetchTeamsApi";

const Teams = () => {
  const [searchString, setSearchString] = useState("");
  const debouncedSearchString = useDebounce(searchString.trim());

  const { data: teams, isLoading } = useFetchTeams(debouncedSearchString);

  return (
    <Input
      type="search"
      value={searchString}
      onChange={({ target: { value } }) => setSearchString(value)}
    />
  );
};

export default Teams;

As we standardized the naming pattern for search term, we were able to directly incorporate the search term value into the URL query parameters. This helped us to retrieve the search term eliminating the need for a separate state.

We also introduced a new component in NeetoMolecules, to handle the search functionality in all products. This Search component will internally handle the debounced updates when the search term changes. It will also update the search_term query param in URL. Here is what the simplified code looks like:

import React from "react";
import Search from "@bigbinary/neeto-molecules/Search";
import { useFetchTeams } from "hooks/reactQuery/useFetchTeamsApi";

const Teams = () => {
  const { searchTerm = "" } = useQueryParams();

  const { data: teams, isLoading } = useFetchTeams(searchTerm);

  return <Search />;
};

export default Teams;

We also updated the Filterable concern mentioned earlier to simplify the logic in the backend. The search_term method retrieves the keyword specified in the URL query parameters, while search? checks whether the results should be filtered based on any keyword.

module Filterable
  extend ActiveSupport::Concern

  # ... previous code

  def search_term
    filter_params[:search_term]
  end

  def search?
    search_term.present?
  end
end

Let's consolidate everything in the TeamsController. In the following code, we need to fetch all the teams belonging to an organization, filter them based on the search term and return the results after sorting and pagination.

def index
  if params[:search_string].present?
    @teams = @organization.teams.filter_by_name(params[:search_string])
  end

  @teams = @teams
    .order(params[:column] => params[:direction])
    .page(params[:current_page])
    .per(params[:limit])
end

We were able to simplify this logic with the help of the methods provided by the Filterable concern. As you can see below, this helped us in removing a lot of boilerplate code and improving the readability.

def index
  @teams = @organization.teams.filter_by_name(search_term) if search?
  @teams = sort_and_paginate(@teams)
end

This standardized approach helped us to extract and centralize the common logic. It also accelerated the development and maintenance cycle as the code structure is now clear and consistent across all the products.