Simplifying code with standardized pagination, sorting, and search

Neeto is a collection of products. Here, we standardize boilerplate and repetitive code into Nanos. The search, sorting, and pagination functionalities are essential for every listing page. We realized that each product had its own custom implementations for these operations, resulting in a lot of duplicated code. We wanted this logic to be abstracted and handled uniformly.

Sorting and pagination

If you have worked with tables, you may already be familiar with the following code. In this example, we are using the Table component from NeetoUI. It is the developer's responsibility to handle the sorting and pagination logic, make the API calls with an updated payload, and ensure that the URL is updated accordingly.

1import React, { useState } from "react";
2import { Table } from "@bigbinary/neetoui";
3
4const Teams = () => {
5  const [page, setPage] = useState(1);
6  const [sortBy, setSortBy] = useState(null);
7  const [orderBy, setOrderBy] = useState(null);
8
9  const handleSort = ({ sortBy, orderBy }) => {
10    setSortBy(sortBy);
11    setOrderBy(orderBy);
12    // Custom sort logic
13  };
14
15  const handlePageChange = page => {
16    setPage(page);
17    // Custom pagination logic.
18  };
19
20  const { data: teams, isLoading } = useFetchTeams({ page, sortBy, orderBy });
21
22  return (
23    <Table
24      currentPageNumber={page}
25      handlePageChange={handlePageChange}
26      onChange={(_, __, sorter) => handleSort(sorter)}
27      {...{ totalCount, rowData, columnData, ...otherProps }}
28    />
29  );
30};
31
32export default Teams;
Copy

To make both the frontend and backend more standardized and reusable, we established some conventions. The NeetoUI Table will handle both sorting and pagination internally. When the user navigates to a different page, NeetoUI will update the page and page_size query parameters in the URL.

Similarly, when a column is sorted, the sort_by and order_by query parameters are updated. We no longer need a dedicated state to store these values. When the query parameter changes, the API will be called with the modified payload.

Here is what the simplified code looks like:

1import React from "react";
2import { Table } from "@bigbinary/neetoui";
3import { useQueryParams } = "@bigbinary/neeto-commons-frontend/react-utils"
4
5const Teams = () => {
6  const { page, sortBy, orderBy } = useQueryParams();
7
8  const { data: teams, isLoading } = useFetchTeams({ page,
9                                                     sortBy,
10                                                     orderBy });
11
12  return (
13    <Table {...{ totalCount, rowData, columnData, ...otherProps }} />
14  );
15};
16export default Teams;
Copy

Here, we utilize a useQueryParams utility helper. It parses the URL and returns the query parameters after converting them to camel case.

We noticed that a similar cleanup can be done on the backend side. After establishing a consistent variable naming convention for page, page_size, order_by, and sort_by in every listing page, it became easier to create common utility functions for the backend. Here's what it looks like:

1module Filterable
2  extend ActiveSupport::Concern
3  include Pagy::Backend
4
5  def sort_and_paginate(records)
6    sorted_records = apply_sort(records)
7    apply_pagination(sorted_records)
8  end
9
10  def apply_sort(records)
11    records.order(sort_by => order_by)
12  end
13
14  def sort_by
15    params[:sort_by].presence || "created_at"
16  end
17
18  def order_by
19    case params[:order_by]&.downcase
20    when "asc", "ascend"
21      "ASC"
22    when "desc", "descend"
23      "DESC"
24    else
25      "ASC"
26    end
27  end
28
29  def apply_pagination(records)
30    pagination_method = records.is_a?(ActiveRecord::Relation) ? :pagy : :pagy_array
31    pagy, paginated_records = send(pagination_method, records, page: params[:page], items: params[:page_size])
32
33    [pagy, paginated_records]
34  end
35end
Copy

The apply_pagination helper method can be used to paginate the results. It uses the Pagy gem internally to perform the pagination. It provides a generic method pagy that works with ActiveRecord out of the box. It also provides a pagy_array method which paginates an array of records. Both these methods returns a Pagy instance along with the paginated records.

The apply_sort method sorts the given records based on a specified column name and direction. By default, records are sorted in ascending order of created_at timestamps, but this can be customized by specifying values for sort_by and order_by params. As sorting and pagination are common requirements of the listing pages, we also added a sort_and_paginate method which performs both these operations on provided records.

Searching

Searching is a common functionality in any web application. Developers need to ensure that the results are refetched whenever the search term changes. The debouncing logic is added to minimize API requests and improve user experience. As you can see in the below snippet, we had to maintain a dedicated state for search term as well.

1import React, { useState } from "react";
2import { useDebounce } from "@bigbinary/neeto-commons-frontend/react-utils";
3import { useFetchTeams } from "hooks/reactQuery/useFetchTeamsApi";
4
5const Teams = () => {
6  const [searchString, setSearchString] = useState("");
7  const debouncedSearchString = useDebounce(searchString.trim());
8
9  const { data: teams, isLoading } = useFetchTeams(debouncedSearchString);
10
11  return (
12    <Input
13      type="search"
14      value={searchString}
15      onChange={({ target: { value } }) => setSearchString(value)}
16    />
17  );
18};
19
20export default Teams;
Copy

As we standardized the naming pattern for search term, we were able to directly incorporate the search term value into the URL query parameters. This helped us to retrieve the search term eliminating the need for a separate state.

We also introduced a new component in NeetoMolecules, to handle the search functionality in all products. This Search component will internally handle the debounced updates when the search term changes. It will also update the search_term query param in URL. Here is what the simplified code looks like:

1import React from "react";
2import Search from "@bigbinary/neeto-molecules/Search";
3import { useFetchTeams } from "hooks/reactQuery/useFetchTeamsApi";
4
5const Teams = () => {
6  const { searchTerm = "" } = useQueryParams();
7
8  const { data: teams, isLoading } = useFetchTeams(searchTerm);
9
10  return <Search />;
11};
12
13export default Teams;
Copy

We also updated the Filterable concern mentioned earlier to simplify the logic in the backend. The search_term method retrieves the keyword specified in the URL query parameters, while search? checks whether the results should be filtered based on any keyword.

1module Filterable
2  extend ActiveSupport::Concern
3
4  # ... previous code
5
6  def search_term
7    filter_params[:search_term]
8  end
9
10  def search?
11    search_term.present?
12  end
13end
Copy

Let's consolidate everything in the TeamsController. In the following code, we need to fetch all the teams belonging to an organization, filter them based on the search term and return the results after sorting and pagination.

1def index
2  if params[:search_string].present?
3    @teams = @organization.teams.filter_by_name(params[:search_string])
4  end
5
6  @teams = @teams
7    .order(params[:column] => params[:direction])
8    .page(params[:current_page])
9    .per(params[:limit])
10end
Copy

We were able to simplify this logic with the help of the methods provided by the Filterable concern. As you can see below, this helped us in removing a lot of boilerplate code and improving the readability.

1def index
2  @teams = @organization.teams.filter_by_name(search_term) if search?
3  @teams = sort_and_paginate(@teams)
4end
Copy

This standardized approach helped us to extract and centralize the common logic. It also accelerated the development and maintenance cycle as the code structure is now clear and consistent across all the products.