Embracing Type Definitions and JSDoc Comments in JS packages

At BigBinary we decided to experiment with type definitions and JSDoc comments to see if using these tools improves how we interact with the JavaScript packages.

Why Type Definitions

We considered using TypeScript. However in the end we opted against it since most of our clients use plain JavaScript. Still we added type definitions for our JavaScript packages to:

• Reduce Ambiguity and Improved Clarity: Type definitions shed light on the expected input and output of functions and components, eliminating ambiguity and allowing developers to understand the code without needing to constantly refer to separate documentation.
• Enhance Productivity and Efficiency: By providing clear insights into the structure of our libraries, including information about arguments, props, and data types, type definitions significantly boost developer productivity and facilitate efficient coding practices.
• Seamless Code Editor Integration: Popular editors like VSCode leverage type definitions to offer powerful features like IntelliSense, further streamlining the development process and minimizing errors.

Unifying Documentation with JSDoc Comments

Although type definitions offered a valuable structural understanding, comprehensive documentation remained a challenge. Here is a typical example of our documentation of an exported function inside a Markdown file:

Navigating between code and Markdown files scattered across the package repository was cumbersome and time-consuming.

To address this pain point, we incorporated JSDoc comments alongside type definitions. This allowed us to:

• Consolidate Essential Documentation: Instead of relying on separate files, we embedded detailed usage instructions, examples, and additional information directly within the JSDoc comments, providing developers with readily available context within their IDE environment.
• Improve Accessibility and Transparency: By integrating documentation with the code itself, JSDoc comments eliminate the need for context switching and offer immediate access to critical information, enhancing developer understanding and decision-making.

Optimizing Documentation Workflow

Maintaining two separate documentation sets – Markdown files and JSDoc comments – proved inefficient. To ensure consistency and to eliminate redundancy, we automated the generation of JSDoc comments from existing Markdown documentation.

This involved:

• Standardization for Efficiency: We implemented a standardized structure and format for the Markdown documentation, facilitating easier readability for a developer as well as parsing for an automated parser.
• Building a Custom Script: A dedicated script was developed to parse the Markdown files, extract relevant documentation for each function or component, and automatically prepend it to the corresponding type definitions.
• Integration with Build Process: To ensure automatic updates and maintain a single source of truth, we integrated the JSDoc generation script into the package build process, seamlessly generating up-to-date documentation alongside the code itself.

Below snippet shows a minimized version of the JSdoc generation script:

import fs from "fs";
import path from "path";

import _generate from "@babel/generator";
import _traverse from "@babel/traverse";
import * as babelTypes from "@babel/types";

const traverse = _traverse.default;
const generate = _generate.default;

const buildJsdoc = () => {
  const fileNamesInsideDocs = getFileNameList(path.resolve(DOCS_FOLDER_NAME));
  const typeFileNames = fs.readdirSync(path.resolve(TYPES_FOLDER_NAME));

  syncTypeFiles(EXPORTED_TYPES_FOLDER_NAME);

  const entityTitleToDescMapOfAllFiles = {};

  fileNamesInsideDocs.forEach(docFileName => {
    const fileContent = getFileContent(docFileName);
    const markdownAST = parseMarkdown(fileContent);

    buildEntityTitleToEntityDescMap(
      markdownAST.children,
      entityTitleToDescMapOfAllFiles
    );
  });

  typeFileNames.forEach(typeFileName => {
    const typeFileContent = getFileContent(
      path.join(EXPORTED_TYPES_FOLDER_NAME, typeFileName)
    );
    const typeFileAST = parseTypeFile(typeFileContent);

    typeFileTraverser({
      typeFileName: `${EXPORTED_TYPES_FOLDER_NAME}/${typeFileName}`,
      typeFileAST,
      entityTitleToDescMapOfAllFiles,
      babelTraverse: traverse,
      babelCodeGenerator: generate,
      babelTypes,
    });
  });

  console.log("Successfully added JSDoc comments to type files.");
};

Let's dive deep into the above snippet to understand how the JSDoc generation script works.

Initial Setup and Imports

The script starts with essential imports and variable initializations. This section sets up the necessary tools and libraries to work with file systems and Abstract Syntax Trees (AST).

import fs from "fs";
import path from "path";

import _generate from "@babel/generator";
import _traverse from "@babel/traverse";
import * as babelTypes from "@babel/types";

const traverse = _traverse.default;
const generate = _generate.default;

Gathering File Information

We created a buildJsdoc function to encapsulate the JSDoc generation process. The buildJsdoc function begins by collecting file names from the documentation (DOCS_FOLDER_NAME) and type definitions (TYPES_FOLDER_NAME) folders. It then copies the type definition files to the EXPORTED_TYPES_FOLDER_NAME.

const fileNamesInsideDocs = getFileNameList(path.resolve(DOCS_FOLDER_NAME));
const typeFileNames = fs.readdirSync(path.resolve(TYPES_FOLDER_NAME));
syncTypeFiles(EXPORTED_TYPES_FOLDER_NAME); // copying the files

Building Entity Descriptions Map

The script parses Markdown documentation files to construct a map linking entity titles to their descriptions. This map serves as a bridge between the documentation and the type definitions. We use unified package and remarkParse plugin for parsing Markdown.

const entityTitleToDescMapOfAllFiles = {};
fileNamesInsideDocs.forEach(docFileName => {
  const fileContent = getFileContent(docFileName);
  const markdownAST = parseMarkdown(fileContent);
  buildEntityTitleToEntityDescMap(
    markdownAST.children,
    entityTitleToDescMapOfAllFiles
  );
});

Updating Type Definitions

Next, the script traverses the AST of each type definition file. For each entity, it finds the corresponding description from the map and adds it to the type definition file. The typeFileTraverser function is responsible for traversing the AST of a type definition file and looks for ExportNamedDeclaration nodes. It then prepends the corresponding JSDoc comment to the node.

typeFileNames.forEach(typeFileName => {
  const typeFileContent = getFileContent(
    path.join(EXPORTED_TYPES_FOLDER_NAME, typeFileName)
  );
  const typeFileAST = parseTypeFile(typeFileContent);

  typeFileTraverser({
    typeFileName: `${EXPORTED_TYPES_FOLDER_NAME}/${typeFileName}`,
    typeFileAST,
    entityTitleToDescMapOfAllFiles,
    babelTraverse: traverse,
    babelCodeGenerator: generate,
    babelTypes,
  });
});

The script concludes by logging a success message, indicating the completion of the JSDoc generation process.

Once everything is set up and released, we were able to access both the types and documentation from the code editor itself.

Conclusion

By leveraging the power of type definitions and JSDoc comments, we were able to provide easy access to documentation and significantly enhance the developer experience and streamline the development process. The automated generation of JSDoc comments from Markdown documentation further optimized our workflow, ensuring clarity, consistency and eliminating redundancy.