In the world of web development, conventions often differ between backend and frontend technologies. This becomes evident when comparing variable naming case conventions used in Ruby on Rails (snake case) and JavaScript (camel case). At neeto, this difference posed a major hurdle: the requirement for manual case conversion between requests and responses. As a result, there was a significant amount of repetitive code needed to handle this conversion.
Here’s a snippet illustrating the issue faced by our team:
1// For requests, we had to manually convert camelCase values to snake_case. 2const createUser = ({ userName, fullName, dateOfBirth }) => 3 axios.post("/api/v1/users", { 4 user_name: userName, 5 full_name: fullName, 6 date_of_birth: dateOfBirth, 7 }); 8 9// For responses, we had to manually convert snake_case values to camelCase 10const { 11 user_name: userName, 12 full_name: fullName, 13 date_of_birth: dateOfBirth, 14} = await axios.get("/api/v1/users/user-id-1");
This manual conversion process consumed valuable development time and introduced the risk of errors or inconsistencies in data handling.
To streamline our workflow and enhance interoperability between the frontend and backend, we made a decision to automate the case conversion.
Implementing automatic case conversion
Implementing automatic case conversion across Neeto products required a thoughtful approach to minimize disruptions and ensure a smooth transition. Here's how we achieved this goal while minimizing potential disruptions:
1. Axios Interceptors for Recursive Case Conversion
We created a pair of Axios interceptors to handle case conversion for requests and responses. The interceptors were designed to recursively convert the cases, managing the translation between snake case and camel case as data traveled between the frontend and backend. This smooth transition simplified the workflow, cutting out the requirement for manual case conversion in most situations.
2. Custom Parameters to Control Case Conversion
To do a smooth rollout without breaking any products and due to certain special APIs requiring specific case conventions due to legacy reasons or external dependencies, we introduced custom parameters transformResponseCase and transformRequestCase within Axios. These parameters allowed developers to opt-out of the automatic case conversion for specific API endpoints. By configuring these parameters appropriately, we prevented unintentional case conversions where needed, maintaining compatibility with APIs that required different conventions.
This is how we crafted our axios interceptors:
1import { 2 keysToCamelCase, 3 serializeKeysToSnakeCase, 4} from "@bigbinary/neeto-cist"; 5 6// To transform response data to camel case 7const transformResponseKeysToCamelCase = response => { 8 const { transformResponseCase = true } = response.config; 9 10 if (response.data && transformResponseCase) { 11 response.data = keysToCamelCase(response.data); 12 } 13 14 return response; 15}; 16 17// To transform error response data to camel case 18const transformErrorKeysToCamelCase = error => { 19 const { transformResponseCase = true } = error.config ?? {}; 20 21 if (error.response?.data && transformResponseCase) { 22 error.response.data = keysToCamelCase(error.response.data); 23 } 24 25 return error; 26}; 27 28// To transform the request payload to snake_case 29const transformDataToSnakeCase = request => { 30 const { transformRequestCase = true } = request; 31 32 if (!transformRequestCase) return request; 33 34 request.data = serializeKeysToSnakeCase(request.data); 35 request.params = serializeKeysToSnakeCase(request.params); 36 37 return request; 38}; 39 40// Adding interceptors 41axios.interceptors.request.use(transformDataToSnakeCase); 42axios.interceptors.response.use( 43 transformResponseKeysToCamelCase, 44 transformErrorKeysToCamelCase 45);
Note that keysToCamelCase, serializeKeysToSnakeCase are methods from our open source pure utils library @bigbinary/neeto-cist.
While rolling out the change to all products, we wrote a JSCodeShift script to automatically add these flags to every Axios API requests in all Neeto products to ensure that nothing was broken due to it. Then the team had manually went through the code base and removed those flags while making the necessary changes to the code.
After the change was introduced the API code was much cleaner without the boilerplate for case conversion.
1// Request 2const createUser = ({ userName, fullName, dateOfBirth }) => 3 axios.post("/api/v1/users", { userName, fullName dateOfBirth }) 4 5// Response 6const { userName, fullName, dateOfBirth } = await axios.get("/api/v1/users/user-id-1");
Pain points
In our work towards automating case conversion within neeto, we encountered several pain points.
1. Manual work is involved
During the rollout phase of our automated case conversion solution, there was an unavoidable requirement for manual intervention. As we transitioned existing code bases to incorporate the new mechanisms for automatic case conversion within Axios, each Axios call needed adjustment to remove the manual case conversion codes written before.
This stage demanded some manual work from our development teams. They updated and modified existing Axios requests across multiple projects to ensure they aligned with the new automated case conversion mechanism. While this manual effort temporarily increased workload, it was a necessary step to implement the automated solution effectively across neeto.
This phase highlighted the importance of a structured rollout plan and meticulous attention to detail. Despite the initial manual workload, once the changes were applied uniformly across the codebase, the benefits of automated case conversion quickly became evident, significantly reducing ongoing manual efforts and improving the overall efficiency of our development process.
2. Serialization Issues
As our initial implementation of automated case conversion, we used keysToSnakeCase method which recursively transformed all the keys to snake case for a given object. It internally used transformObjectDeep function to recursively traverse through each key-value pair inside an object for transformation.
1import { camelToSnakeCase } from "@bigbinary/neeto-cist"; 2 3const transformObjectDeep = (object, keyValueTransformer) => { 4 if (Array.isArray(object)) { 5 return object.map(obj => 6 transformObjectDeep(obj, keyValueTransformer, objectPreProcessor) 7 ); 8 } else if (object === null || typeof object !== "object") { 9 return object; 10 } 11 12 return Object.fromEntries( 13 Object.entries(object).map(([key, value]) => 14 keyValueTransformer( 15 key, 16 transformObjectDeep(value, keyValueTransformer, objectPreProcessor) 17 ) 18 ) 19 ); 20}; 21 22export const keysToSnakeCase = object => 23 transformObjectDeep(object, (key, value) => [camelToSnakeCase(key), value]);
However, this recursive transformation approach led to a serialization issue, especially with objects that required special treatment, such as dayjs objects representing dates. The method treated these objects like any other JavaScript object, causing unexpected transformations and resulting in invalid payload data in some cases.
To mitigate these serialization issues and prevent interference with specific object types, we enhanced the transformObjectDeep method to accommodate a preprocessor function for objects before the transformation:
1const transformObjectDeep = ( 2 object, 3 keyValueTransformer, 4 objectPreProcessor = undefined 5) => { 6 if (objectPreProcessor && typeof objectPreProcessor === "function") { 7 object = objectPreProcessor(object); 8 } 9 10 // Existing transformation logic 11};
This modification allowed us to serialize objects before initiating the transformation process. To facilitate this, we introduced a new method, serializeKeysToSnakeCase, incorporating the object preprocessor. For specific object types requiring special serialization, such as dayjs objects, we leveraged the built-in toJSON method, allowing the object to transform itself to its desired format, such as a date string:
1import { transformObjectDeep, camelToSnakeCase } from "@bigbinary/neeto-cist"; 2export const serializeKeysToSnakeCase = object => 3 transformObjectDeep( 4 object, 5 (key, value) => [camelToSnakeCase(key), value], 6 object => (typeof object?.toJSON === "function" ? object.toJSON() : object) 7 );
This resolved the serialization issue for the request payloads. Since the response is always in JSON format, all values are objects, arrays, or primitives. It won't contain such 'magical' objects. So we need this logic only for request interceptors.
Conclusion
In simplifying our web development workflow at neeto, automating case conversion proved crucial. Despite challenges during implementation, refining our methods strengthened our system. By streamlining data translation and overcoming hurdles like serialization issues, we've improved efficiency and compatibility across our ecosystem.
If you're starting a new project, adopting automated case conversion mechanisms similar to what we've built in Axios can offer significant advantages. Implementing these standards from the beginning promotes consistency and simplifies how data moves between your frontend and backend systems. Introducing these practices early in your project's lifecycle helps sidestep the difficulties of adjusting existing code and establishes a unified convention throughout your project's structure.
For existing projects, adopting automated case conversion might initially come with a cost. Introducing these changes requires careful planning and execution to minimize disruptions. The rollout process might necessitate manual updates across various parts of the codebase, leading to increased workload and potential short-term setbacks.