BigBinary Blog

We write about Ruby on Rails, React.js, React Native, remote work, open source, engineering and design.

Rails 6.1 adds strict_loading to warn lazy loading associations

This blog is part of our Rails 6.1 series.

Rails 6.1 adds strict_loading mode which can be enabled per record, association, model or across the whole application.

strict_loading mode is an optional setup and it helps in finding N+1 queries.

Let's consider the following example.

class Article < ApplicationRecord
  has_many :comments
end

class Comment < ApplicationRecord
  belongs_to :article
end

Mark a record for strict_loading

When strict_loading mode is enabled for a record then its associations have to be eager loaded otherwise Rails raises ActiveRecord::StrictLoadingViolationError.

Let's see this use case by setting strict_loading mode for an article record.

2.7.2 :001 > article = Article.strict_loading.first
  Article Load (0.2ms)  SELECT "articles".* FROM "articles" ORDER BY "articles"."id" ASC LIMIT ?  [["LIMIT", 1]]
 => #<Article id: 1, title: "First article", content: "First content", created_at: "2020-12-01 07:23:38.446867000 +0000", updated_at: "2020-12-01 07:23:38.446867000 +0000">
2.7.2 :002 > article.strict_loading?
 => true
2.7.2 :003 > article.comments
Traceback (most recent call last):
ActiveRecord::StrictLoadingViolationError (`Comment` called on `Article` is marked for strict_loading and cannot be lazily loaded.)

strict_loading mode forces us to eager load the associated comments by raising the ActiveRecord::StrictLoadingViolationError error.

Let's fix the strict_loading violation error.

2.7.2 :004 > article = Article.includes(:comments).strict_loading.first
  Article Load (0.7ms)  SELECT "articles".* FROM "articles" ORDER BY "articles"."id" ASC LIMIT ?  [["LIMIT", 1]]
  Comment Load (0.2ms)  SELECT "comments".* FROM "comments" WHERE "comments"."article_id" = ?  [["article_id", 1]]
 => #<Article id: 1, title: "First article", content: "First content", created_at: "2020-12-01 07:23:38.446867000 +0000", updated_at: "2020-12-01 07:23:38.446867000 +0000">
2.7.2 :005 > article.comments
 => #<ActiveRecord::Associations::CollectionProxy [#<Comment id: 1, desc: "Great article", article_id: 1, created_at: "2020-12-01 07:23:58.832869000 +0000", updated_at: "2020-12-01 07:23:58.832869000 +0000">  , #<Comment id: 2, desc: "Well written", article_id: 1, created_at: "2020-12-01 07:24:02.853376000 +0000", updated_at: "2020-12-01 07:24:02.853376000 +0000">]>

strict_loading mode on article record automatically sets strict_loading mode for all the associated comments as well.

Let's verify this in Rails console.

2.7.2 :006 > article.comments.all?(&:strict_loading?)
 => true

Mark an association for strict_loading

strict_loading mode can be set up for a specific association.

Let's update our example to see strict_loading in action when it is passed as an option to associations.

class Article < ApplicationRecord
  has_many :comments, strict_loading: true
end

class Comment < ApplicationRecord
  belongs_to :article
end

Let's verify this in Rails console.

2.7.2 :001 > article = Article.first
  Article Load (0.2ms)  SELECT "articles".* FROM "articles" ORDER BY "articles"."id" ASC LIMIT ?  [["LIMIT", 1]]
 => #<Article id: 1, title: "First article", content: "First content", created_at: "2020-12-01 07:23:38.446867000 +0000", updated_at: "2020-12-01 07:23:38.446867000 +0000">
2.7.2 :002 > article.strict_loading?
 => false
2.7.2 :003 > article.comments
Traceback (most recent call last):
ActiveRecord::StrictLoadingViolationError (`comments` called on `Article` is marked for strict_loading and cannot be lazily loaded.)


2.7.2 :004 > article = Article.includes(:comments).first
  Article Load (0.2ms)  SELECT "articles".* FROM "articles" ORDER BY "articles"."id" ASC LIMIT ?  [["LIMIT", 1]]
  Comment Load (0.2ms)  SELECT "comments".* FROM "comments" WHERE "comments"."article_id" = ?  [["article_id", 1]]
 => #<Article id: 1, title: "First article", content: "First content", created_at: "2020-12-01 07:23:38.446867000 +0000", updated_at: "2020-12-01 07:23:38.446867000 +0000">
2.7.2 :005 > article.comments
 => #<ActiveRecord::Associations::CollectionProxy [#<Comment id: 1, desc: "Great article", article_id: 1, created_at: "2020-12-01 07:23:58.832869000 +0000", updated_at: "2020-12-01 07:23:58.832869000 +0000">, #<Comment id: 2, desc: "Well written", article_id: 1, created_at: "2020-12-01 07:24:02.853376000 +0000", updated_at: "2020-12-01 07:24:02.853376000 +0000">]>

Configure strict_loading per model

We can set strict_loading_by_default option per model to mark all of its records and assications for strict_loading.

Let's update our example to set strict_loading_by_default for the Article model.

class Article < ApplicationRecord
  self.strict_loading_by_default = true

  has_many :comments
end

class Comment < ApplicationRecord
  belongs_to :article
end

Let's verify this setting in the Article model.

2.7.2 :001 > article = Article.includes(:comments).first
  Article Load (0.2ms)  SELECT "articles".* FROM "articles" ORDER BY "articles"."id" ASC LIMIT ?  [["LIMIT", 1]]
  Comment Load (0.2ms)  SELECT "comments".* FROM "comments" WHERE "comments"."article_id" = ?  [["article_id", 1]]
 => #<Article id: 1, title: "First article", content: "First content", created_at: "2020-12-01 07:23:38.446867000 +0000", updated_at: "2020-12-01 07:23:38.446867000 +0000">
2.7.2 :002 > article.strict_loading?
 => true
2.7.2 :003 > article.comments.all?(&:strict_loading?)
 => false
2.7.2 :004 > article.comments
 => #<ActiveRecord::Associations::CollectionProxy [#<Comment id: 1, desc: "Great article", article_id: 1, created_at: "2020-12-01 07:23:58.832869000 +0000", updated_at: "2020-12-01 07:23:58.832869000 +0000">, #<Comment id: 2, desc: "Well written", article_id: 1, created_at: "2020-12-01 07:24:02.853376000 +0000", updated_at: "2020-12-01 07:24:02.853376000 +0000">]>

Make strict_loading default across all models

We can make strict_loading default across all models by adding the following line to the Rails configuration file.

config.active_record.strict_loading_by_default = true

Configure strict_loading violations to show only in logs

By default, associations marked for strict loading always raise ActiveRecord::StrictLoadingViolationError for lazy loading.

However, we may prefer to log such violations in our production environment instead of raising errors.

We can add the following line to the environment configuration file.

config.active_record.action_on_strict_loading_violation = :log

Check out pull requests #37400, #38541, #39491 and #40511 for more details.

Dinesh Panda in Rails, Rails 6.1
Jan 06, 2021
Share

Rails 6.1 allows default_scope to be run on all queries

This blog is part of our Rails 6.1 series.

Before Rails 6.1 if a default_scope was defined in a model it would be applied only for select and insert queries. Rails 6.1 adds an option all_queries: true that could be passed to default_scope to make the scope applicable for all queries.

default_scope -> { where(...) }, all_queries: true

Consider the Article class below.

class Article
  default_scope -> { where(organization_id: Current.organization_id) }
end

@article.update title: "Hello World"
@article.delete

The update and delete methods would generate SQL queries as shown below. As we can see that default_scope is missing from these queries.

UPDATE "articles" SET "title" = $1 WHERE "articles"."id" = $2 [["title", "Hello World"], ["id", 146]]

DELETE FROM "articles" WHERE "articles"."id" = $1  [["id", 146]]

In Rails 6.1 we can solve this problem by passing all_queries: true to the default_scope.

class Article
  default_scope -> { where(organization_id: Current.organization_id) }, all_queries: true
end

Then the generated SQL changes to this:

UPDATE "articles" SET "title" = $1 WHERE "articles"."id" = $2 AND "articles"."organization_id" = $3  [["title", "Hello World"], ["id", 146], ["organization_id", 314]]

DELETE FROM "articles" WHERE "articles"."id" = $1 AND "articles"."organization_id" = $2  [["id", 146], ["organization_id", 314]]

Ability to make default_scopes applicable to all queries is particularly useful in the case of multi-tenanted applications, where an organization_id or repository_id is added to the tables to support sharding.

Check out the pull request for more details on this feature.

Unnikrishnan KP in Rails 6.1
Dec 29, 2020
Share

Ruby 3 is released - The list of Ruby 3 features

This blog is part of our Ruby 3.0 series.

For all Rubyists, 2020 was a special year. Why wouldn't it be? Ruby 2 was released in 2013. We have been using Ruby 2.x for almost 7 years and we have been waiting to see Ruby 3 get released. Finally, the wait is over now. Ruby 3.0.0 has been released. It's time to unwrap the gift box and see all the Ruby 3 features we got.

Ruby 3 major updates

The number 3 is very significant in the Ruby 3 release. Be it release version number, making performance 3x faster, or the trio of core contributors(Matz, TenderLove, Koichi). Similarly, there were 3 major goals of Ruby 3: being faster, having better concurrency, and ensuring correctness.

Ruby 3 features

1. Ruby 3 performance

One of the major focuses for Ruby 3 was the performance. In fact, the initial discussion for Ruby 3 was started around it. Matz had set a very ambitious goal of making Ruby 3 times faster.

What is Ruby 3x3?

Before discussing this, let's revisit Ruby's core philosophy.

"I hope to see Ruby help every programmer in the world to be productive, and to enjoy programming, and to be happy." - Matz

About Ruby 3x3, some asked whether the goal was to make Ruby the fastest language? The answer is no. The main goal of Ruby 3x3 was to make Ruby 3 times faster than Ruby 2.

“No language is fast enough.” - Matz

Ruby was not designed to be fastest and if it would have been the goal, Ruby wouldn't be the same as it is today. As Ruby language gets performance boost, it definitely helps our application be faster and scalable.

"In the design of the Ruby language we have been primarily focused on productivity and the joy of programming. As a result, Ruby was too slow." - Matz

There are two areas where performance can be measured: memory and CPU.

CPU optimization

Some enhancements in Ruby internals have been made to improve the speed. The Ruby team has optimized the JIT(Just In Time) compiler from previous versions. Ruby MJIT compiler was first introduced in Ruby 2.6. Ruby 3 MJIT comes with better security and seems to improve web apps’ performance to a greater extent.

CPU optimization

MJIT implementation is different from the usual JIT. When methods get called repeatedly e.g. 10000 times, MJIT will pick such methods which can be compiled into native code and put them into a queue. Later MJIT will fetch the queue and convert them to native code.

Please check JIT vs MJIT for more details.

Memory optimization

Ruby 3 comes with an enhanced garbage collector. It has python's buffer-like API which helps in better memory utilization. Since Ruby 1.8, Ruby has continuously evolved in Garbage collection algorithms.

Automatic Garbage Compaction

The latest change in garbage collection is Garbage Compaction. It was introduced in Ruby 2.7 where the process was a bit manual. But in version 3 it is fully automatic. The compactor is invoked aptly to make sure proper memory utilization.

Objects Grouping

The garbage compactor moves objects in the heap. It groups dispersed objects together at a single place in the memory so that this memory can be used even by heavier objects later.

Memory optimization

2. Parallelism and Concurrency in Ruby 3

Concurrency is one of the important aspects of any programming language. Matz feels that Threads are not the right level of abstraction for Ruby programmers to use.

“I regret adding Threads.” - Matz

Ruby 3 makes it a lot easier to make applications where concurrency is a major focus. There are several features and improvements added in Ruby 3 related to concurrency.

Fibers

Fibers are considered a disruptive addition in Ruby 3. Fibers are light-weight workers which appear like Threads but have some advantages. It consumes less memory than Threads. It gives greater control to the programmer to define code segments that can be paused or resumed resulting in better I/O handling.

Falcon Rack web server uses Async Fibers internally. This allows Falcon to not block on I/O. Asynchronously managing I/O gives a great uplift to the Falcon server to serve requests concurrently.

Fiber Scheduler

Fiber Scheduler is an experimental feature added in Ruby 3. It was introduced to intercept blocking operations such as I/O. The best thing is that it allows lightweight concurrency and can easily integrate into the existing codebase without changing the original logic. It's an interface and that can be implemented by creating a wrapper for a gem like EventMachine or Async. This interface design allows separation of concerns between the event loop implementation and the application code.

Following is an example to send multiple HTTP requests concurrently using Async.

require 'async'
require 'net/http'
require 'uri'

LINKS = [
  'https://bigbinary.com',
  'https://basecamp.com'
]

Async do
  LINKS.each do |link|
    Async do
      Net::HTTP.get(URI(link))
    end
  end
end

Please check fibers for more details.

Ractors (Guilds)

As we know Ruby’s global VM lock (GVL) prevents most Ruby Threads from computing in parallel. Ractors work around the GVL to offer better parallelism. Ractor is an Actor-Model like concurrent abstraction designed to provide a parallel execution without thread-safety concerns.

Ractors allows Threads in different Ractors to compute at the same time. Each Ractor has at least one thread, which may contain multiple fibers. In a Ractor, only a single thread is allowed to execute at a given time.

The following program returns the square root of a really large number. It calculates the result for both numbers in parallel.

# Math.sqrt(number) in ractor1, ractor2 run in parallel

ractor1, ractor2 = *(1..2).map do
  Ractor.new do
    number = Ractor.recv
    Math.sqrt(number)
  end
end

# send parameters
ractor1.send 3**71
ractor2.send 4**51

p ractor1.take #=> 8.665717809264115e+16
p ractor2.take #=> 2.251799813685248e+15

3. Static Analysis

We need tests to ensure correctness of our program. However by its very nature tests could mean code duplication.

“I hate tests because they aren't DRY.” - Matz

To ensure the correctness of a program, static analysis can be a great tool in addition to tests.

The static analysis relies on inline type annotations which aren't DRY. The solution to address this challenge is having .rbs files parallel to our .rb files.

RBS

RBS is a language to describe the structure of a Ruby program. It provides us an overview of the program and how overall classes, methods, etc. are defined. Using RBS, we can write the definition of Ruby classes, modules, methods, instance variables, variable types, and inheritance. It supports commonly used patterns in Ruby code, and advanced types like unions and duck typing.

The .rbs files are something similar to .d.ts files in TypeScript. Following is a small example of how a .rbs file looks like. The advantage of having a type definition is that it can be validated against both implementation and execution.

The below example is pretty self-explanatory. One thing we can note here though, each_post accepts a block or returns an enumerator.

# user.rbs

class User
  attr_reader name: String
  attr_reader email: String
  attr_reader age: Integer
  attr_reader posts: Array[Post]

  def initialize: (name: String,
                   email: String,
                   age: Integer) -> void

  def each_post: () { (Post) -> void } -> void
                   | () -> Enumerator[Post, void]
end

Please check RBS gem documentation for more details.

Typeprof

Introducing type definition was a challenge because there is already a lot of existing Ruby code around and we need a tool that could automatically generate the type signature. Typeprof is a type analysis tool that reads plain Ruby code and generates a prototype of type signature in RBS format by analyzing the methods, and its usage. Typeprof is an experimental feature. Right now only small subset of ruby is supported.

“Ruby is simple in appearance, but is very complex inside, just like our human body.” - Matz

Let's see an example.

# user.rb
class User

  def initialize(name:, email:, age:)
    @name, @email, @age = name, email, age
  end

  attr_reader :name, :email, :age
end

User.new(name: "John Doe", email: 'john@example.com', age: 23)

Output

$ typeprof user.rb

# Classes
class User

  attr_reader name : String
  attr_reader email : String
  attr_reader age : Integer

  def initialize : (name: String,
                    email: String,
                    age: Integer) -> [String, String, Integer]

end

Other Ruby 3 features and changes

In the 7-year period, the Ruby community has seen significant improvement in performance and other aspects. Apart from major goals, Ruby 3 is an exciting update with lots of new features, handy syntactic changes, and new enhancements. In this section, we will discuss some notable features.

“We are making Ruby even better.” - Matz

One-line pattern matching syntax change

Previously one-line pattern matching used the keyword in. Now it's replaced with =>.

Ruby 2.7
  { name: 'John', role: 'CTO' } in {name:}
  p name # => 'John'
Ruby 3.0
  { name: 'John', role: 'CTO' } => {name:}
  p name # => 'John'

Find pattern

The find pattern was introduced in Ruby 2.7 as an experimental feature. This is now part of Ruby 3.0. It is similar to pattern matching in Elixir or Haskell.

users = [
  { name: 'Oliver', role: 'CTO' },
  { name: 'Sam', role: 'Manager' },
  { role: 'customer' },
  { name: 'Eve', city: 'New York' },
  { name: 'Peter' },
  { city: 'Chicago' }
]

users.each do |person|
  case person
  in { name:, role: 'CTO' }
    p "#{name} is the Founder."
  in { name:, role: designation }
    p "#{name} is a #{designation}."
  in { name:, city: 'New York' }
    p "#{name} lives in New York."
  in {role: designation}
    p "Unknown is a #{designation}."
  in { name: }
    p "#{name}'s designation is unknown."
  else
    p "Pattern not found."
  end
end

"Oliver is the Founder."
"Sam is a Manager."
"Unknown is a customer."
"Eve lives in New York."
"Peter's designation is unknown."
"Pattern not found."

Endless Method definition

This is another syntax enhancement that is optional to use. It enables us to create method definitions without end keyword.

def: increment(x) = x + 1

p increment(42) #=> 43

Except method in Hash

Sometimes while working on a non Rails app I get undefined method except. The except method was available only in Rails. In Ruby 3 Hash#except was added to Ruby itself.

user = { name: 'Oliver', age: 29, role: 'CTO' }

user.except(:role) #=> {:name=> "Oliver", :age=> 29}

Memory View

This is again an experimental feature. This is a C-API that will allow extension libraries to exchange raw memory area. Extension libraries can also share metadata of memory area that consists of shape and element format. It was inspired by Python’s buffer protocol.

Arguments forwarding

Arguments forwarding (...) now supports leading arguments.

It is helpful in method_missing, where we need method name as well.

def method_missing(name, ...)
  if name.to_s.end_with?('?')
    self[name]
  else
    fallback(name, ...)
  end
end

Other Notable changes

  • Pasting in IRB is much faster.
  • The order of backtrace had been reversed. The error message and line number are printed first, rest of the backtrace is printed later.
  • Hash#transform_keys accepts a hash that maps old keys with new keys.
  • Interpolated String literals are no longer frozen when # frozen-string-literal: true is used.
  • Symbol#to_proc now returns a lambda Proc.
  • Symbol#name has been added, which returns the symbol's name as a frozen string.

Many other changes can be checked at Ruby 3 News for more details.

Transition

A lot of core libraries have been modified to fit the Ruby 3 goal needs. But this doesn't mean that our old applications will suddenly stop working. The Ruby team has made sure that these changes are backward compatible. We might see some deprecation warnings in our existing code. The developers can fix these warnings to smoothly transition from an old version to the new version. We are all set to use new features and get all new performance improvements.

Conclusion

With great improvements in performance, memory utilization, static analysis, and new features like Ractors and schedulers, we have great confidence in the future of Ruby. With Ruby 3, the applications can be more scalable and more enjoyable to work with. The coming 2021 is not just a new year but rather a new era for all Rubyists. We at BigBinary thank everyone who contributed towards the Ruby 3 release directly or indirectly.

Happy Holidays and Happy New Year folks!!!

Datt Dongare in Ruby, Ruby 3
Dec 25, 2020
Share

Catch 404 urls in Next.js and write them to firebase

We recently jumped on the Jamstack bandwagon and moved our BigBinary website to use next.js. We also migrated BigBinary Blog to using next.js.

In the process of migration we knew we might have missed handling a few urls. We wanted to know all the urls which are resulting in 404 now.

Tradtionally a static site is not able to catch all the 404s. However with next.js we can capture the urls resulting in 404 and we can write those urls to firebase.

Setting up Firebase

Get Started with Firebase and create an account. Add a project there and then add a "Web app" inside that project. After that, you will find web app’s Firebase configuration something like this.

var firebaseConfig = {
apiKey: “XXXXXXXXXXXXXXXXXXXXXXXX”,
authDomain: “test-XXXX.firebaseapp.com”,
databaseURL: “https://test-XXXX-default-rtdb.firebaseio.com",
projectId: “test-XXXX”,
storageBucket: “test-XXX.appspot.com”,
messagingSenderId: “00000000000”,
appId: “1:00000000:web:XXXXX00000XXXXXXX”
};

Edit rules in Rules section like this.

{
  "rules": {
    ".read": false,
    ".write": true
  }
}

Creating custom 404

To create a custom 404 page create a pages/404.js file. At build time this file is statically generated and would serve as the 404 page for the application. This page would look like this.

import { useEffect } from "react";
import firebase from "firebase";

export default function Custom404() {
  useEffect(() => {
    const firebaseConfig = {
    apiKey:XXXXXXXXXXXXXXXXXXXXXXXX,
    authDomain: “test-XXXX.firebaseapp.com”,
    databaseURL: “https://test-XXXX-default-rtdb.firebaseio.com",
    projectId: “test-XXXX,
    storageBucket: “test-XXX.appspot.com”,
    messagingSenderId:00000000000,
    appId:1:00000000:web:XXXXX00000XXXXXXX    }

    firebase.initializeApp(firebaseConfig).database().ref().child("404s").push(window.location.href);
  }, []);

  return <h1>404 - Page Not Found</h1>;
}

Now all the 404s will be caught and would be written to the firebase.

Piyush Sinha in Next.js
Dec 23, 2020
Share

Rails 6.1 adds where.associated to check association presence

This blog is part of our Rails 6.1 series.

Rails 6.1 simplifies how to check whether an association exists by adding a new associated method.

Let's see an example of it.

class Account < ApplicationRecord
  has_many :users, -> { joins(:contact).where.not(contact_id: nil) }
end

This will return all users with contacts. If we rephrase that sentence then we can say that "this will return all users who are associated with contacts".

Let's see how we can do the same with the new associated method.

class Account < ApplicationRecord
  has_many :users, -> { where.associated(:contact) }
end

Now we can see that the usage of associated decreases some of the syntactic noise we saw in the first example. This method is essentially a syntactic sugar over the inner_joins(:contact).

Check out the pull request for more details.

Nithin Krishna in Rails, Rails 6.1
Dec 18, 2020
Share

Subscribe to our newsletter