Stop using manual preloading in your Rails application; use this instead.

It’s a popular internet recommendation to use preloading in Ruby on Rails applications to eliminate the N+1 query problem. It may seem like a blessing initially, but it has drawbacks. Gladly, there is a much better solution I want to share with you.

No more includes

Before we dive into the ultimate solution, let’s quickly explore the current approach’s downsides.

If you know what you don’t like about preloading, scroll right to the last section for a much better solution.

Imagine we have the following query somewhere in a controller or any other place according to your architecture.

User.includes(:account, :payments, referral: :user)

This will execute several SQL queries.

SELECT * FROM users
SELECT * FROM accounts WHERE user_id = ?
SELECT * FROM payments WHERE user_id = ?
SELECT * FROM referrals WHERE user_id = ?
SELECT * FROM users WHERE id = ?

What do you think is wrong with that?

Potential unnecessary loading

What if we need to show referrals only for users without an account?

- if user.acccount.nil?
  = user.referral.user.full_name

With initial includes, we will query referrals and users tables even when the block is never reached, e.g., all users are missing an account. This brings an unnecessary loading that we would like to avoid as well.

Existing approaches to resolve the issue take a lot of work to maintain.

users = User.includes(:account, :payments)

# Check if any user has an account
if users.any?(&:account)
  # Keep in mind that this class changed its API in ActiveRecord 7.
  ActiveRecord::Association::Preloader.new(records: users, associations: {referral: :user}).call
end

So now, you must keep the consistency of your condition logic when you load and present the data. Even with extracting the logic and reusing it in both places, it’s still hard to ensure consistency as more places could be involved later.

Manual maintenance of zero N+1

Assuming that no code is unchanged by fixing bugs, refactoring or developing new features, you must keep your eyes sharp on N+1.

For example, one day, you want to show users’ countries on the view. So you adjust the view, and with the help of tools like bullet, you find a new N+1 issue. You go back to the loading logic and update it.

User.includes(:account, :payments, :country)

“The job is done” — you think. No more N+1, so you are free to take on another task.

While it may look like the case, unfortunately, it isn’t always true. What if the magic tool didn’t spot N+1? What if your testing/development data wasn’t producing N+1? There are many reasons for what could go wrong.

One more frequent thing is keeping preloading when it isn’t needed anymore. Let’s say one day you decided to stop showing account information. So an engineer updates the view by removing a couple of lines.

Now again, a lot of things may happen:

• you don’t have an automatic tool to find redundant preloading
• you are afraid to remove preloading as it might be used for another place
• you were focused on business value and forget to think about the technical part

While reasoning I mentioned can be questionable by some, my main point is simple: why not make developing easier having the same quality/performance?

Do you like the topic so far? I would be delighted if you subscribe:

• Medium
• LinkedIn
• Twitter
• GitHub

It’s time for the ultimate solution I promised at the beginning of the article.

When we first query for users, we could bind them with a shareable context so that every user from the group knows the whole batch at any moment.

users = User.all

users.each { |user| user.batch_group = users }

Now, all we need to do before querying the associations is to preload them for every object in the batch.

unless user.association(:account).loaded?
  ActiveRecord::Association::Preloader.new(records: user.batch_group, associations: :account).call
end

user.account

The idea is so trivial and yet so powerful. Without a surprise, tools are available for you out of the box, so you don’t need to think about preloading anymore.

• ArLazyPreload
• Goldiloader

With them, you don’t need ever to remember about includes.

It just works!

The configuration is simple (steps for ArLazyPreload):

• add the gem to your Gemfile (gem ‘ar_lazy_preload’)
• enable it globally (ArLazyPreload.config.auto_preload = true)
• that’s it.

I encourage you to check them out and give them a try. I believe you will like it a lot!

Thank you for your time! Please subscribe for further topics about #ruby and #rails.

• Medium
• LinkedIn
• Twitter
• GitHub