Effective enqueuing of background jobs

TLDR;

We can increase efficiency of the background jobs by meticulously enqueuing only those jobs which are actually going to be executed and avoid jobs getting enqueued and discarded immediately.

Let's say we have a background job to send a webhook notification to Slack.

# app/services/events_service.rb

class EventsService
  def process
    @event = create_event
    SlackJob.perform_later(@org, @event)
  end
end

# app/jobs/slack_job.rb

class SlackJob < ApplicationJob
  def perform(org, payload)
    if org.slack_enabled?
      SendSlackNotification.new(org, payload).process
    end
  end
end

The job is enqueued for sending the notification and then inside the job we are checking if the organization has enabled slack or not. Only if slack is enabled then the job will be executed. But it still gets enqueued in the queue and the worker process has to pick it up and start executing. Only then it realizes that the job does not satisfy required conditions and discards it.

This means that each time the SlackJob gets enqueued and gets picked up for execution by the worker process.

A better approach will be to enqueue only the jobs that are going to be actually executed. This will avoid unnecessarily enqueuing jobs which are going to be discarded immediately.

# app/services/events_service.rb

class EventsService
  def process
    @event = create_event
    if @org.slack_enabled?
      SlackJob.perform_later(@org, @event)
    end
  end
end

# app/jobs/slack_job.rb

class SlackJob < ApplicationJob
  def perform(org, payload)  
    SendSlackNotification.new(org, payload).process
  end
end

This pattern should be used in cases where the decision of whether to execute the job or not too complex and does not depend on too many entities. If the job has to take decision based on complex logic then it can he handled in the job itself instead of doing it before enqueuing the job. But for simple conditionals, we can check them beforehand to avoid enqueuing of the job and immediately discarding.  

Interested in knowing more about my thoughts on web programming using Ruby on Rails? Subscribe here or follow me on Twitter.