We sent newsletter campaigns from our Rails app. One of the main requirement of such campaign is how many emails bounced?. We need to track all bounced emails and map them again to a specific campaign because there will be multiple campaigns going on all the time.
To track bounced emails efficiently, we need to set unique return path for every recipient. This technique is called
Variable Envelope Return Path or VERP
Postfix supports VERP with
-V switch. For eg.
sendmail.postfix -V -f bounced
will generate return path for
[email protected] as
If you want more control and more information like from which campaign this email was sent, we need to send more information in the return-path.
So it is best to generate our own return path pattern and parse it once it is received. We generated a pattern where return-path will generated from all the required information for tracking. This pattern was given to
mail( from: '[email protected]', to: '[email protected]', return_path: generate_verp_pattern )
So now the first problem was solved. We were sending unique return-path for every email that was going out from the system.
Next part is to track it once it bounces and update database.
Postfix allows piping incoming email to a particular address to a script. So if we pipe all incoming emails to
[email protected] to our script then we can parse the incoming address and update database.
For this, we have to edit
/etc/postfix/aliases)file as follows:
This means all incoming emails to
[email protected] will be piped to our script. This actually means that the whole email with body, headers, attachments etc is forwarded to our script.
/etc/aliases file is a text file that is used by postfix as a table to redirect mail for local recipients. To rebuild this table after a change, we need to run
This will rebuild the table for postfix. Now we have completed the second part of the process.
The shell script will get the bounced email content now. There can be multiple bounced emails generated at the same time. So we can’t directly pass them to rails runner scripts or rake tasks. Because that will kill our server by launching multiple rails instances. Instead we need to use some background tasks mechanism.
We were already using
resque, so decided to use it for bounced emails also. So a resque worker will actually update the database. Our script just has to enqueue the job for resque.
We broke this enqueuing process into two parts.
First - A shell script which will use correct RVM Ruby version and call the ruby script.
Second - A Ruby script which will enqueue the job to resque.
So the shell script looked like -
#!/bin/bash rvm use [email protected]_gemset ruby /path/to/ruby/script
And Ruby script looked like
require 'rubygems' require 'resque' # Adds the incoming bounced_email to background job class BouncedEmail def initialize(content) Resque.enqueue_to(:bounced_email_receiver, 'BouncedEmailReceiver', content) end BouncedEmail.new($stdin.read)
We had to go in 2 steps here because we had multiple apps using multiple rubies on same server. If you have only one ruby then you can make a executable ruby script instead of shell script which decides which ruby to use.
Now its upto resque worker to parse the content and update database.
For that, we used bounced_email gem which detects lot of things such as
type of failure etc. As it is integrated with
which was unique pattern generated by us only) and were able to parse it to update the database. With
bounced_email we got some more relevant information for free :)