Never send email from a web page…

… synchronously. Recently I had to deal with nonsensical email bounces from a site I occasionally update, errors of the type:

unrouteable mail domain "hotmail.com"

Of course, domains such as hotmail.com, yahoo.com, etc. are perfectly capable of receiving all the messages one wants to throw at them, so the problem was in our own mail server. It turns out that the mail server returns this bogus message when you've tried to send too many message over a certain period (say for example more than 20 pieces per minute, the actual limit may vary from configuration to configuration), and to make matters worse, the limit is a global one: all the sites hosted on the same box concur to the limit, so if site A sends 15 pieces and site B sends 5, then site C would not be able to send email at all in that minute.

My site was indeed sending messages synchronously, after a form submission, and it tried to send more than 250 pieces each time; this also made the user interface completely unresponsive, at the point that some users repeatedly clicked on the "Send" button, aggravating the problem even further. Also, all the bounces arrived in my inbox…

Here's how I solved the problem: I created a new database table EMAIL_MESSAGES to store the messages with the following structure

create table EMAIL_MESSAGES (
  id integer not null primary key,
  sender varchar2(200),
  reply_to varchar2(200),
  destination varchar2(200),
  subject varchar2(200),
  body clob,
  creation_date date default sysdate,
  sent_date date,
  sent_flag integer default 0
);

and then changed the form submission code to insert into the table a copy of the message for each recipient. I created a script that selects X messages maximum with sent_flag = 0 and sends each copy, setting the flag to 1 afterwards: the trick is scheduling the script with the correct frequency, not too often to avoid crossing the limits and not too rarely to freak out people not receiving emails. We also can selectively resend only the messages that bounced and not the whole lot, and the user interface has greatly improved since it returns immediately after the form submission. Why this solution is scalable? Because at any time I could easily schedule another instance of the script to send via a different SMTP server…

Old messages do not stay in the database indefinitely, actually the first thing the script does is deleting messages older than 15 days, irrespective of whether they've been sent successfully or not.

While my actual implementation was in PHP/MySQL (difficult to find a worse combination…) the principles of this solution are applicable to any language and database: for example Tom Kyte wrote a while back how to do the same thing with the Oracle database, ATG has the TemplateEmailSender class and many others undoubtedly exist out there. Make sure you look for one of them the next time you have to send some emails.