Posts from September 2011.

Managing spam with google

If you use google to manage your personal domain email I have worked out a hack to have the best spam filter possible.  This article is all about unsolicited junk email and stopping it dead.

Why

I have an irrational need to make sure I get all my email.  So, if I’m e-vited to the latest social event in our neighborhood (wine tasting, happy hour, etc.) but they misspelled my user name, I still want to get the e-vitation. (If they misspell the domain name, sadly there’s not much I can do.) So, in google mail I set up a “catch all” which means that if google doesn’t find a valid user account to deliver the mail message to, it will drop the message into the designated “catch all” user account instead of bouncing it back to the sender as undeliverable.  That’s great for never missing an invitation to drink alcohol with friends, but that’s horrible for trying to manage spam messages.  Spammers no longer have to guess a valid user name, they can send it anywhere to my domain and it’ll wind up in my SPAM folder.  Which isn’t so bad, but I hate SPAM folders.

I hate SPAM folders because I feel the need to go through them to see if legitimate messages were placed in there on accident.  The thought of missing that wine-tasting e-vitation because it had one too many links in the message is personally disquieting.  When you get thousands of SPAM messages a week, searching for that one party invitation is very time consuming, tedious, and really annoying.  So I gave myself a goal, “Minimize the chances of missing an email while also minimizing the amount of spam that made it into the SPAM folder.”

Lofty goal, huh?  I know, I’m going to tackle hunger and world peace next week.

So, I own my own domain.  When you send a message to a user at wiredsage.com, it comes to my email account no matter what the user name was.  Minimizing missed messages isn’t quite that easy, because they can still get delivered, categorized as spam, and lost in the plethora of junk mail waiting to be deleted never to be seen by human eyes.  I had to figure out a way of killing spam before it got to the SPAM folder in google.

White-List, Are you Human, Black-List?

You might be thinking to yourself, what they need is a “white-list” or a list of people who would never send you spam and would always send you good email. With the proliferation of malware, lots of people who would have been on a “white-list” of mine have in the past sent me crap.  Plus, what about the person I don’t know?  The person that is sending me a legitimate job offer, the job offer of my dreams.  I want that email to come right to my Inbox without delay.  So not only is this not a great solution, it’s not offered by google.

I’ve seen some hosting companies send an auto-reply message to the sender, ask them to visit a website, and complete a captcha (validating that you are human), before allowing the email message to be delivered.  That stops spam pretty much in it’s tracks.  It can be defeated though.  I’ve seen spammers use third world data entry workers to complete captchas in other situations before.   And could you imagine a recruiter sending out a bulk email message, about the best tech job ever to be published to a list of highly qualified potential applicants, going through all that crap to deliver one email to one potential employee.  Opportunity only knocks once.  This is not only overkill, it’s also not offered by google.

What I really needed was a “black list”.  A filter that said, if the email meets this criteria, return it to sender as undeliverable.  Ideally, I would be able to use regular expressions on any email field, header, or the body of the message.  I would be able to count how many links were in the email, I would be able to compare the senders to my contact list, I could count the number or recipients were in the message, to create a robust set of rules which would allow me to decide, “this message was too spammy to accept”.  Now note that I want to bounce the messages and return them to sender with an error for the one in a million emails that weren’t spam.  Well, guess what?  Google doesn’t offer this either.

What to do?

I wondered if I could do anything, so I investigated.  In computer lingo I hacked.  I was looking for a way to use the existing infrastructure provided by google in a non-intended way to get my desired result.  So I hacked… and hacked… and hacked…

I found that if you disable an account, the email is then bounced back to the sender as undeliverable.  I also found that you could apply aliases to a user account.  So I made a user called junkmail-01 and gave it a name of “junkmail filter”.  I then opened up my SPAM folder and looked at all the TO: fields to get the email addresses that were being spammed.  I took those usernames out of the SPAM folder and made them aliases to junkmail-01.  I then tried to send email to those email addresses.  Bounced mail.  I had found my black-list.

How it works

Now when I go to a site shopping, like geeks.com, and they want my email address I give them geeks.com@wiredsage.com.  Then if they sell or lose my email address and I start getting emails for “natural male enhancement” I just have to add “geeks.com” as an alias to my junkmail-01 user and I will get no more mail from them ever again.  Well, they betrayed my trust, they don’t deserve to communicate with me.  If I absolutely still need to get email from them, I update the username they have on file to something like “geeks.com-2@wiredsage.com” and then I know how many times they lost my email address and whether I want to trust them with my credit card information in the future.

Junkmail-01?

You might be wondering, why junkmail-01?  Why not just junkmail?  Well, google only allows you to add 20 or so aliases to a user account.  I’ve gone beyond 20 blacklisted email addresses.  So, you just add another user, junkmail-02, disable the account, and start adding more aliases.

The Result

This is very effective because I’m filtering on the destination address, not the source address.  So no matter what bot network is sending the spam, or what domain the spam originates from, if it’s destined for the blacklisted account, it gets bounced.  Every once in a very long while I get a spam message in my SPAM folder in my catch-all account.  I then have to login to the domain management portion of google mail and add an alias to the highest numbered junkmail account.  To stop it from happening again.  I see it as an effective strategy for managing spam with the existing tools google provides.

Closing Thoughts

I think this is a sweet hack.  But it is just that, a hack.  I would love for google to come out with a regular expression based rule filter for their email.