Improving Email Deliverability

08/20/2011

by David Addison

In today's world e-mail marketing plays a significant role in any online business. In our experience e-mail plays a role in one-third to one-half of all e-commerce activity. Unfortunately, many businesses deploy unethical practices when sending e-mail. As a result, Internet service providers (ISPs) are faced with overwhelming amounts of spam. The Radicati Group, a technology market research firm, estimated in May 2009 there were about 1.4 billion e-mail users worldwide. With 2.8 million e-mail messages sent every second, the average e-mail address is sent 176 messages per day. Better than 80% of the messages are spam and viruses. The three major e-mail ISPs, Yahoo!, MSN/Hotmail/Live and AOL/AIM, spend an incredible amount of time and energy to identify and thwart unwanted e-mail. ISPs are constantly putting new systems in place to fight spam, phishing, and e-mail viruses.

Some Definitions

  • Blacklist is a list of e-mail domains and IP addresses that includes senders which should be automatically blocked from delivering e-mail messages, regardless of their content.
  • Bounces are automated electronic mail messages from a recipient mail system notifying the sender of the original message about a delivery problem.
  • CName or Canonical Name is a type of record that is a pseudonym of another canonical name record.
  • Domain Name System (DNS) could be referred to as the phone book for the Internet which translates human-readable domain names into IP addresses and is capable of storing a lot of extra data about the domain infrastructure.
  • Feedback Loop (FBL) is a service that ISPs generally provide to organizations that send large volumes of e-mail.
  • HyperText Markup Language (HTML) is the language for text, images, and objects that comprise Web pages.
  • Internet Protocol (IP) is a numerical label assigned to each device connected to a computer network, such as the Internet, for the purpose of identifying each device and controlling routing and security.
  • Phishing refers to the fraudulent attempt to steal user sensitive information by pretending to be a trustworthy entity, such as your bank, in electronic correspondence.
  • PTR or pointer record is part of a sender’s computer network configuration.
  • Sending Policy Framework prevents spam e-mail and phishing.
  • Simple Mail Transfer Protocol (SMTP) is a set of rules governing transmission of e-mail across the Internet.
  • X-header is a customized graphic header.
  • Whitelist is a list of e-mail domains and IP addresses marked as acceptable and trustworthy.

Getting mail to the user's inbox is not an easy task. Even with reputable major service providers like Constant Contact, AWeber Communications, and Silverpop we find weaknesses and room for improvement. The entire sending process has to be optimized and tailored to meet specific ISP features and requirements. Marketing professionals who both understand the myriad complex factors involved and devote considerable resources to e-mail deliverability provide their clients with competitive advantage over those who ignore e-mail intricacies. Internet marketing firms that constantly leverage and deploy best practices and continuously strive to improve their deliverability are winning the game for their clients.

Here are some of our best practices

  • Content—Most of the filtering systems implement some type of content assessment mechanisms which mark e-mails having content similar to that found in spam e-mails with a red flag. Including too many images in the e-mail, or having too many phrases like Buy Now, Click Here, Free, Sale or Save will negatively affect deliverability to inboxes. Seemingly innocuous words and phrases—unsubscribe, oral medication, percent, percentage, gluten–free—to name a few, present serious problems. The e-mail has to be concise and to the point, with a relevant subject line and an easily accessible unsubscribe link. Many e-mail systems will return e-mail with spam-triggering words in the e-mail header. If you know which tools to use and where to look, however, it is possible to weed out negative content.

  • Bounces—Collecting e-mail bounces and acting on them is crucial to e-mail deliverability. There are multiple types of bounces: hard bounces; soft bounces (i.e., general, DNS failure, mailbox full, message size too large); general bounces; bounce with no e-mail address; mail block bounces (general, known spammer, spam–detected, attachment–detected, relay–denied); auto reply; transient bounce; subscribe request; unsubscribe request; and challenge response.

Spammers are prone to harvesting, stealing, or purchasing bounced e-mail lists. Thus they have high volumes of unknown users (i.e., hard bounces) in their e-mail lists. Most e-mail hosting companies begin rejecting e-mail from a particular sender after a certain threshold of unknown users is reached. For example, if the marketer sends a big e-mail blast and in the first hour of sending 10% of e-mail sent to AOL turns out to be hard bounces, AOL will start rejecting any further e-mails sent to its users even if the remainder of the e-mail list contains valid e-mail addresses.

Closely monitoring and proactively addressing bounces are absolutely crucial to maintaining a clean list and avoiding deliverability issues.

  • Complaints—Most of the big ISPs employ junk mail responder (JMR) programs that allow valid e-mail senders to enroll in feedback loop (FBL) services. FBLs allow senders to receive notifications when recipients click "this is spam," "junk," or other similarly named buttons in their e-mail applications. A high JRM rate (2.9% for Hotmail, 2% for Yahoo!, and one complaint per million messages for SpamCop) is enough to get your e-mail blocked. When custom x-headers are injected into outbound e-mail messages, it is possible to mechanically remove complaining recipients from your e-mail distribution lists and identify which marketing streams generate most of the end-user frustration and concentrate on optimizing those. Aggressively managing FBLs is obligatory for any serious sender.

  • Challenge-response (CR)—CR is a type of spam filtering that automatically sends a reply with a challenge to the sender of an incoming e-mail. In the reply, the sender is asked to perform some action that only a human could perform. CR systems send challenges to unknown senders. Once a sender performs the challenging action all future e-mail from that sender is automatically whitelisted. Earthlink, SpamArrest, iPermitMail and SpamBlock are popular systems for verifying relationships between you and your valued contacts. The users of EarthLink and PeoplePC get challenge-response functionality for free. Not responding to CR e-mails can negatively impact your reputation.

  • List Maintenance—Inactive subscribers are more likely to mark your e-mail as junk. This drives FBL complaints and leads to delivery issues. Moreover, e-mail addresses that were once valid are sometimes repurposed as spam traps or honey pots by e-mail servers for those addresses that haven't been active in a long time (18 months seems to be the threshold). Consistent delivery to addresses with no open or click activity is a sign of sloppy e-mail marketing. At Dirigo, we use a best practice of segmenting the house file by opt-in age and activity. Marketers must willfully shrink their house files by purging inactive addresses.

  • Consistency—Consistency matters! ISPs like to see a consistent pattern to e-mail sending (i.e., volume, speed, from address, etc.). As spammers don't care, consistency tends to be an indicator of responsible e-mail senders. Moreover, if a recipient has not heard from the marketer in a long time, chances are they have already forgotten that they had opted in or think the e-mails from that particular sender are not relevant anymore.

  • E-mail Certification—Several whitelisting services allow you to achieve higher deliverability rates by subscribing to fee-based programs and receiving preferential treatment across major ISPs by complying with the policies of these services. The largest organizations include Return Path's Sender Score Certified (SSC) Program and Goodmail Systems. Dnswl.org is a free whitelisting service that attempts to protect against false positives by including valid senders in their whitelist, but it doesn't come close to fee-based services.

    Participation in the SSC, formerly the Iron Port Bonded Sender Program, is a prerequisite for any serious marketer. Subscribing to SSC is an annual business expense that begins at approximately $2,500. The SSC certification process accepts only legitimate senders using best practices.

    What distinguishes Goodmail from other certification services is that ISPs are contractually required to accept Goodmail–certified e-mail. Other certification services make recommendations to recipient ISPs. These recommendations are factored into the e-mail assessment algorithm. Goodmail is guaranteed to deliver your e-mail. They do this by charging for sent e-mails and sharing the revenue with participating ISPs. At launch in mid-2007 CertifiedEmail participants included Comcast, Cox Communications, Time Warner Cable's Road Runner, and Verizon. They have since added AOL, AT&T, MySpace and Telus.

    In an interesting development in 2009, Goodmail dropped a number of clients and ceased accepting new clients in an effort to improve the quality of their network. Yahoo! discontinued offering preferred delivery to Goodmail–certified e-mail in early 2010, citing that they no longer wished to be contractually obligated to deliver e-mail. Goodmail–certified e-mail has continued to realize good delivery at Yahoo!, however. CertifiedEmail is labeled with Goodmail's blue trust seal, which calls attention to legitimate messages from participating senders. The service also automatically renders links and images. JMR thresholds are extremely low with CertifiedEmail (less than 0.25%). Few legitimate e-mailers qualify for enrollment in their system. The cost of using GoodMail can be quite high. Typically, the expense is worth the cost (e.g., a healthy ROI).

  • Infrastructure—Spammers tend to rent remote servers for just a few days and send unbelievable amounts of e-mails in a short period of time. As a result, the Internet protocol (IP) addresses from which spammers send e-mails get blacklisted in the majority of the spam-detecting systems. The only way for spammers to deliver more e-mails is to change the IP address. Thus they jump from server to server to consistently send e-mail from fresh IP addresses. Senders that send large volumes of e-mail once a month or less often look like spammers. Thus, following best practices is the only way to deliver consistently large e-mail volumes to inboxes.

  • Domain Name System (DNS) Setup—SMTP servers and ISPs implement various systems that allow senders to identify themselves as valid senders on the DNS level and prevent imposters from sending e-mail on their behalf.

    A DNS with a valid Sender ID record (the SPF or Sender Policy Framework, as defined in RFC 4408) contains your policy for sending e-mail, which IPs/domains are allowed to send e-mail on your behalf, and what to do with e-mail that appears to be coming from an approved domain but whose IP doesn't match the policy. If a domain publishes an SPF record, phishers and spammers are far less likely to forge e-mails pretending to be from that domain. This is because forged e-mails are more likely to be caught by ISPs that check the SPF record. Since all the big ISPs rely on SPF technology, an SPF-protected domain is less attractive to spammers.

    You need to have a valid rDNS, or Reverse DNS record (PTR or pointer record, as defined in RFC 1033 and RFC 1912 Section 2.1 ), which points to the IP address from which you are sending e-mail. Computer networks use DNS to determine the IP address associated with a domain name. This process is known as forward DNS resolution. Reverse DNS lookup is the inverse process of this, the resolution of an IP address to its designated domain name. rDNS was originally used for network troubleshooting (e.g., tools such as traceroute, ping, and the received trace header field for SMTP e-mail). These days, rDNS is used to identify spam. Checking the domain name in the rDNS can easily identify e-mail from dynamically assigned addresses, dialup users, or other inexpensive internet services. Owners of such IP addresses typically assign generic rDNS names that don't tie to the IP address. Most mail systems reject e-mail that fails reverse lookup.

    The Domain Keys Identified Mail (DKIM, as defined in RFC 4871) mechanism is another standard in modern e-mail setup that the Dirigo team deploys for our clients. DKIM is a method for e-mail authentication whereby an organization takes responsibility for an e-mail message it has sent in a way that can be validated by a recipient. The e-mail sender adds a digital signature of the e-mail to the message header using one of the public-key cryptography algorithms. The recipient recovers the signer's public key using DNS and verifies that the e-mail message hasn't been altered during transmission. Signing all outgoing e-mail with DKIM signatures and publishing public keys to your DNS demonstrates to SMTP servers that you are authentic and ensures any phishing e-mail gets dropped by ISPs.

  • Segmenting E-mail Campaigns—Separating transactional e-mail from newsletters and marketing e-mail, and sending each segment from a distinct IP address are techniques that allow senders to identify segments with poor reputation and concentrate on improving deliverability for those specific segments without negatively affecting the rest of the e-mail volume.

  • Whitelisting/Blacklisting—Proactively adding the e-mail address/domain/IP to global whitelisting services along with monitoring blacklists and removing your sending e-mail address from their databases will improve your overall deliverability. Motivating users to add your e-mail address to their whitelists is also a good technique to ensure bulletproof e-mail delivery. Cisco IronPort's www.senderbase.org is a great resource for reviewing volume and blacklist data. Other good tools include www.dnsstuff.com, Google Groups abuse, www.spamcop.com, and www.mxtoolbox.com.

  • Avoiding suspicious e-mail addresses—Remove all@domain.com, sales@domain.com, marketing@domain.com, abuse@domain.com, nospam@domain.com and postmaster@domain.com e-mail addresses from your distribution lists. They are either e-mail lists or spam traps.

  • Double Opt-In—A technique for verifying your subscribers with a quick sign-up confirmation e-mail. This process prevents an individual from intentionally registering the e-mail addresses of other people to receive marketing material, and is a requirement for enrolling in major whitelisting services. E-mail marketing has much to do with list size; the larger the list and better your list the more successful your campaign. All of the large e-mail marketing services strongly encourage the use of a double opt-in method. AOL will not whitelist your domain unless you use double opt-in techniques.

    The major obstacle for marketers is that double opt-in drives down list growth. It's hard to grow an e-mail list these days. People, wary of getting lots of unwanted e-mail, just don't subscribe as freely as they did a few years ago. Single opt-ins are legal under the Controlling the Assault of Non-Solicited Pornography and Marketing Act of 2003 (known as CAN-SPAM). Organizations that use double opt-in techniques lose 20–40% of the opt-ins in the confirmation process. All of these reasons make double opt-in a "hard sell." Dirigo strongly recommends double opt-in.

  • Multipurpose Internet Mail Extensions (MIME)—ISP systems are more likely to identify HyperText Markup Language (HTML)-only e-mails as spam. We recommend sending both HTML and plain-text versions simultaneously. This has a significant impact on deliverability of e-mails and demonstrates a best practice.

Connect with Dirigo Design & Development and Connect with E-Marketing at Its Best

Getting e-mail to the inboxes of your customers and prospects is a determined task that requires skill and vigilance. As an experienced Internet marketing organization, Dirigo deploys best practices like those noted above, plus myriad others, to create an environment for optimal inbox placement. We deliver more than 15 million e-mail messages per month. Most of that e-mail is relayed directly from dedicated client servers using proprietary Dirigo methods.

To discuss the ways we can help your organization maximize its inbox placement, as well as its online potential, call us at 207-347-7360. We specialize in abandoned cart e-mails, trigger e-mails, reactivation e-mails, list cleansing, CRM integration, e-mail response management systems and more. We're here to help you connect with your customers and grow your organization.

Principal Author: David Addison
Date of Publication: 08/20/2011
Last Modified: 01/21/2013

Thanks!

Thank you for contacting us!

We'll be in touch!

Back Home ×