Sender Policy Framework [SPF]

Introduction
Typical email headers
Verify SPF entries
Recent news/commentary
Other helpful links

Introduction

SPAM has become a real nuisance in addition to wasting limited Internet resources. One effort underway to help reduce the number of spam message received by individuals is to implement Sender Policy Framework [SPF]. Wikipedia provides good back ground and basic information on SPF. Another definition at whatis.com is also helpful. Both other links to other technical terms/concepts used in their explanations.

Many of us should be interested in SPF, because we work with customers who will be affected. On a personal level, we all deal with individual ISPs, whose implementation of SPF may well affect our in-boxes.

While many 'open standards' initiatives sound good in principle, many languish because the 'big boys' ignore them. This is not true with SPF, particularly because AOL announced its intention to adopt this standard 'soon'. The first alert was back in January 22, 2004. AOL has mentioned this several times, as recently as June 9, 2004. However, a critical consideration, is that as of this writing [August 27, 2004], AOL has yet to publish a date when it will begin enforcing SPF for incoming messages. Microsoft, owner of HotMail, however has set October 1, 2004 as the date they will start using SPF.

Although AOL and Microsoft's announcements have caused the biggest stir, EarthLink and Yahoo are also part key players in the spam war and co-founders of the Anti-Spam Technical Alliance (ASTA). This June 25, 2004 release chronicles their efforts.

AOL has a long tradition of 'advising' other email senders about AOL's preferences. AOL's SPF page and links are worth a look. As often happens with efforts to implement new standards, vendors try to protect and/or sway organizations to their own products. Microsoft has proposed its own 'Caller ID' approach. This press release from August 16, 2004, provides ancillary information about how AOL, SPF, 'Caller ID', and the AOL WhiteList may interact.

A joint version of SPF and 'Caller ID', known as 'Sender ID' has been sent to the Internet Engineering Task Force according to this June 25, 2004 press release. Initial indications are that this is moving forward based on the August 4, 2004 report.

Typical email headers

Most of us have our email software set to display only the basic header information. Looking at the full header information, makes it easier to understand what SPF will be dealing with. As you may notice, there is considerable variety in header entities.

Sample message #1:

From - Thu Aug 12 11:35:16 2004
X-UIDL: 20040812104123s11004vvp8e000035
X-Mozilla-Status: 0001
X-Mozilla-Status2: 00000000
Received: from out007.topica-platinum-w.com ([65.77.104.27])
          by sccrmxc11.comcast.net (sccrmxc11) with SMTP
          id <20040812104122s110097jqne>; Thu, 12 Aug 2004 10:41:23 +0000
X-Originating-IP: [65.77.104.27]
To: afp-l@topica.com
From: afp-l@topica.com
Subject: Digest for afp-l@topica.com, issue 1273
Date: Thu, 12 Aug 2004 03:41:20 -0700
Message-ID: <477227193-1463792126-1092307280@boing.topica.com>
Errors-To: 
X-Topica-Id: <1092307279.svc001.637.1142696>
List-Help: 
List-Unsubscribe: 
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable

Sample message #2:

From - Sun Aug 15 22:55:42 2004
X-UIDL: UID13075-1082504892
X-Mozilla-Status: 0001
X-Mozilla-Status2: 00000000
Return-path: 
Envelope-to: d30008880-mbotos@pop.purehost.com
Delivery-date: Sun, 15 Aug 2004 18:56:45 -0400
Received: from mail01.yourhostingaccount.com ([10.1.1.72] ident=exim)
		 by scan02.yourhostingaccount.com with spamscanpop (Exim)
		 id 1BwTvh-0000ZG-NI
		 for d30008880-mbotos@pop.purehost.com; Sun, 15 Aug 2004 18:56:45 -0400
Received: from mail01.yourhostingaccount.com ([10.1.1.72] helo=inc-mail01.yourhostingaccount.com)
		 by scan02.yourhostingaccount.com with esmtp (Exim)
		 id 1BwTvh-0000ZD-H8
		 for mbotos@botos.com; Sun, 15 Aug 2004 18:56:45 -0400
Received: from dbyec-ns1.pb.com ([152.144.128.6] helo=dbyec-stage1.pb.com)
		 by mail01.yourhostingaccount.com with esmtp (Exim)
		 id 1BwTvf-0004G5-Bv
		 for mbotos@botos.com; Sun, 15 Aug 2004 18:56:43 -0400
Received: from d3prodappserv4 (dby-lb2-snat.ct.pb.com [152.144.124.73])
		 by dbyec-stage1.pb.com (8.12.8/8.12.8) with ESMTP id i7FMp80w001854
		 for ; Sun, 15 Aug 2004 18:51:08 -0400 (EDT)
Date: Sun, 15 Aug 2004 18:51:08 -0400 (EDT)
Message-ID: Rome-437579-payroll_do_not_reply@pb.com
From: payroll_do_not_reply@pb.com
Reply-To: payroll_do_not_reply@pb.com
To: mbotos@botos.com
Subject: Your Pitney Bowes Statement
Errors-To: payroll_do_not_reply@pb.com
Mime-Version: 1.0
Content-Type: multipart/mixed; 
		 boundary="----=_Part_43951_5024564.1092610599830"
X-Mailer: RomeMessage Id=437579
X-OriginalIP: 152.144.128.6
X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on 
		 mail.yourhostingaccount.com
X-Spam-Result: flush
X-Spam-Status: No, hits=3.4 required=10.0 tests=CLICK_BELOW,INVALID_MSGID,
		 NO_REAL_NAME autolearn=no version=2.60
X-Spam-Level: ***

Some key points you may have missed in the discussion about implementing SPF:

Receiving email servers [not your individual email software] will be upgraded to begin SPF checking
SPF checking involves grabbing the domain name in the email header, although not necessarily the FROM: line of the email messages
The SPF check, is really a DNS query by the local email server to validate whether the IP address of the email server that placed the message on the Internet is listed as valid email server for the domain name
If determined not to be valid, the local email server will simply trash the message
Actually the local email server has other options like marking the message as suspect, etc. It is likely that end users will receive no notification for SPF trashed messages. You may also see local actions referred to as 'local policy'.
SPF implementation may affect business owner who employ 3rd parties to send out email messages on their behalf from other domains. The business owners are responsible for adding the IP addresses of any other legitimate email servers who will be sending messages on their behalf.

One more fly in the ointment: There are multiple implementations of SPF and not all of them use the exact same header records. 'Policy' at each receiving server may be slightly different, but all rely on the fact that domain owner must publish an SPF record.

Verifying SPF entries

So how will you know if your update took effect and is correct?

The DNSstuff site provides a handy 'SPF Tester' tool. The tool lets you enter a hypothetical email address and a hypothetical IP address from which the message might be sent. Then, the tool does a lookup to process the information and display the results. The tool can be found at: http://www.dnsstuff.com/pages/spf.htm.

Recent news/commentary

This section contains recent news and/or commentary on the implementation effort of SPF. This is not intended to be a exhaustive list. For that, go 'google' SPF.

Popular spam fighter's effectiveness questioned -Study shows SPF's loopholes - September 4, 2004 - Boston Globe

Internet Task Force Shuts Down Anti-Spam Working Group - September 22, 2004 - eWeek

AOL to Support Sender ID E-Mail Standard - October 25, 2004

Sender ID Rises From the Dead, Spooky Patent and All - October 28, 2004 - good commentary on the actual situation.

Sender Policy Framework [SPF]

Introduction

Typical email headers

Verifying SPF entries

Recent news/commentary

Other helpful links

Administrivia: