SPF works by domains publishing "reverse MX" records to tell the world what machines send mail from the domain. When receiving a message from a domain, the recipient can check those records to make sure mail is coming from where it should be coming from.

With SPF, those "reverse MX" records are easy to publish: one line in DNS is all it takes.

No. You should publish spf records for each and every domain you wish to protect from being used by spammers/virusses. If, for example, your domain is somedomain.tld and you furthermore have a subdomain www.somedomain.tld registered, you would publish for both somedomain.tld and www.subdomain.tld (the latter probably being set to "v=spf1 -all"). Note that you will have to publish for each and every A record, including any wildcard (*) or @ entries in your dns.

Often, you will want to allow certain servers to send mail through your smtp server. For example, if you have machines on your dmz that must be able to send status messages or you have some machines on your LAN that need to send out mail from your domain. In this case you will generally not want to publish these services in your public spf record (eg. "v=spf1 ip4:10.0.0.0/8 a -all") since this information is potentially valuable for hackers.

This can be solved in two ways. First of all, many spf implementations provide you with an option to put these adresses in a whitelist. This is just a list of hosts, which only needs to be available to your smtp server. A second option is to implement a 'local policy'. For details, consult the documentation that came with your specific spf implementation or search the list archives for 'whitelist' and 'local policy'.

You can not use your local dns server to publish txt records for your internet domains. You will have to use the dns server that serves your domain to the internet. Contact your dns provider and/or your hosting company, or look in the webpanel your dns/domain hoster provides.

Suppose a spammer forges a hotmail.com address and tries to spam you.

He connects from somewhere other than hotmail.

When his message is sent, you see MAIL FROM: <forged_address@hotmail.com>, but you don't have to take his word for it. You can ask Hotmail if the IP address comes from their network.

(In this example) Hotmail publishes an SPF record. That record tells you (your computer) how to find out if the sending machine is allowed to send mail from Hotmail.

If Hotmail says they recognize the sending machine, it passes, and you can assume the sender is who they say they are. If the message fails SPF tests, it's a forgery. That's how you can tell it's probably a spammer.

SPF was designed to protect the envelope sender. That means the return-path that shows up in "MAIL FROM", and to a lesser extent the HELO argument that is supposed to be an FQDN.

The vast majority of SPF implementations today use the return-path as the subject of authentication and do not get involved with the header "From:".

Protecting authorship information is an important goal. However, the technical issues associated with protecting the "From:" header are much more numerous and challenging. The best way to protect the header "From:" is by using a cryptographic signature such as S/MIME, PGP, or (when it is released) Yahoo DomainKeys.

If you want to use the "From:" header as the subject of authentication with SPF, you need to be familiar with the following:

• mailing lists
• /etc/aliases-style forwarding
• MUA "resend this message to"
• web-generated email
• the Sender header
• the Resent-Sender and Resent-From headers

Stuart Gathman's opinion is recorded at http://archives.listbox.com/spf-discuss@v2.listbox.com/200410/0488.html.

Yes, it does. You'll have to switch from forwarding, where the envelope sender is preserved, to remailing, where the envelope sender is changed. But don't worry, we're working on providing SRS patches for the four major opensource MTAs, so that when you upgrade to an SPF-aware version, this problem will be solved also.

If your forwarding runs through a commercial service like pobox.com, you shouldn't have to do anything. They have to change with the times, and perform the above rewriting automatically for you. SRS is a structured standard that helps them adapt.

Until the SRS patches are ready, the following workarounds will preserve the important functionality.

    :0
    * !^FROM_DAEMON
    ! foo@bar.com

    # loop protection
    :0 fw
    * !^X-Loop: foo@bar.com
    | /usr/bin/formail -A'X-Loop: foo@bar.com'

    :0 A
    {
        :0
        * !^FROM_DAEMON
        ! foo@bar.com

        :0 B
        * !foo@bar.com
        ! foo@bar.com
    }

You can't make an omelet without breaking eggs, and unfortunately forwarding is the egg that breaks. We're doing our best to patch it back together with SRS.

More and more DNS service providers are supporting TXT records. You can find a list of them at http://www.kitterman.com/spf/txt.html. If yours is not on the list, write to them and ask for TXT support!

At a high level, the answer is that we're moving from one paradigm to another: from "assumed innocent until proven guilty" to "assumed guilty unless proven innocent". The Aspen Framework brings two important tools to bear: reputation and accreditation. (A cartoon guide is available.)

We agree that throwaway domains will be the next step in the arms race. We can counter with:

1. fast automated blacklisting using spamtraps and attack detectors
2. simple reputation systems based on factors such as
   • age of domain according to whois
   • email profile of domain, eg. "too many unknown recipients"
   • call-back tests to see if the sender domain is able to receive mail.
   The reputation system can advise a receiving MTA to defer or reject.
3. legal methods following the paper trail of who paid for the domain.

Here's an example of automated blacklisting in action:

1. A spammer spams.
   • The spam comes from an SPF-conformant domain.
     • That domain is on a widely published sender-domain blacklist.
       • The MTA rejects the message.
     • That domain is a throwaway, just-registered domain, and does not yet appear on blacklists.
       1. The spam gets accepted by unsophisticated MTAs which do not use other traffic-analysis methods to impose a crude reputation system on unrecognized senders.
       2. The spam also gets accepted by automated spamtraps.
       3. The spamtraps add the domain to the blacklist.
       4. (advanced) Some time later, the user checks email. Immediately before the display phase, the MUA re-tests the message against the blacklists, and discards it.
       5. Thanks to the greater level of sender accountability, lawsuits may begin against the spammers, and registrars may be subpoenaed for domain owner information. SPF strengthens administrative and legal methods.
   • The spam comes from a non-SPF-conformant domain.
     • Initially,
       1. Most legitimate mail will fall into this category.
       2. Normal content filters get to do their job.
       3. The usual false-positive/false-negative results apply.
     • Later,
       • Most legitimate mail will be SPF-conformant.
       • Some legitimate mail will not be SPF-conformant.
       • SPF-conformant receivers SHOULD receive non-conformant mail but MAY choose to perform additional filtering on it.
2. Eventually, as SMTP improves its immunity to spam, we hope spammers will get discouraged.

If the volume of spam decreases, legal and administrative approaches become more effective; right now they are simply swamped. If there are only 10 spammers in the world, law enforcement can focus on catching each one. If there are 10,000 spammers, law enforcement throws up its hands, calls it a societal problem, and says it doesn't have enough resources to tackle it.

• The spam domain was registered with a domain registrar.
• If the registrar is cooperative, we can find out from the registrar who the spammer was; and the registrar can stop accepting their registrations.
• If the registrar is uncooperative, or if a spammer buys and runs a registrar, we can default-blacklist all their domains, in a political move similar to SPEWS's approach.
• Alternatively, since spam is becoming increasingly illegal, we can subpoena the registrar to find out who registered the domain, and sue the spammer directly.
• If the spammer registered the domain using false information, we can still go back to the credit card.
• If the credit card was stolen, that's a crime which can be addressed using traditional means.

(20040702) Scott Kitterman has posted a suggested refinement to the above plan.

If you run a personal domain, you can either not publish SPF records at all, or set up "v=spf1 ?all" for your domain, and you'll be able to send mail from your laptop no matter where you are.

If you are the customer of an ISP that publishes SPF records, your ISP should provide you with an SMTP server that you can authenticate to, using either POP-before-SMTP or SASL AUTH. Or you can ask them to exclude you from SPF using the a user-specific "exists" mechanism.

I'm looking at an envelope. The return address claims to be from Blue Blob. Let's compare that to the postmark. Yup, the postmark says the message comes from Blue Blob. That means the return address is probably not made up. The message is good.

In email, the envelope sender, also known as the return path, is the return address. The SMTP client is the post office that applied the postmark. The blue blob is the domain name. If they don't match, maybe the message is a forgery. Maybe a bad guy made up the return address to fool you into opening the envelope. Maybe a spammer forged a Paypal return address to try to get your credit card number.

You can display the SPF logo on your website. Choose one that's right for you.

The spam problem was never as bad in the past as it is now.

People are willing to put up with a lot more change and pain.

People who have shown interest in supporting SPF include Qualcomm (makers of Eudora), Tim O'Reilly (publisher of geek books), SpamAssassin, ActiveState (makers of PureMessage), MailArmory, Declude JunkMail, and others.

Mail::SPF::Query provides a best_guess method, which pretends the domain had declared "a/24 mx/24 ptr". This is remarkably good at detecting unforged messages from domains which have not yet implemented SPF. It helps reduce false positives.

There are two types of SPF testers available. There are those that you send email to, which are good for testing how actually email will respond from the computer you are sitting at. There are also those that let fill in the appropriate information and can simulate an SPF check from anyone and from anywhere.

Email based SPF testers

• You can also send mail to spfenabled@pobox.com and see what happens. If you send mail from an unlisted server it will be rejected. Please don't make up bogus addresses if that would cause random third parties to get mysterious bounce messages.
• Port25.com also provides an tool to test whether SPF is working. Send an email to check-auth@verifier.port25.com and you will receive a reply containing the results of the SPF check.
• The ESPC also provides an SPF verification tool

Form based SPF testers

• Scott Kitterman's SPF tester
• the standard reference why.html page
• DNS Stuff (home of Declude JunkMail)

There are four actually. The big ones are for discussion and announcements. The little ones are for getting help with setting up SPF, and for developers who are implementing SPF related software like client libraries and SRS rewriters.

The lists receive between 5 and 50 messages a day.

There is a dedicated page all about these mailing lists and how to subscribe.

If I get mail from pielovers.demon.co.uk, and there's no SPF data for pielovers, should I go back one level and test SPF for demon.co.uk?

No. Each subdomain at Demon is a different customer, and each customer might have their own policy. It wouldn't make sense for Demon's policy to apply to all its customers by default; if Demon wants to do that, it can set up SPF records for each subdomain.

So the advice to SPF publishers is this: you should add an SPF record for each subdomain or hostname that has an A or MX record.

Sites with wildcard A or MX records should also have a wildcard SPF record, of the form: * IN TXT "v=spf1 -all"

(Thanks to Stuart Cheshire.)

Spammers may spoof entire TCP sequences to get their mail out.

Knowledgeable people consider this unlikely. If you think this is a concern, you are invited to demonstrate an attack.

I thought you could only have letters, numbers, and dashes in domain names, no underscores.

Underscores create a new secret dimension: that's the only way to guarantee you don't collide with an existing subdomain. SRV records use underscores for exactly this reason.

When a domain has no MX records, we assume that an A record will suffice.

That's what the Exists mechanism is for.

SRV records were, roughly speaking, meant for letting people add this sort of thing to DNS without having to add new record types. See also RFC2761, Extension Mechanisms for DNS.

Yes. But SRV records are hard for people to understand, and TXT records are easy. Fast widespread adoption is our goal. The Right Thing To Do is to get our own RRtype, and we will apply for it. We just don't expect to get it anytime soon.

(for SRV records, see http://dqd.com/~mayoff/tools/djbdns/make-record.adp)

All this seems like a lot of work. The spam filter that runs in my MUA or MDA catchs 99% of spam. Why should I care?

You may not care as an end-user, but your ISP does. Your ISP has to pay for the bandwidth of delivering all those messages to you. Your ISP spends time and money getting them down the pipe to wherever you delete them. And your ISP passes these costs on to you. If your ISP can block forgeries without having to download them at all, ultimately, you benefit.

Another reason to participate is that you can protect your own domain from being used by virusses, spammers and the lot.

Spammers are increasingly using compromised machines on fast connections to send spam.

Broadband providers will have to either force port 25 to go through their servers where they can keep an eye on traffic, or suffer the reputation costs of having their "forged" domain name in the envelope sender line. I know people who block Hotmail, Yahoo, and AOL simply because those are the most commonly forged domains; that hurts their legitimate users and ultimately it hurts them.

Each SMTPd in a spam run may send a DNS query to the forged host's nameservers. At a million SMTPds, that's 100 megabytes of traffic!

DNS queries are still smaller than bounce messages. And most SPF lookups can be cached; only the relatively uncommon exists mechanism doesn't benefit from caching.

When the sender address is <>, you don't have any domain to look up!

There are two solutions, one strong, one weak.

The weak solution uses the domain name presented in the HELO/EHLO command instead. This may work to catch careless spammers.

Careful spammers will find a way to get around this: consider a DynDNS-registered broadband host that has been compromised. The strong solution accepts the message, takes it apart, determines if it is in fact a bounce, and if it is a bounce, it extracts the Message-ID of the original message, and if the the Message-ID did not in fact originate from the system, it discards the message. This takes a bit of work, but in a world where automated spamtraps do most of the work and would add the hostname to known-spammer lists anyway, a spammer might as well use a non-null sender address anyway.

In either case an MTA should reject messages from null senders that have more than one recipient.

The SPF draft says that SPF checks are only meaningful in certain situations.

MTAs can block a lot of spam even before SPF checks occur.

Here are some suggestions that will block a lot of spam. Only messages that get past all these rules need to be SPF tested.

1. The envelope sender domain must have either an A or MX record.
2. The A or MX record of that sender domain must not be in:
   • 0.0.0.0/8
   • 10.0.0.0/8
   • 127.0.0.0/8
   • 172.16.0.0/12
   • 192.168.0.0/16
3. The connecting client IP address must have a PTR record.
4. The HELO hostname must be a well formed FQDN that has an A record.

Note that rules 3 and 4 are often violated by legitimate but clueless domains who don't pay attention to these kinds of detail.

You can configure these settings in Postfix at http://www.postfix.org/uce.html

1. Vanity domain holders may point to you using an include: directive.
2. You should support SASL AUTH on ports 25 and 587. When your users need to phone home from an Internet cafe, port 25 may be blocked, but port 587 often isn't.
3. You may want to set a default of "unknown" by saying "?all" instead of "-all" for the first few months, to accommodate legitimate users who have been sending mail through somebody else's servers.
4. You can tell who these users are by adding an "exists:%{l}.%{i}._spf.ISP.COM" record, and grepping in your DNS server logs to see who's mailing through what machines. Then you can contact all nonconformant users and tell them to use your SMTP server.
5. You should never publish -all for customers domains without prior notice. Keep your customers informed about your spf roll-out, this will prevent them from being unpleasantly surprised by mail that suddenly is not delivered due to spf FAIL results.

It used to, but that name turned out to be too slippery for the human mind to grasp, so we in early February 2004 we changed it to the more accurate "Sender Policy Framework".

Trivia: for marketing reasons we picked the acronym before we picked the expansion. First we went to Google to see if SPF already stood for anything outside the sunblock world. It is the name of a networking concept --- Shortest Path First --- but that's okay; every good acronym deserves at least two expansions. So we chose that name for the brand and today we're #1 on Google for "SPF".