Is it me? or do a lot IT/DNS admins not know how to configure SPF correctly?

Forums › IT Pro and developers › Is it me? or do a lot IT/DNS admins not know how to configure SPF correctly?

1 | 2 | 3

2287 posts

Uber Geek

Trusted

Lifetime subscriber

#2835327 18-Dec-2021 19:58

dimsim: ... i think the posts here make it quite clear that we agree that some organisations and the people responsible for managing their email systems have a mis EDIT: poor understanding of SPF

gorringS: gorringefamily.co.nz Lookup - SPF-Record which shows results of that record I shared with you

dimsim, meet gorringS. gorringS, say hello to dimsim

Please keep this GZ community vibrant by contributing in a constructive & respectful manner.

Tradepub

Free professional, reference and technical white papers (affiliate link).

BlakJak

1245 posts

Uber Geek

Trusted

#2835328 18-Dec-2021 20:09

The error is that the zone has no defined MX record in DNS.

That likely doesn't actually break mail delivery (non-host A record will be used instead, and I can see an exchange server on it) but I imagine SPF resolvers will have 'fun' with it...

That being said it's using a SOFTFAIL return so whether the SPF passes or fails is at worst, going to trigger a log entry.

(and to prove it, I sent myself an email impersonating postmaster@gorringefamily.co.nz and it was allowed.)

Yeah not a great to have 'mx' in the SPF record if there is no MX, but email from the 'a' record will pass a check so the 'mx' bit may not even be evaluated...?

No signature to see here, move along...

KiwiSurfer

1397 posts

Uber Geek

ID Verified

Lifetime subscriber

#2835341 18-Dec-2021 22:38

Just an observation from someone who used to run their own mail server about ~10 years ago before realising the folly of trying to keep up with the ever changing challenges around email delivery. Currently I use FastMail but I still keep an keen eye on developments in the email space.

Sophisticated spam detection systems won't flag an email message as spam just because of a SPF failure.

The spam detection system I'm most familiar with is SpamAssassin, which is common in the Linux world and used by some of the smaller email providers (e.g. FastMail and likely many other providers who use Linux). SpamAssassin assign points to various factors (SPF failures being one of many things it will test) and if there is enough red flags (e.g. SPF failure plus other things like server on a blacklist, suspicious language, non-conformance to certain standards, etc to name a few of a long list of tests it does when processing emails) it can flag the email as spam. Some things it will assign more points to, other things it will be more lenient about. So if there is something that is much more likely to be spam it would have more points (and thus more likely to cross the threshold for flagging as spam). Many systems like FastMail even allow you to adjust the thresholds for rejection and quarantine so you can be more strict or more lenient.

One could reject an email message just because of a SPF failure on its own but that would as you've seen that would cause problems for a non-significant proportion of legitimate mail.

I recall setting up my email server to reject because of simple things like failing SPF tests, being on a blacklist, etc--very quickly I discovered that this approach is very blunt and ineffective and ends up rejecting too many legitimate emails. I ended up using SpamAssassin instead which is indeed more complex but ultimately much more effective at weeding out the worst spam while letting a few false-negatives through.

If your goal is to deliver as many legitimate email as possible while minimising spam, it makes more sense to only reject if there is another red flag in addition to the SPF failure. My observation is that major provider (Google, O365, et al) won't always reject or flag an email as spam simply because of one reason. Their systems seem to take SPF only as one of many input into their system. Taking into account all the data they have related to a message (SPF being just one among other data they have from the message itself, historical data they have on the server IP work out its 'reputation', data they get/share from/with other providers e.g. public blacklists, etc) their system make a determination as to whether the message is likely to be spam or not.

Sadly the side effect of this softy approach is that incorrect SPF configuration can sometimes slip under the radar if it is the only issue a message has. E.g. if messages are otherwise well formed, comes from well known servers with good reputation, doesn't have spammy language, etc spam detection systems may not flag the message despite the SPF failure. so people think 'Oh my email system is working fine' not realising they're possibly right on the borderline of having their messages not be delivered.

It is not a ideal state of affairs. Google, Microsoft, et al have a balancing act and they often tend to err on the side of whatever generates the less complaints.

fearandloathing

497 posts

Ultimate Geek

ID Verified

Lifetime subscriber

#2835342 18-Dec-2021 22:39

leaplae:

I've set up SPF/DKIM/DMARC for many companies and organisations. The largest blocker to fully implementing DMARC on a domain is all the SaaS services that don't support DKIM, and don't let you change the mail servers they send out from to your own... Questioning when this feature will be made available is usually 'you're the first to ask' or 'its on our roadmap, but not a priority'.

The only realistic option here, and in my opinion, is highly recommended regardless of whether DKIM is supported or not. Is to split off SaaS applications to subdomains, e.g. saas@mailer.example.com with only SPF & DMARC if DKIM is not supported, protecting the primary domain user@example.com with SPF, DKIM & DMARC.

gorringS

71 posts

Master Geek

#2835353 19-Dec-2021 07:32

KiwiSurfer:

Sophisticated spam detection systems won't flag an email message as spam just because of a SPF failure.

After reviewing what you said here I went and check thing at my dns records there seem to be glitch which I past back to my provider and appears now that it spf record etc are working correctly . Something about @ record wasnt been pickup correctly . So if want to verify goto MXtoolbox and do spf record check on gorringefamily.co.nz

[Mod edit (JDB): Excessive quoting]

gorringS

71 posts

Master Geek

#2835355 19-Dec-2021 07:44

BlakJak: The error is that the zone has no defined MX record in DNS. That likely doesn't actually break mail delivery (non-host A record will be used instead, and I can see an exchange server on it) but I imagine SPF resolvers will have 'fun' with it... That being said it's using a SOFTFAIL return so whether the SPF passes or fails is at worst, going to trigger a log entry. (and to prove it, I sent myself an email impersonating postmaster@gorringefamily.co.nz and it was allowed.) Yeah not a great to have 'mx' in the SPF record if there is no MX, but email from the 'a' record will pass a check so the 'mx' bit may not even be evaluated...?

Just so you know dns records were correct but there was glitch at dns provider end regarding the @ record been pickup which they fix . This only reason why you were able acheive what you did as spf plus some other key records werent operating 100%

BlakJak

1245 posts

Uber Geek

Trusted

#2835385 19-Dec-2021 11:11

I see MX records in DNS now but you still publish a SOFTFAIL so people will not reject email that fails the check (you're telling them not to).

Using SOFTFAIL to add to a spam score is a more advanced possibility but it's then only a factor on a decision only influenced by SPF, not made by it.

No signature to see here, move along...

BlakJak

1245 posts

Uber Geek

Trusted

#2835387 19-Dec-2021 11:22

KiwiSurfer:

One could reject an email message just because of a SPF failure on its own but that would as you've seen that would cause problems for a non-significant proportion of legitimate mail.

If a hardtail is published this is what you're expected to do. If you do anything else then you're in breach of the RFC.

OP publishes SOFTFAIL so has not set that expectation.

I recall setting up my email server to reject because of simple things like failing SPF tests, being on a blacklist, etc--very quickly I discovered that this approach is very blunt and ineffective and ends up rejecting too many legitimate emails. I ended up using SpamAssassin instead which is indeed more complex but ultimately much more effective at weeding out the worst spam while letting a few false-negatives through.

If your goal is to deliver as many legitimate email as possible while minimising spam, it makes more sense to only reject if there is another red flag in addition to the SPF failure.

Sure. SPF is just one tool, but to be clear, spamassassin is a much more complex beast to tune than simply evaluating SPF. And both of those are harder than simply setting up DNS records.

They will use multiple inputs but they will also use an SPF hardfail the same as anyone else. SOFTFAIL will go into their mix, but my experience is that they will filter negatively on the absence of positive measures. For example my privately operated MTA is being persistently filtered into quarantine by one Microsoft tenancy right now with zero reason... Except that I haven't got around to DKIM and DMARC yet. But the emails concerned come from a Clean IP and pass SPF. *Shrug*

No signature to see here, move along...

raytaylor

4001 posts

Uber Geek

Trusted

#2835501 19-Dec-2021 12:16

Your right. Many domain admins are not very good at SPF at all.
Though I have never understook DKIM or took the time to learn, but SPF does well for me most of the time.
I always use a hard fail on my instructions for our domain to other mail servers.

One of the things you can do is include an smtp server address of the external company in your domain spf record.
Eg. If something like Xero? is sending invoices to customers on your behalf and you trust them, you can include their smtp address as an allowed source of messages.

As long as they are using the same ip addresses for outbound mail then its possible for them to maintain their own list of ip addresses and it will create a chain of lookups for the recipients mail server to check and verify.

In the case of google apps for business, if you add spf.google.com to your spf record hosted elsewhere, it automatically allows all the gmail/google outbound servers to send on your behalf.

By using a hard fail instruction to other servers, I encourage our customers to relay through our smtp server so that we can verify their credentials, set outbound hourly limits and maintain a good reputation.

Ray Taylor

There is no place like localhost

Spreadsheet for Comparing Electricity Plans Here

jarledb

Webhead
3249 posts

Uber Geek

Moderator

ID Verified

Trusted

Lifetime subscriber

#2835549 19-Dec-2021 15:35

gorringS:

Hi you will probably find wont be spf record alone that stopping you sending emails . txt record

@ v=spf1 a mx ~all

This kind of showcases why SPF can be difficult.

Your SPF record would do zero good on a lot of sites that I host.

The A record would fail: Most sites have Cloudflare as the A record, but that is not the ip-address of the web server that will be sending email.

The MX record would fail for the web server, because that is an external service. And for the customer that use Google Workspace it wouldn't work unless you use an include: for their SPF records.

Of course, you could put anything in that SPF record since you are softfailing and not rejecting, but still.

For the customers that want to hard fail on the SPF records I always try my best to make sure they are not using any other email service for sending emails than what is used on the web server and/or email service. A lot of customers will be using their ISPs SMTP servers, which would hard fail all emails they send out that goes via those if those are not included in the SPF record.

Jarle Dahl Bergersen | Referral Links: Want $50 off when you join Octopus Energy? Use this referral code
Are you happy with what you get from Geekzone? Please consider supporting us by making a donation or subscribing.

dimsim

842 posts

Ultimate Geek

Trusted

Lifetime subscriber

#2835749 20-Dec-2021 07:51

raytaylor:

Your right. Many domain admins are not very good at SPF at all.
Though I have never understook DKIM or took the time to learn, but SPF does well for me most of the time.
I always use a hard fail on my instructions for our domain to other mail servers.

One of the things you can do is include an smtp server address of the external company in your domain spf record.
Eg. If something like Xero? is sending invoices to customers on your behalf and you trust them, you can include their smtp address as an allowed source of messages.

As long as they are using the same ip addresses for outbound mail then its possible for them to maintain their own list of ip addresses and it will create a chain of lookups for the recipients mail server to check and verify.

In the case of google apps for business, if you add spf.google.com to your spf record hosted elsewhere, it automatically allows all the gmail/google outbound servers to send on your behalf.

By using a hard fail instruction to other servers, I encourage our customers to relay through our smtp server so that we can verify their credentials, set outbound hourly limits and maintain a good reputation.

Agree... a softfail as far as i understand was to be used when transitioning between outgoing mail servers to insure mail flow. Softfail nowadays seem to be for people of have a poor understanding of which servers are sending outgoing FROM @domain mail for them and is fairly pointless other than possibly adding to an increased spam score.

I've always set a hard fail for my and my clients SPF as I want the receiving server to FAIL ALL mail coming from a server not listed in SPF.

BlakJak

1245 posts

Uber Geek

Trusted

#2836068 20-Dec-2021 16:41

jarledb:

gorringS:

Hi you will probably find wont be spf record alone that stopping you sending emails . txt record

@ v=spf1 a mx ~all

This kind of showcases why SPF can be difficult.

Your SPF record would do zero good on a lot of sites that I host.

The A record would fail: Most sites have Cloudflare as the A record, but that is not the ip-address of the web server that will be sending email.

The MX record would fail for the web server, because that is an external service. And for the customer that use Google Workspace it wouldn't work unless you use an include: for their SPF records.

Of course, you could put anything in that SPF record since you are softfailing and not rejecting, but still.

For the customers that want to hard fail on the SPF records I always try my best to make sure they are not using any other email service for sending emails than what is used on the web server and/or email service. A lot of customers will be using their ISPs SMTP servers, which would hard fail all emails they send out that goes via those if those are not included in the SPF record.

Um... i'm not quite sure you get how this works from the above.

With A and MX you are declaring that:

- If the domain name has an address record (A or AAAA) that can be resolved to the sender's address, it will match.

- If the domain name has an MX record resolving to the sender's address, it will match (i.e. the mail comes from one of the domain's incoming mail servers).

... "Most sites have Cloudflare as the A record" is irrelevant. If an email platform receives an email with the sender of your domain, and your domain SPF record says "A" and you resolve the IP address that makes the inbound connection to your A record, it passes. Job done. Sure, someone who uses Cloudflare _and_ chooses to point their A record at Cloudflare, may not be able to use that field. But OP hasn't done that. So your statement makes little sense.

"The MX record would fail for the webserver". Not sure where you're going. By saying 'MX' you're saying 'query my domains MX records; if the party sending email on my behalf resolves in DNS to the same place as my MX record, it passes". Webserver is irrelevant to this statement. This is about email, not websites.

If you want a third party to send email on your behalf, and your third party publishes an SPF record of relevance, you can 'include' their SPF record in your own through the use of the include:their.spf.record element. True. But this user hasn't done that. Nor have they professed a need to.

Anyone who wants to use a hardfail SPF record needs to ensure their SPF record 100% represents the parties authorised to send email from their domain. Softfail records are appropriate where you don't have this certainty and are still figuring out the details. So you choose the result (hardfail, softfail, etc) for your own circumstances.

No signature to see here, move along...

MadEngineer

4220 posts

Uber Geek

Trusted

#2836084 20-Dec-2021 17:58

dimsim:
I used to have my Exchange Server fail anything that didn't pass SPF but found I was constantly blocking email that I actually wanted.

I've since removed the block so I can actually receive those emails but prior to this made contact with several local businesses and helped them configure SPF correctly so I could receive their communications.

A big IT supplier of mine (no names mentioned) happens to think it's a great idea to send email as my domain when sending out MS licensing and just thought that this would work but without consultation and from servers outside of my SPF record. Other suppliers (IT/Networking space) that should know better have also failed in this regard and despite pointing out the glaring errors think there is nothing wrong with sending via a mail service like mailchimp but not adding the mailchimp include record to their SPF record.

Does anyone else see issues like this at all?

An Exchange server that blocks all mail from servers that fail SPF checks is itself incorrectly configured.

You're not on Atlantis anymore, Duncan Idaho.

ANglEAUT

2287 posts

Uber Geek

Trusted

Lifetime subscriber

#2836105 20-Dec-2021 19:27

BlakJak: ... Anyone who wants to use a hardfail SPF record needs to ensure their SPF record 100% represents the parties authorised to send email from their domain. Softfail records are appropriate where you don't have this certainty and are still figuring out the details. So you choose the result (hardfail, softfail, etc) for your own circumstances.

💯 Agree. It does seem to be one of those set & forget settings though. When will it ever be changed to a hard fail? In most IT shops, the mantra is "If it ain't broke, don't mess with it" 😭

Please keep this GZ community vibrant by contributing in a constructive & respectful manner.

dimsim

842 posts

Ultimate Geek

Trusted

Lifetime subscriber

#2836109 20-Dec-2021 19:42

MadEngineer:
dimsim:

I used to have my Exchange Server fail anything that didn't pass SPF but found I was constantly blocking email that I actually wanted.

I've since removed the block so I can actually receive those emails but prior to this made contact with several local businesses and helped them configure SPF correctly so I could receive their communications.

A big IT supplier of mine (no names mentioned) happens to think it's a great idea to send email as my domain when sending out MS licensing and just thought that this would work but without consultation and from servers outside of my SPF record. Other suppliers (IT/Networking space) that should know better have also failed in this regard and despite pointing out the glaring errors think there is nothing wrong with sending via a mail service like mailchimp but not adding the mailchimp include record to their SPF record.

Does anyone else see issues like this at all?

An Exchange server that blocks all mail from servers that fail SPF checks is itself incorrectly configured.

I take the view that if a domain has specified a Hard Fail in their SPF record then essentially they are saying that email shouldn't be trusted - hence I have simply failed those messages previously.

I have a hard fail on my SPF record and wouldn't have any issue if a receiving mail server rejected a message purporting to me From my domain from a server outside of my SPF record.

1 | 2 | 3