>I get 99% deliverability with 0 repuation since i do dkim and spf correct.
Your anecdote of success doesn't matter to the others that correctly configured DKIM/SPF and still don't get their emails delivered to Gmail/Outlook/Yahoo/etc. E.g. : https://news.ycombinator.com/item?id=32715437
One of the reasons for hard-to-diagnose sending failures is that Gmail/Outlook have "extra invisible rules" that override correct DKIM/SPF settings because spammers and phishers also have correct DKIM/SPF. So they use extra heuristics such as "ip reputation" etc.
And even after one gets it working, e.g. "submit some form" to Microsoft and wait a few days to get things unblocked... the deliverability may break again because of another "invisible heuristic".
EDIT to reply: >No, that's because your relay overwrites part of the header which makes dkim strict break. Change to relaxed or don't modify the header on your relay.
Delivery reliability can still break without using a relay.
In fact, this unreliability of 100% self-hosting at home is why some self-hosters split it into a hybrid setup and add an external relay for outgoing SMTP and only keep self-hosting for receiving email.
Get this. I owened a /23 for 7 years (still own it today) and kept the mail server ip on a /27 just for the mail server on a /24 that was not used for anything production (firewalled and maybe 3 ip's responded on port 443). My mails were banned for bad reputation. The provider which hosted my /23 was well known for responding to abuse, even falsely flagging my account as abusive in the early days for simply _sending_ valid smtp mails.
IP reputation turned out to mean, if they never saw your IP, you were in the banned bucket. How do you even fight against that
No, that's because your relay overwrites part of the header which makes dkim strict break. Change to relaxed or don't modify the header on your relay.
Outlook business will accept your mail, Outlook private may filter, but the rates fluctuate so heavy i suspect its rules based on user behaiviour/interests. I dono, cant have both spamfree inbox and 0 false positives.
I think i found a loophole for the google and outlook ones... I have had my domains on both providers, and then left to my own (but left a couple of google and ms txt records by mistake) and never had any issues delivering to both providers. Thinking of doing the same thing again honestly, but looking at good providers at the moment.
I hate the fact that your comment got flagged / greyed out / whatever even though it's perfectly correct. I'm one of those people who had configured everything perfectly. Score of 100 on mail-tester, SPF, DKIM, DMARC, you name it. Examining the headers in an e-mail sent to gmail: pass, pass, pass. Everything green.
Microsoft however? Denied, 100% of the time. Spam folder, or even plain rejected. Why? No idea, they won't say. They redirect you to their shitty partner that you can PAY in order to HOPE you get approved.
I don't know why our experiences are considered "anecdotes", and not the other way round. What's the incentive for big players to accept e-mail from home servers or small dedicated servers? "Sure it could be Standard Nerd from HN running their own stuff for street cred points, or it could be one of the bazillion spam factories sending fake UPS scams. In doubt, let's reject."
I add it here so you can successful self-host: You need strict DMARC for Microsoft. If you change the header on your relay DMARC relaxed filters will pass the mail, but not strict.
Because this adds the need to sign every single mail for every single recipient (expensive) its safe to filter for this as a SPAM-Server will sign mail once, then distribute.
That's why your mail is filtered - not because your non-blacklisted IP is the problem or whatever.
>I hate the fact that your comment got flagged / greyed out / whatever even though it's perfectly correct. [...] I don't know why our experiences are considered "anecdotes", and not the other way round.
It's because people who successfully self-host think their situation universally applies to everyone.
Here's another example from 2017 of someone replying to my previous reasonable comment about self-hosting by overconfidently saying I was exaggerating the issues : https://news.ycombinator.com/item?id=15526127
So they end up solving it by "outsourcing" the outbound email to a relay (SendGrid).
So my comment gets downvoted for explaining what others had to do in the real world.
The following should not be a controversial statement but for some reason it is: Correctly configuring SPF/DKIM/DMARC and getting 100% green score on https://www.mail-tester.com/ for your self-hosted setup ... does not universally mean your outbound email will get accepted by all the services.
Read the logs from Gmail and Microsoft, they will tell you exactly why the mail was filtered. Act on that problem and have your mail appear in inboxes.
It's usually relaxed DMARC triggering Microsoft. Gmail accepts relaxed.
Your anecdote of success doesn't matter to the others that correctly configured DKIM/SPF and still don't get their emails delivered to Gmail/Outlook/Yahoo/etc. E.g. : https://news.ycombinator.com/item?id=32715437
One of the reasons for hard-to-diagnose sending failures is that Gmail/Outlook have "extra invisible rules" that override correct DKIM/SPF settings because spammers and phishers also have correct DKIM/SPF. So they use extra heuristics such as "ip reputation" etc.
And even after one gets it working, e.g. "submit some form" to Microsoft and wait a few days to get things unblocked... the deliverability may break again because of another "invisible heuristic".
EDIT to reply: >No, that's because your relay overwrites part of the header which makes dkim strict break. Change to relaxed or don't modify the header on your relay.
Delivery reliability can still break without using a relay.
In fact, this unreliability of 100% self-hosting at home is why some self-hosters split it into a hybrid setup and add an external relay for outgoing SMTP and only keep self-hosting for receiving email.