It's unfortunate that email hosting and email infrastructure can really be done only well by major players. The days of people running and maintaining their own are pretty much long gone.
Fwiw, not a knock against CF. I like their products, mostly simple, fair pricing, etc. Just a bit unfortunate commentary on the state of email infra on the internet.
I run my own email server and you couldn't pay me to use a commercial provider like Google instead. The privacy benefits are huge and there is no one to restrict my storage or change my "terms and conditions" overnight.
The days of people running their own servers are gone because of the shortsightedness and laziness of IT managers. They though the "cloud" would be easier and cheaper, and they are now trapped.
I entertained the idea of running my own mail servers for a while. After researching the topic it turned out that the internet now runs on an IP reputation system. Major email services like gmail assume that anything sent from unknown IPs is malicious.
So it looks like we've gotta be well connected to federate with the other email servers now. A nobody like me can't just start up his own mail server at home and expect to deliver email to his family members who use gmail or outlook. So I became a Proton Mail customer instead.
I've run my own mail servers for many decades and have never had any deliverability issues. I've also never used bargain basement cloud VPS services with horrible reputations.
The best way to ensure a good reputation is to obtain your own address space from a RIR. Barring that, you need to choose a provider with a decent reputation to delegate the space to you.
Not true, at least for ARIN. If you have an IPv6 allocation, you can obtain one or more IPv4 /24 allocations, so long as their stated purpose is to provide IPv4/IPv6 compatibility (e.g. for dual-stack services or NAT): https://www.arin.net/participate/policy/nrpm/#4-10-dedicated...
From your HN profile, I see you're in Brazil, which is part of the region IANA has delegated to LACNIC. Per [0], LACNIC has further delegated numbering authority in Brazil to Registro.br.
Before I even start this bureaucratic process, I need to create an actual organization. Then I need to be assigned an ASN. Only then I'll be allowed to beg them for IPs. Once all that's taken care of, I need to tell them things like what the IPs will be used for and what my infrastructure is. If they like my answer, then they'll approve my request and finally tell me what the prices are.
I've been through the process about 10 times now at various companies, and the paperwork (at least for ARIN) is no more difficult than what would be expected to justify IP space from your typical ISP. If anything, the ARIN folks are more responsive and technically competent than your average ISP support agent, which makes the process easier.
> After researching the topic it turned out that the internet now runs on an IP reputation system. Major email services like gmail assume that anything sent from unknown IPs is malicious.
You have to buy/rent a dedicated IP address (that you'll be able to keep long term), and it warm it up by gradually increasing mail volume over a few months to weeks. But once you have, deliverability shoudl be fine.
I think the bigger issue is needing to keep on top of mainenance of the server.
Like the parent have ran Email servers for many years now. If you get a bad IP, as long as you get the DKIM records right, over time it will 'warm' up the IP. And the more you use the email on that IP and NOT spam people. The IP will warm up. Make sure you actually own that IP!!! It will become valuable.
Key point - own the IP. We own our IPs and we also buy elastic IPs from AWS. The entire AWS subnet (it seems their entire address space) is universally garbage and unwarmable. Our own IPs have hummed along for years with zero issues.
FWIW, a huge percentage of the spam I get is via Sendgrid, and at some point in the past year or two their abuse reporting mechanisms all turned into black holes, so mail sent via Sendgrid is heavily penalized in my spam rules.
Sending reputation is just as applicable if you're using a third party as if you're hosting it yourself, but much less under your control.
I don't have deliverability issues to the big providers, but that comes down to the age of my domain and my IP in a clean non-residential block. But you won't have reputation issues if your friends and family also run their own server and don't enforce such arbitrary requirements. Running your own servers, not only for email, is the only way to regain control over your computing.
I have arrived at the opinion that what I would do if I moved to selfhost would just be to pay some trivial amount for outbound email via a provider like sendgrid as someone else in these comments has also mentioned. Since I send out maybe a half dozen emails a month I don't think this would be a big deal.
But when I relied on selfhosted email several years ago, I was always inundated with spam, which SpamAssassin was wildly undermatched to handle -- that was one of the main reasons I moved to gmail. So I'm curious what people who are happy self-hosting today are using.
My suggestion would be to use a unique alias for each website/company. This way, if you start receiving spam at that address, you know who leaked it, and can simply delete the alias. You should also then publicly name and shame the source of spam.
I also run SpamAssassin on my server, but I don't believe it ever had to do anything.
I've run my own mail for 10 years (postfix/dovecot/rspamd), no issues. Reverse DNS, SPF, and DKIM records need to be in place, but that's a small lift.
Well, one time I was unable to send mail to a guy with an ancient @att.com email address from his ISP. I got a nice bounce message back with instructions to contact their sysadmins to get unblocked.
To my surprise, they unblocked the IP of my mail server in a matter of hours.
Private email will have no problems. I also ran my own mail server for personal use and had almost zero problem (and this was on an AWS IP!).
Where people will absolutely have problems is trying to run a marketing campaign through their own IP. You absolutely will (and should) get blocked. This is why these mixer companies exist and why you pay for an intermediary to delivery your mail.
I suspect if you shared more info about your mail infrastructure, it might reveal that what is working for you is too complicated for 99.9% of people to set up and maintain themselves.
I don't think the goal is that every non technical person can host their own mail infra.
But most people who can run a server should be able to setup OpenSMTPd with the DKIM filter and Dovecot. It's much easier than configuring postfix like we had to do in the past.
To answer a sibling comment, the last time I received an answer is a few minutes ago. The correspondent's email infra is hosted by Google.
You're right, it used to be a bit complicated. Now you just need to have a reputable and clean IP address, and knowledge of running some services in docker and of course understanding DNS and its crucial role for running a mail server.
I used to run all the components and maintain it (even that wasn't bad), but I changed to mailu[1] about a year ago
It is probably because you have run it so long that you have good reputation and less issues. Too bad we don't have time machine to go back to ninties to start building up reputation.
The problem is that Gmail will bounce any emails from DigitalOcean IP, even if you sit on this IP for years (so no recent spam), even if replying to someone, even if you registered as 'Postmaster' on Google.
So if you want to selfhost, you'll first need to find an IP that's not blocked to begin with.
> It's not hard, if you do it in a way that you can't send to like 50% of the recipients.
So it's hard (to do well)
>The problem is that Gmail will bounce any emails from DigitalOcean IP, even if you sit on this IP for years (so no recent spam), even if replying to someone, even if you registered as 'Postmaster' on Google.
>So if you want to selfhost, you'll first need to find an IP that's not blocked to begin with.
I'd say this is just the thing antitrust was made for. Hopefully some incumbent can get them to court.
That is not my experience at all. Using a pretty fresh IP and domain I get pretty good deliverability as long as I have proper rDNS and all the other normal steps (like DKIM, etc.)
Cloudflare's customers are companies that have to send out, say, reset password emails and verify account emails and other crumbs of the modern web. You want me to build my own infrastructure for that? Personally I can't wait for them to expand to SMS and crush Twilio.
> The days of people running and maintaining their own are pretty much long gone
This is very much a myth. There's a lot of FUD around how mail is "hard", but it's much less complicated than, say, running and maintaining a k8s cluster (professionally, I'm responsible for both at my org, so I can make this comparison with some authority).
Honestly `apt install postfix dovecot` gets you 90% of the way there. Getting spambinned isn't a problem in my experience, as long as you're doing SPF and DKIM and not using an often-abused IP range (yes, this means you can't use AWS). The MTA/MDA software is rock-solid and will happily run for years on end without human intervention. There really isn't anything to maintain on a regular basis apart from patches/updates every few months.
I think that there's a mindset among younger coders that "if it's not a modern post-AWS cloud provider, servers will take ages to come online and aren't going to give me full access, that's why EC2 exists." And this is conflated with the myth that running a mail server is hard.
But in practice, you can find any number of VPS providers, running in local datacenters, with modern self-service interfaces, with at least some IPs that aren't already spam flagged (and you can usually file a ticket to get a new IP if you need it), that are often cheaper per month than AWS, and give full root and everything. Find a service that will help you warm the IPs before you send to customers, and you're good to go!
This is 100% my experience too. Self-hosting email isn't any harder than self-hosting something else and there is no maintenance beyond apt update and apt upgrade. Even if you choose to do this in hard mode using postfix/dovecot instead of a dockerized stack, you can get a working config in a few minutes from an LLM these days.
> There's a lot of FUD around how mail is "hard", but it's much less complicated than, say, running and maintaining a k8s cluster
The main difference is that you're fully in control of the k8s cluster, but no matter what you do, you don't have control over the email infrastructure, because deliverability depends on the receiver. On every receiver you send to.
People say "I don't have deliverability problems!" but how do you know? Most places don't tell you they rejected your email.
Meh, one could also complain they don't have control over backbone networks, transit, peering agreements, and intermediary routing therefore hosting a service on k8s is futile without using a managed provider / PaaS.
> People say "I don't have deliverability problems!" but how do you know?
Because people reply to my emails.. because I email documents to family/friends/landlord/etc and they receive it as expected..
> Most places don't tell you they rejected your email.
> intermediary routing therefore hosting a service on k8s is futile without using a managed provider / PaaS.
Except that a managed service doesn't solve that for you. They are no better at that than you are. Email services are better at deliverability than you are, because they spend lots of time building their IP reputations and more importantly negotiating with mail providers to guarantee their emails show up.
> Because people reply to my emails.. because I email documents to family/friends/landlord/etc and they receive it as expected..
I'm guessing you don't confirm every email you send with every person though.
> Of course they do, this is what DMARC is for.
I was involved in the creation of DMARC (and SPF and DKIM) so I know how it's supposed to work, but in the real world, most providers do not honor the "reject" flag and actually send the bounces. Last time I dealt with it was a few years ago, maybe it's better now.
For context, I started my career at Sendmail, and I worked on the SPF and DKIM specs, so I've dealt with deliverability for 25+ years. I also ran my own mail server until around 2009. But I switched to Gmail as my primary around 2008, when deliverability just got too hard. But I still worked on commercial deliverability for years after that.
Granted, SPF and DKIM wasn't widely adopted at that point (and DMARK didn't exist), so maybe it's easier now. But at the same time, most of AWS/Azure/GCP are marked as bad automatically, as well as most home internet blocks.
So if you want to run your own mail server, you can't do it on your home router anymore, you have to rent a server in a rack and get a clean IP that's just for you. That costs $$$.
I see this common pattern where a previously private infrastructure is opened up (usually from low abstraction), and the ecosystem is split into an open base and a private thin layer, and that private layer might just reimplement the same tradeoffs that the incumbent private monoliths made.
Examples being Git/Github, Crypto/Centralized Exchanges, and as per the topic, email.
But I think that it's an important distinction that the base infrastructure is open, and that technically an incumbent could join the fray, albeit with a lot of catching up to do, and mix it up.
We are working on an open-source, self-hosted solution [0] to make this easier. When you correctly set up DKIM, SPF, reverse/forward DNS for IPs, it is not much hard to get emails delivered. IPs can still get blacklisted and you need to monitor blacklists and contact them if it happens. Solutions like Postfix are great, but they lack observability. In our solution, we have developed dashboards and health checks to make it easier to find problems with the set up.
We are currently running beta tests (really appreciate it if you can join).
There is a sweet spot between Gmail and self-hosting. I use Runbox and generally separate contexts, with CF being an exception as I use CF pages for static blog websites, some of their core services, AND as a registrar. For the latter, the default setting is porkbun. The reason for this is not CF's mandatory in-house DNS servers, but the simple fact that they do not register .de domains.
> It's unfortunate that email hosting and email infrastructure can really be done only well by major players. The days of people running and maintaining their own are pretty much long gone.
Its really not. Everyone can do that (doesn't mean everyone should). I'm running it for millions of emails daily and don't see why I would crappy proprietary service instead.
Fwiw, not a knock against CF. I like their products, mostly simple, fair pricing, etc. Just a bit unfortunate commentary on the state of email infra on the internet.