Note that SES only retries delivery for a fixed 840 minutes (not configurable), which is an annoyingly too short amount of time.
A serverless email server on AWS using S3 and SES
251–260 of 266 posts
Re: A serverless email server on AWS using S3 and SES
#252> This stack was created out of frustration due to the fact that to this day there's no easy way to have a full email server without the overhead of installing and configuring all servers needed to handle incoming and outgoing messages. Interesting approach, though I solved this frustration by the use of a Docker and kept "my data is mine" + "no vendor lock-in" + "I control all the gears" approach. (Though, it's not…
Don't you have problems with deliverability? I've found sending emails directly from an AWS IP (i.e. not via SES) has major issues with reputation management. It's really easy to have outgoing emails mysteriously spam filtered.
Re: A serverless email server on AWS using S3 and SES
#253Earlier quoted context omitted.
As I wrote in another comment, there is a world of software outside of web development. And I'm not especially fearful of being left behind. I spend almost all my free time learning new things. But I wouldn't really jump at the opportunity of learning something that locks me into a single company's services. Learning the fundamentals of computers seems like a much better use of my time.
A lot of concepts that apply to serverless and other cloud computing services are applicable across providers as well. Even if the details are different. I would liken it to learning programming languages. Learning new languages isn't considered "locking you into a language". It's even widely considered beneficial because it makes learning more languages even easier and brings new perspectives to the table. Edit: An…
> I would liken it to learning programming languages. Learning new languages isn't considered "locking you into a language". It's even widely considered beneficial because it makes learning more languages even easier and brings new perspectives to the table.
Partly true, if you don't plan on using the language. But this seems only true, because the majority of programming languages today are open source and portable across different platforms. On the other hand, some time ago C# implied .Net Framework and that would lock you into the Microsoft+Windows ecosystem.
In the same way, using SQL that's understood only by PostgreSQL isn't going to be as much painful as using something specific to Oracle databases when the company behind it goes haywire as Oracle did with Java (if not for OpenJDK).
Re: A serverless email server on AWS using S3 and SES
#254Earlier quoted context omitted.
You can set up autoscaling groups via terraform just fine, with a little bit of care taken to ensure that you trigger on the right metrics. If anything mail is pretty much the easiest thing you can possibly pick to scale, because the inbound mail will be automatically retried. And haproxy in front of SMTP servers works just fine (really, any load balancer that can load balance raw TCP connections, but I've used hapro…
Sure, but for heavy/bursty traffic, you can still have downtime while new VMs spin up. Retries might save you or they might make the problem worse, depending on the size and pattern of the burst and how your auto-scaling config interacts with the retry config of various hosts. It may seem like a nitpick or something not worth worrying about, and for most that's probably the case. But for some businesses it could be a…
That is no different for serverless. You don't magically escape startup times - you need to carefully ensure that cold startup times are low enough, or that you maintain excess capacity to compensate.
The precise extent is different between different platforms depending on overheads, but that just means the point at which you need to trigger scaling is different.
You can find lots of descriptions of approaches people have taken to keep serverless instances running to avoid the cold start delays to work around this... For autoscaling groups you'd instead configure the alarm points used to trigger the scaling accordingly.
Serverless platforms tends to assume the startup will be fast enough to keep the connection open rather than return an error, but that is a load balancer config issue - you can set up retries and wait for other platforms too if it makes sense.
(Though for email this really does not matter - retries for all commonly used mail servers follow some form of exponential backoff up to many hours at least; retries works just fine in practice)
Re: A serverless email server on AWS using S3 and SES
#255Earlier quoted context omitted.
Sure, but for heavy/bursty traffic, you can still have downtime while new VMs spin up. Retries might save you or they might make the problem worse, depending on the size and pattern of the burst and how your auto-scaling config interacts with the retry config of various hosts. It may seem like a nitpick or something not worth worrying about, and for most that's probably the case. But for some businesses it could be a…
That is no different for serverless. You don't magically escape startup times - you need to carefully ensure that cold startup times are low enough, or that you maintain excess capacity to compensate. The precise extent is different between different platforms depending on overheads, but that just means the point at which you need to trigger scaling is different. You can find lots of descriptions of approaches people…
> That is no different for serverless. You don't magically escape startup times - you need to carefully ensure that cold startup times are low enough, or that you maintain excess capacity to compensate.
Serverless deployments are just another ladder step up the abstraction level, continuing the tradition that hardware doesn't matter. Similar to code compiled into assembly or a garbage collector managing memory. In the common cases, these cases are harmless (otherwise they wouldn't be popular), but they generally hide what's actually happening. Doing a garbage collection on a 200MB app is generally pretty snappy. But doing one on a 32GB server app can take seconds or minutes.
Abstractions like these are fine, as long as the limits of the abstraction are well understood. Sadly, that is rarely the case.
Re: A serverless email server on AWS using S3 and SES
#256Earlier quoted context omitted.
> Not a single post on Hacker News can use the term "serverless" wits nhout the exact same replies being posted every time. As a serverless skeptic/critic, the buzzword bingo aspect of serverless computing is far from being the problem, and it's a gross missrepresentation of the problems posed by serverless computing. The main problem with server applications such as AWS lambda, along with lack of control, is how utt…
There are absolutely valid criticisms of serverless (as you called out, vendor lock in and complexity for large applications being a big ones). But it's still an important and interesting technology that many individuals and companies are invested in, sharing side projects, open source tools (like the OP's), etc. In many cases, it can be cost efficient, or even (human) resource efficient to use a serverless setup. So…
> But it's still an important and interesting technology that many individuals and companies are invested in, sharing side projects, open source tools (like the OP's), etc.
That says nothing about the technical merits. In fact, it's just an appeal to popularity mixed with a touch of appeal to authority. Technologies should be judged by their technical merits.
> In many cases, it can be cost efficient, or even (human) resource efficient to use a serverless setup.
The argument in favor of the cost effectiveness of Function-as-a-Service (FaaS) offerings is exclusively in the side of the service provider. By convincing clients to give up reserving VM instances and instead switch to a glorified batch job service, service providers are able to increase their capacity with the exact same infrastructure. That's the main selling point. Afterwards, charging customers a premium to use the service is a secondary bonus.
> So it gets quite repetitive to still be debating that, yes, we know there are servers running the code, it's just a name.
Again, this line of argument is simply silly and entirely misses (or worse, avoids) the actual criticism being directed at the whole "serverless" fad. It'suite bluntly nothing more than a strawman.
> Also, a lot of the value in serverless lies in time saved managing said servers
This whole line of argument also makes no sense once we factor in the fact that containerization is a thing and nowadays launching web services is as hard as launching a batch job. In fact, some container orchestration services already support plain old batch jobs through Docker containers. Just build a container with your "serverless" code, launch the container, let the "serverless" code do its thing, and wait until completion. No "server" involved whatsoever.
Re: A serverless email server on AWS using S3 and SES
#257>his day there's no easy way to have a full email server without the overhead of installing and configuring all servers needed to handle incoming and outgoing messages.
Re: A serverless email server on AWS using S3 and SES
#258Earlier quoted context omitted.
Fraud detection is such a frustrating double edged sword. They can’t share what was detected or why because the bad guys will start taking it into account. That leaves us with manual human review as the only means to address false positives. But that doesn’t scale so it’s either backlogged, low quality or nonexistent.
Rather seems likely laziness hiding behind the guise of "security". Such a bad user experience is inexcusable.
I think the issue is that not very many customers want to maintain their own mail server anyway, so it's just easier to turn down these requests unless the customer asking spends enough money that you don't want to offend them.
Re: A serverless email server on AWS using S3 and SES
#259Earlier quoted context omitted.
The SES and S3 side of things would be free? (62,000 free per month) https://aws.amazon.com/ses/pricing/ Yes, complexity would be a factor for sure.
For receiving they only get 1,000 free, and for both after that it's $0.10/1,000 emails, which isn't very much at all.
what if you get 10 billion unwanted messages? 900k bill?
Re: A serverless email server on AWS using S3 and SES
#260Earlier quoted context omitted.
There are absolutely valid criticisms of serverless (as you called out, vendor lock in and complexity for large applications being a big ones). But it's still an important and interesting technology that many individuals and companies are invested in, sharing side projects, open source tools (like the OP's), etc. In many cases, it can be cost efficient, or even (human) resource efficient to use a serverless setup. So…
> But it's still an important and interesting technology that many individuals and companies are invested in, sharing side projects, open source tools (like the OP's), etc. That says nothing about the technical merits. In fact, it's just an appeal to popularity mixed with a touch of appeal to authority. Technologies should be judged by their technical merits. > In many cases, it can be cost efficient, or even (human)…
Afterwards, charging customers a premium to use the service is a secondary bonus.
If I have a process that either only needs to run sporadically and/or is very spikey - web request, events, etc. why wouldn’t it be cheaper if you only need peak capacity 20% instead of paying for peak capacity all of the time when you don’t need it?
It doesn’t take an advanced degree in math to determine where the break even point between paying more per second for extreme elasticity is cheaper than paying for an always available resource.
If you’re using a cluster, traditional you’re still paying for peak capacity of your cluster all of the time.
But guess what? AWS also offers Serverless Docker - Fargate and the same math applies, you pay slightly more for the elasticity and not having to spend time managing a cluster of servers for the convenience and elasticity.
But either way, if your business is so undercapitalized that the difference between serverless and reserved capacity, you have bigger issues. Also, the more pets you can get rid of and the more automation and training of your developers you can do, if you can just reduce your infrastructure staff by one person, it will more than pay for itself.