Live data from Hacker News

A serverless email server on AWS using S3 and SES

github.com

131–140 of 266 posts

Re: A serverless email server on AWS using S3 and SES

#131
Just a friendly reminder, since I've worked with SES in the past: Don't forget about bounces when using SES [0].

From [0]:

> If your bounce rate is 5% or greater, we'll place your account under review.

To sum it up, try to keep track of bounced e-mails by using the SES notifications [1].

[0] https://docs.aws.amazon.com/ses/latest/DeveloperGuide/e-faq....

[1] https://docs.aws.amazon.com/ses/latest/DeveloperGuide/monito...

Re: A serverless email server on AWS using S3 and SES

#132
post #38

Earlier quoted context omitted.

I still fail to understand why 4 years in, people seem to be confused about the idea that definitions of words involve over the years and that the computer industry has both been coining and adopting phrases. But a quick Google search is all that it would take to “understand it” https://en.wikipedia.org/wiki/Serverless_computing

I've found that "I don't even get what the word is supposed to mean" is someone's first experience with tech fear. In my experience (including my own personal anecdata) they tend to mean "I don't want to learn this new thing and because of that I'm worried I'll be left behind if everyone switches to this"

As I wrote in another comment, there is a world of software outside of web development. And I'm not especially fearful of being left behind. I spend almost all my free time learning new things. But I wouldn't really jump at the opportunity of learning something that locks me into a single company's services. Learning the fundamentals of computers seems like a much better use of my time.

Re: A serverless email server on AWS using S3 and SES

#133

Earlier quoted context omitted.

I still fail to understand why 4 years in, people seem to be confused about the idea that definitions of words involve over the years and that the computer industry has both been coining and adopting phrases. But a quick Google search is all that it would take to “understand it” https://en.wikipedia.org/wiki/Serverless_computing

Under this link I read: > Serverless computing is a cloud-computing execution model in which the cloud provider runs the server, and dynamically manages the allocation of machine resources. Pricing is based on the actual amount of resources consumed by an application, rather than on pre-purchased units of capacity. So I don't see too much of a difference from a VPS I pay for that will cost me more if I start using mo…

If I don’t have any messages in my queue/request to my API/messages in my stream, I am not paying for VMs or networking.

If I have thousands of messages in my queue, enough runtimes are started to handle the messages with the throughput I configure.

And as far as worrying about one vendor, the lock in boogeymen is vastly overrated. Once you have any type of scale, even if you are just using a bunch of VMs (and if you are you’re spending more money for a cloud provider than just using a colo without any of the benefits), once you have your data in the databases, network configurations, your security rules, your connection to the cloud provider with sitters Direct Connect, a site to site VPN, client to site VPNs for all of your developers, you’ve gone through security audits, etc. you’re for all intents and purposes locked in anyway where the pain of migration will be a multi month project with little to show for it.

It’s like all of those bushy tailed architects who are going to use the repository pattern and not use any database specific SQl just in case one day the CTO decides to move away from their six figure a year Oracle installation to Mysql.

Re: A serverless email server on AWS using S3 and SES

#134

I don't know anything about serverless --- to this day I fail to understand what this word is even supposed to mean. And the deployment diagram[1] sure looks complicated to me. I think I prefer old-school servers. [1]: " rel="nofollow">https://raw.githubusercontent.com/0x4447/0x4447-product-s3-e...

IaaS = your data + code to run it + create servers to run the code. PaaS = your data + code to run it. Servers automatically provided. SaaS = your data. Code and servers are automatically provided. Serverless is just another name for PaaS where your bring your code, which can be deployed as a single JS function in a text file or an entire container running anything.

PaaS is a collection of SaaS's. Your platform is made up of x,y,z services. Your services are made up of x,y,z infrastructure.

Re: A serverless email server on AWS using S3 and SES

#135

Earlier quoted context omitted.

As if the time to manage servers is free. But lambda is far from expensive. If you’re spending even $100 a month on lambda, you probably have a successful business. And everything is “complex” if you don’t know how to do it optimally.

> As if the time to manage servers is free. I work in projects which make heavy use of function-as-a-service, and I have first-hand accounts of how FaaS offerings have lead developers to waste a couple of weeks getting FaaS to do what could have been done in a couple of hours with a tried-and-true web service. > But lambda is far from expensive. Relatively to the cost of a VM instance? Yes, it's expensive. We're in a…

Let me give you another example.

We have a Jenkins server for builds. Most of the time, you push code and the build starts immediately. However, at times we have 10 builds in the queue and you’re sitting there waiting for your small build behind larger builds.

On the other hand, we’ve started moving to CodeBuild “Serverless builds”. When we aren’t running builds, we aren’t paying for it. However when crunch time comes, we can run up to 50 builds simultaneously (you can request a larger limit). How much would it cost to have build servers that could handle peak demand and how much of a waste would it be to have that server sitting idle?

And if you’re worried about the difference in cost between $100 and $10, you’re not running a business.

Will that $10 a month server scale up with demand?

As far as it taking “two or three weeks” - “its a poor craftsmen who blames his tools”. Don’t blame AWS for the missing skillset of your developers. There is no reason that it should take your developers any longer to build an API with lambda instead of a server.

In fact, you can host the same APIs with lambda by only adding two or three lines of code and using proxy integration.

For instance in Node:

https://github.com/awslabs/aws-serverless-express

Re: A serverless email server on AWS using S3 and SES

#136

Earlier quoted context omitted.

> When you are just using a VM - it’s your responsibility to keep the VMs and the runtimes patched, design a scaling solution, etc. Your comment did nothing to address the point, which is that your definiiton of "serverless" is actually just a buzzword used to refer to the old as the web commercial practice of managed hosting.

So was “running in the cloud” just a buzzword? Serverless is not just “managed hosting” in the cgi-bin since - serverless implies there is no server to manage and it’s for all intents and purposes “infinitely scalable”. For instance, Amazon’s hosted versions of Mysql, Postgres, Sql Server, etc are not what they consider “Serverless” you still have to size it appropriately and if your needs increase, have to move to a…

> So was “running in the cloud” just a buzzword?

Please don't digress. The questions regarding you assertions on "serverless" don't change or are addressed with your attempt to divert attentions to other buzzwords.

> Serverless is not just “managed hosting” in the cgi-bin since - serverless implies there is no server to manage and it’s for all intents and purposes “infinitely scalable”.

You're running in circles by trying to pin a buzzword on managed hosting.

> For instance, Amazon’s hosted versions of Mysql, Postgres, Sql Server, etc are not what they consider “Serverless”

I wouldn't use AWS as an example of coherent classification, as AWS's database offerings have multiple ovelapping concerns, including an overreaching gray area of "serverless-ness". See for example Amazon RDS and Amazon Aurora.

Re: A serverless email server on AWS using S3 and SES

#137
post #12

> This stack was created out of frustration due to the fact that to this day there's no easy way to have a full email server without the overhead of installing and configuring all servers needed to handle incoming and outgoing messages. Interesting approach, though I solved this frustration by the use of a Docker and kept "my data is mine" + "no vendor lock-in" + "I control all the gears" approach. (Though, it's not…

> Interesting approach, though I solved this frustration by the use of a Docker and kept "my data is mine" + "no vendor lock-in" + "I control all the gears" approach.

Yeah, I totally understand the desire. And even hosting this on a cloud, you benefit from SMTP TLS sometimes, presuming no active MITM and the cloud service not actively abusing its privileges on your VM or storage. Which is probably not happening widely. At least as opposed to the protocol level logging that SES or similar services do for sure.

I recently opened some new accounts at AWS and other large one for a new company... first thing to do is setup mail of course. Both denied my request to allow SMTP. AWS ominously rejected me with some vague "maybe one or more of these reasons" including prior ToS violations or payment issues with "linked" accounts.

Frankly it's scary, and so far they are stonewalling me on any details as to what I "may" have done. AWS is one place I sure don't want to have a bad reputation with.

Re: A serverless email server on AWS using S3 and SES

#138
I’ve been working on a small side project that involves processing incoming email. In particular, it’s an app that needs to do something for each email it receives from (hopefully paying!) users.

I am not interested in storing user mail, so SES is just too costly, at least according to a quick worst-case calculation.

That leaves me with two options:

1. Self-hosted Postfix

2. Mail service like Mailgun

With (1), there is no need to worry about overages, but scaling the mail server might be challenging.

The advantage of (2) over SES is that you are only charged a flat fee for each email, regardless of size. Emails are then automatically deleted after some period of time. Scaling up and down is easy.

For now, I am using Mailgun, but I am writing the mail processing daemon in a way that will make it easy to transition to Postfix, if needed.

Also, I decided to write the mail processing backend in Rust, so I’ve been learning the language as I go!

Re: A serverless email server on AWS using S3 and SES

#139

Earlier quoted context omitted.

I still fail to understand why 4 years in, people seem to be confused about the idea that definitions of words involve over the years and that the computer industry has both been coining and adopting phrases. But a quick Google search is all that it would take to “understand it” https://en.wikipedia.org/wiki/Serverless_computing

Not a single post on Hacker News can use the term "serverless" without the exact same replies being posted every time. It's as if a certain portion of the HN crowd simply cannot fathom that a new term exists and is in use, and instead resort to the same, tired responses.

Seems that you edit your comment quite a bit, eh?

---

> Not a single post on Hacker News can use the term "serverless" without the exact same replies being posted every time.

Indeed.

> It's as if a certain portion of the HN crowd simply cannot fathom that a new term exists and is in use, and instead resort to the same, tired responses.

If was only "a certain portion/crowd" it will have no responses at all.

I see more like:

"It's as if a certain portion of the HN crowd simply cannot fathom that servers exists and is easy to use, and instead resort to the same, tired responses."

Re: A serverless email server on AWS using S3 and SES

#140
post #124

Earlier quoted context omitted.

>cgi-bin didn’t have any type of process isolation "Lambdas" don't by nature either. I can run cgi-bin in a FirecrackerVM-like ecosystem with one script per VM. >Also, it didn’t tie in to other none web based events. Webhooks aren't new. I'm not disputing the popularity and convenience of serverless. I'm just noting it's somewhat like Docker. Packaging existing tech in an easier-to-use bundle.

Lambdas by nature do exactly that. AWS’s runtime launches the “VMs” in response to an event. And with webhooks I’m required to keep enough server capacity running all of the time to handle the events. Can a web hook automatically instantiate enough VMs on demand without reducing throughput?

"Can a web hook automatically instantiate enough VMs on demand without reducing throughput?"

Yes, there are various cgi-bin implementations with horizontal auto-scaling.

Post reply on HN