Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A serverless email server on AWS using S3 and SES

github.com

Post reply on HN
Print view View original

171–180 of 217 posts

Re: A serverless email server on AWS using S3 and SES

#171

Post by jpalomaki »

kureikain wrote: post #76

This is great but it's a bit misleading. It only store email on S3, that mean you cannot read email with your email client. Seem thing with outgoing, you upload email to s3 in a certain format to be send out, not smtp protocol This is more like an API for sending and receiving. You cannot say generate SMTP credential and drop in your mail client. Also, one of the limitation with AWS SES is you cannot sent FROM or RET…

Back in the days I remember storing emails in similar way and reading using an email client. It was called ”maildir” [1].

Emails were stored as files in folder structure. I think this goes quite nicely with the unix ideas as you could use many standard tools to work with the messages.

[1] https://en.m.wikipedia.org/wiki/Maildir and

Re: A serverless email server on AWS using S3 and SES

#172

Post by dahfizz »

greatgib wrote: post #166

I'm a little bit disappointed by the solution of this article. They start with: "This stack was created out of frustration due to the fact that to this day there's no easy way to have a full email server without the overhead of installing and configuring all servers needed to handle incoming and outgoing messages." Ok, so the point is that it is too complicated/frustrating to deploy an email server. But look at their…

There are a class of engineers who have very little Linux experience, but use AWS extensively. For them, "Setting up a server" is always going to be more complex and frustrating than using the platform that they are familiar with, even when the AWS setup they come up with to replace a single server is way more complex to an "outsider".

Personally, I don't think there is an excuse to not learn Linux. It's usually the right tool for the job, and you can use tooling like ansible and AWS instances to get your stack cloud based, stateless, automated, yada yada buzzword . But I can't really blame people for using the tools they are comfortable with.

Re: A serverless email server on AWS using S3 and SES

#173

Post by danielvinson »

irae wrote: post #104

This is a brilliant architecture. Everyone here talking about no email clients, IMAP and AWS restrictions is missing the point. Although you could use this as a personal account, or as a company. This fully automated system is a perfect ground layer to a number of applications with a very cleaver API. Say you are a small startup, and you will need some email functionality? Give your web server access to writing to ou…

As somebody who has been an early engineer at many startups, I would never use this in almost any case because a managed solution is just better. If I can pay somebody else to do something hard with zero maintenance so I can focus on shipping a good product, I’m going to do that every time.

Re: A serverless email server on AWS using S3 and SES

#174

Post by OJFord »

kureikain wrote: post #165
OJFord wrote: post #114

Earlier quoted context omitted.

I do something similar to OP [0] - and currently forward to another provider (single address, but that's only the 'envelope' To; so it doesn't matter, I preserve the original whatever.to.address). That's mostly an artefact of transitioning from that provider to my own on AWS, but I stopped working on it for some time and never continued. My plan was/still is to write a client that can read 'Just a Bunch of .Eml' form…

Yes, I used something like that as well. But, are you sure you can set "From" header to anything(even for email/domain that aren't verified with SES). So that on receiver side(gmail for example) it will show the original sender. In my account(in production mode, not being sandbox), I couldn't. When I attempted to set "From" header(or "Return-Path") I got these error 554 Message rejected: Email address is not verified…

No, you can't set it to anything, you'll fail DKIM/DMARC/SPF. Just leave it as it is!

You just wrap the original email in an 'envelope' with envelope-to (you @ other provider) and envelope-from (whatever @ verified with SES).

You can see this here, if it helps at all, though it's a bit DSL-y: https://github.com/OJFord/amail/blob/c602f4d36c1bb9df4da1744...

Re: A serverless email server on AWS using S3 and SES

#175

Post by antihero »

dahfizz wrote: post #172
greatgib wrote: post #166

I'm a little bit disappointed by the solution of this article. They start with: "This stack was created out of frustration due to the fact that to this day there's no easy way to have a full email server without the overhead of installing and configuring all servers needed to handle incoming and outgoing messages." Ok, so the point is that it is too complicated/frustrating to deploy an email server. But look at their…

There are a class of engineers who have very little Linux experience, but use AWS extensively. For them, "Setting up a server" is always going to be more complex and frustrating than using the platform that they are familiar with, even when the AWS setup they come up with to replace a single server is way more complex to an "outsider". Personally, I don't think there is an excuse to not learn Linux. It's usually the…

Frankly I don't understand why they didn't use this using CDK.

Re: A serverless email server on AWS using S3 and SES

#176

Post by antihero »

twistedpair wrote: post #157
throwaway894345 wrote: post #36

Earlier quoted context omitted.

> serverless server? This isn’t a contradiction. “Server” is an overloaded term and the “server” in “serverless” refers to a different meaning of “server” than the “email server” bit. “Serverless” means there is no physical or virtual hosts to manage, you just supply the request handler i.e., the email server . It's amusing because it sounds like a contradiction, but it's not one.

Cloud 2.0 "serverless" often expects: - Just add code/logic - Scales to zero (zero calls to your endpoints, $0/mo) - Scales up automagically to match load - You don't do anything to provision/patch servers (e.g. FaaS) Classic PHP shared hosts don't meet this definition because: - You paid a flat monthly hosting fee (e.g. Dreamhost) - You only scaled to 1 node (need more, too bad) True, you didn't usually need to mess…

I wonder if you could, for reasons, make a set-up where you upload your PHP files to an S3 bucket, and then have a API Gateway or Lambda which basically just serves the files from the bucket, and if they are PHP, executes them with a Lambda PHP runtime.

So you hit /index.php -> checks S3 bucket for index.php -> executes that with lambda and gets string of response -> returns it to original lambda -> serves up to client.

Re: A serverless email server on AWS using S3 and SES

#177

Post by throwaway894345 »

mvanbaak wrote: post #155
throwaway894345 wrote: post #40

I was trying to set up Amazon Cognito (an authentication service) recently which basically needs a verified email address to send users emails for user verification and password reset, and I was shocked that Amazon has no basic email hosting service. Of course it’s easy enough to work around for my use case, but I was surprised nonetheless.

They have workmail.

I saw that, but it was like $5/user/month and geared toward employee email rather than automation (my use case is my hobby projects for which my goal is <$5/month in total).

Re: A serverless email server on AWS using S3 and SES

#178

Post by throwaway894345 »

vinay_ys wrote: post #168
twistedpair wrote: post #159

Earlier quoted context omitted.

Amen. How many times did you update your Node.js AWS Lambdas or GCP Cloud Functions because of the Linux kernel CVE of the week? You didn't because all you're responsible for is your few lines of Node.js logic that kept on scaling and humming along. The cloud vendor cares for the rest.

Just because you didn't doesn't mean they did and you didn't actually have any vulnerabilities. There is no such provable attestation of security in serverless model.

> Lambda provides support for these runtimes by continuously scanning for and deploying compatible updates and security patches, and by performing other runtime maintenance activity.

https://docs.aws.amazon.com/whitepapers/latest/security-over...

I'm not sure if that addresses your concern (maybe you're worried they're lying or they have a bug in their process)?

Re: A serverless email server on AWS using S3 and SES

#179

Post by kureikain »

OJFord wrote: post #174
kureikain wrote: post #165

Earlier quoted context omitted.

Yes, I used something like that as well. But, are you sure you can set "From" header to anything(even for email/domain that aren't verified with SES). So that on receiver side(gmail for example) it will show the original sender. In my account(in production mode, not being sandbox), I couldn't. When I attempted to set "From" header(or "Return-Path") I got these error 554 Message rejected: Email address is not verified…

No, you can't set it to anything, you'll fail DKIM/DMARC/SPF. Just leave it as it is! You just wrap the original email in an 'envelope' with envelope-to (you @ other provider) and envelope-from (whatever @ verified with SES). You can see this here, if it helps at all, though it's a bit DSL-y: https://github.com/OJFord/amail/blob/c602f4d36c1bb9df4da1744...

By envelope-from (whatever @ verified with SES), do you mean the `MAIL FROM` command in SMTP protocol or do you mean the `header FROM` in the email header?

I don't know how `send_raw` work under the hood in Rust, but without a way to set `from` header. how do recipient(say an @gmail.com) address show the original from too?

Did I miss something here? It would be great if somehow the email is forward as it's in so it appear in my inbox and all the information(header) is retain.

> just wrap the original email in an 'envelope'

It would be great if you can help clear my mind on how to do that. Say I got an email as this:

  FROM: original-from
  DKIM: dkim sign
  Return-Path: etc
  Other header:

  Body of email appear here
How can I use AWS SES to forward that email as-is to my @gmail.com account for example while still keep in the original header intact(so DKIM/DMARC still work. SPF will be break but it's fine for me)

Re: A serverless email server on AWS using S3 and SES

#180

Post by disgrunt »

asperous wrote: post #161
benbristow wrote: post #110

Earlier quoted context omitted.

Different thing though. When you uploaded your PHP script, you uploaded it to a single server, and it ran on a single server. Unless your provider were doing some kind of load balancing (doubt it). With serverless you upload your 'script' to the 'cloud' and then it will run on any number of servers, usually assigned to one when it dry-runs. Then it'll unload from the server afterwards. And therefore unless you have a…

At least with Mediatemple’s shared hosting, they had a “grid” service that would host your site on a cluster of servers and claimed to be able to handle large bursts of traffic. It worked because each page request ran a complete cgi php script to generate the page view and then terminated. There was no long running server.

It worked. But I remember Media Temple being slower than their competition. Seemed to have to pay a performance penalty for the "grid".
Post reply on HN