Live data from Hacker News

Ask HN: Have you shipped anything serious with a “serverless” architecture?

news.ycombinator.com

121–130 of 207 posts

Re: Ask HN: Have you shipped anything serious with a “serverless” architecture?

#121

Yes. We use both Lambda and Lambda@Edge, but for different reasons. Virtually all async processing we do is achieved by the main service (running as a normal microservice - no serverless stuff) pushing into an SQS queue, then a Lambda function running every minute pulls from the queue to e.g. report policies to our underwriters, issue policy documents, capture payments, etc. Essentially anything which happens after t…

As of May 14, lambda@edge supports node8 https://aws.amazon.com/about-aws/whats-new/2018/05/lambda-at...

Aha! That’s really excellent and will make it so much easier for us to support Lambda@Edge.

Thanks for letting me know :)

Re: Ask HN: Have you shipped anything serious with a “serverless” architecture?

#122
post #48

Just the opposite of serious for me. Example: there's a website for all the weather radar stations in Canada that lets you see the last two hours of 10-minute radar snapshots. I wanted a dump of them to try some ML algorithms, but even after requests and emails there simply wasn't one. So I set up a lambda to run every hour, load the website for all 31 weather stations and save all 6 images from the last hour to S3.…

'Serverless' is an adjective. I'm happy to trade some Internet Points™ for the chance to point that out.

Re: Ask HN: Have you shipped anything serious with a “serverless” architecture?

#123
At ipdata.co we use the same APIG+Lambda setup replicated in 11 regions to have the lowest latencies globally. We had to do some extra work to get API keys and rate limiting working but it was worth it. Our setup averages ~44ms response times - https://status.ipdata.co/.

We wrote about our setup in detail on the Highscalability blog [1]

A few things have changed since we wrote that article;

- We implement custom authorizers, which have helped lower our costs and the auth caching means authentication only happens once per x minutes and all subsequent requests are much faster.

- We use redis and a couple of Kinesis consumers running on a real server to sync data across all our regions. This setup has been battle tested and has successfully processed more than a hundred million API calls in a single day in near real time. [Use pipes and mget in redis for speed]

Here are some answers to a few specific things you raise in your question;

1. Use Sentry for lambda for error handling. The logs you get are incredibly detailed and have single handedly given us the greatest visibility into our application, moreso than any other tool we've tried (like AWS Xray).

2. Cloudwatch logs are tough. You might want to consider piping your logs to an Elasticsearch cluster, that might be a bit costly if you use AWS's Elasticache.

3. We use terraform for deploying our lambda functions and other resources. I'd strongly recommend it.

[1] https://highscalability.com/blog/2018/4/2/how-ipdata-serves-...

Re: Ask HN: Have you shipped anything serious with a “serverless” architecture?

#125
post #97

I can't name any names for obvious reasons or give you more hints about what industry this company is in but I just did DD on a very impressive outfit that ran their entire company on Google's cloud platform, it held about 500T of data and held up amazingly well under load. I was super impressed with how they had set this all up and they were extremely well aware of all the limitations and do's and dont's of that par…

Can you comment on their serverless architecture, which was the point of this post?

Well, given that it is serverless the infrastructure is operated by the provider, in this case Google.

That leaves the company to use the various APIs.

So you use Google Cloud Functions to ingest data and do all preliminary processing, store the data in one of the various persistent storage options (Spanner, Bigtable, whatever is best suitable for the job) processing optionally using background functions or containers for further processing or presentation.

Given that a reasonably short while ago I did not yet see Google as a serious contender in this space I'm actually surprised how far they have come.

You can basically create an enterprise class application dealing with vast amounts of data and never even know on what silicon (or where...) your processes are running.

Of course you still have to give some parameters, such as in which DC you want to run your stuff but on the whole it is about as painless as it can be.

For more info this would be a good starting point:

https://cloud.google.com/functions/

Important notes about the execution environment limits:

https://cloud.google.com/functions/quotas

Re: Ask HN: Have you shipped anything serious with a “serverless” architecture?

#126
post #107

Earlier quoted context omitted.

Jacques can absolutely tell the difference between CGP and serverless architecture, so if no more information is forthcoming, it's due to non-disclosure agreements ;)

In other words, someone wrote a cryptic comment and is unable to elaborate how much of it is relevant to the topic discussed because important non-disclosure. Cool.

I will disclose what I can short of anybody being able to determine the nature of the business, the vertical they operate in or which business it is.

Re: Ask HN: Have you shipped anything serious with a “serverless” architecture?

#127

Developing a SaaS product based on completely proprietary stack that you can't even host yourself is VERY dangerous!!! Just yesterday there was a report that twitter bought a company and immediately closed their API access to all customers. What will you do, if Amazon decide to close your AWS account for some reason? What if they discontinue one of the services you use? Here is a huge list of products discontinued by…

Most of the stuff in GCP won't be closed like the list as they are used by enterprise customers (Or at least, there will be proper notice, migration path etc). I agree that at the end of the day you have little control over the infrastructure, its impossible for a lot of companies to maintain them which is why they are going to cloud. If you are very worried, you can just use the VMs (EC2 or GCP's VMs etc) and not us…

I think the other more important path mentioned would be: "what happens in case of a ban/restriction?". Not with AWS, but we've previously had accounts locked/closed "accidentally" - thankfully they were non-mission critical.

Re: Ask HN: Have you shipped anything serious with a “serverless” architecture?

#128

Earlier quoted context omitted.

DynamoDB is as serverless as Lambda. (There are servers somewhere for both, but in neither case do you operate them.)

I'll give you there are no hosts to manage, but Lambda is serverless in the sense of "stateless" and "ephemeral". DynamoDB is still persistent storage. It seems that every every "hosted" solution is now being dubbed "serverless". :\ Oh and of course 5+ if-statements is now "AI".

Lamdas aren't stateless or ephemeral either. Anything that occurs on code load will persist on that container (i.e., if you initialize something at the module level in Node, it will persist between calls to the same container; this can cause all kinds of weirdness if you aren't aware of it. For instance, I have seen where a dev read some data at load time, and then performing destructive operations on it as part of data transformations in the code, and then wondered why he was getting non-deterministic results back). And there's a half gig of temp space on each container you can write to as well.

While the definition of what constitutes 'serverless' is pretty ambiguous, no one includes ephemeral state as a systems requirement, else you have something useless.

DynamoDB is generally viewed as serverless because there's no management of an underlying VM, and for some definitions because it can scale out horizontally automatically, without downtime, to meet demand (as compared with RDS, or another managed database solution that can only scale vertically).

Post reply on HN