81–90 of 114 posts
Earlier quoted context omitted.
Theoretically, if container start up times are around 125ms it should be possible to achieve this with Fargate + Knative's "scale to zero" functionality[1]. AWS is already working on improving Fargate/Kubernetes integration. [1] https://cloud.google.com/knative/
Doesn't a Kubernetes master on Fargate cost a significantly non-zero amount of money? Pretty sure it does.
What this is allows, and I'm hoping a full fledged service will be announced on Thursday or Friday, is running containers as Lambdas. i.e. if you application starts fast enough, you can just set a container to start and run as a request comes in. It can also shut down when it's done running. This allows things like per second billing for container runs, serverless containers (there's no container running 24/7, only w…
Earlier quoted context omitted.
The VPC startup times are insane, so we quickly move our lambdas out of that, accepting the trade off. Our normal cold starts are in the 1-2 second range, and the app initialization comes after. Too high for an API facing users :/
We got around this with a bit of a hack - use a CloudWatch event to trigger a dummy invocation of your function every five minutes. This keeps the container "hot" and reduces the start time (and is negligible cost-wise). This won't fix the cold starts when the function scales up, but it does reduce latency for 99% of our API requests.
The obvious solution would be to just merge microservices into the same lambda, but then we'd rather switch to EKS or smth, and actually be able to utilize microservice architecture fully.
To give a little more context, we have a bunch of microservices exposing GraphQL endpoints/schemas. We then have a Gateway which stitches these together, and exposes a public GraphQL schema. Because of the flexibility (by design) of GraphQL, we can easily end up invoking multiple parallel calls to several microservices, when the schema gets transformed in the Gateway.
This works really well, and gives a lot of flexibility in designing our APIs, especially utilizing microservices to the full extent. It also works really well when the lambdas are already warm, but when we then get one cold start, amongst them all, suddenly we go from responses in ms, to responses in seconds, which I don't think is acceptable.
We've been shaving off things here and there, but we are at the mercy of cold starts more of less. So our current plan is to migrate to an EKS setup, we just need to get a fully automated deployment story going, to replace our current CI/CD setup, which heavily uses the serverless framework.
Earlier quoted context omitted.
Then again, you are doing the hacks you describe because it is not scaling up nearly instantly. The cold start delays are not only an issue when scaling from zero to one, they hit you whenever you scale the capacity up.
This. I continue to hear about hacks of people running ping to keep a single instance warm. But that doesn't cover periodic changes in capacity needs nor spikes. I would think to avoid cold starts all together you'd need a pinger that sent exactly the load difference between peak load and current load. I would love to hear if anyone is keeping Lambdas warm at more than n=1 capacity.
There are various ways to do it, but I feel that it's a very suboptimal solution, and it still won't guarentee no cold starts happen.
I've personally come to the conclusion, that lambda is very nice for anything non-latency sensitive. We are still using it to great effect for e.g. processing incoming IoT data samples, which can vary quite a lot, but only happens in the backend, and nobody will care if it's 1-2 seconds delayed.
Earlier quoted context omitted.
I need to run 1+ Fargate containers 24/7, which is useless and wasteful. With Fargate-Lambda crossover I wouldn't be running anything 24/7, and it would be a lot less resource intensive than one Lambda-Container per request as well. Google's App Engine gets / got this right when they first launched, but to make it work they had to demand apps be written for their sandbox (like AWS Lambda), because of which the model…
I think with the App Engine Standard generation 2 runtimes you don't have to write to their sandbox anymore. It still has to be one of their supported languages though, instead of any arbitrary server. https://cloud.google.com/blog/products/gcp/introducing-app-e...
Earlier quoted context omitted.
Why run a container? What value does that abstraction provide here? To my mind this completely negates any value proposition of the container. The only thing missing, at face value, is something as straightforward as the Dockerfile for building base images. I imagine that shouldn't be hard using things like guestfish etc in guestfstools.
Want to run Ruby on Rails on Lambda, with no changes from the servers I run on my laptop. Or maybe I want to run Crystal. Or maybe I’m writing my own language. Doesn’t really matter. Lambda works great as a deployment and execution model. This allows anything to run on Lambda, not just specially prepared runtimes.
still seems much slower than the model used by Cloudflare for what they call "workers."[1] A recent blog post a few weeks back was the subject of considerable discussion here[2], and it seems to me to be doing much the same thing as Firecracker, but still faster because there's less overhead. But maybe I'm missing something. [1] https://blog.cloudflare.com/cloud-computing-without-containe... [2] https://news.ycombina…
Earlier quoted context omitted.
I need to run 1+ Fargate containers 24/7, which is useless and wasteful. With Fargate-Lambda crossover I wouldn't be running anything 24/7, and it would be a lot less resource intensive than one Lambda-Container per request as well. Google's App Engine gets / got this right when they first launched, but to make it work they had to demand apps be written for their sandbox (like AWS Lambda), because of which the model…
I think with the App Engine Standard generation 2 runtimes you don't have to write to their sandbox anymore. It still has to be one of their supported languages though, instead of any arbitrary server. https://cloud.google.com/blog/products/gcp/introducing-app-e...
(and, as thesandlord notes below, arbitrary containers is the idea behind g.co/serverlesscontainers)
Disclaimer: I work on GCP.
The crosvm and Rust have me intrigued. I was hoping for something like this since I saw the first hints of Rust showing up in ChromeOS in crosvm. A compare/contrast with Kata Containers would also be interesting. Their architectures look similar. (Kata Containers [1] being another solution for running containers in KVM-isolated VMs, that has working integrations with Kubernetes and containerd already. Not affiliated,…
Kata Containers is a lot of infrastructure for running containers and it uses QEMU to run the actual VMs. Firecracker just replaces the QEMU part and we're eager to work with folks like the Kata community. I love QEMU, it's an amazing project, but it does a ton and it's very oriented towards running disk images and full operating systems. We wanted to explore something really focused on serverless. So far, I'm really…