I'm curious about restarts and snapshotting: would it be feasible to reset a microVM for each incoming request?
Firecracker: Secure and fast microVMs for serverless computing
51–60 of 107 posts
Re: Firecracker: Secure and fast microVMs for serverless computing
#52Firecracker is great We use it to run fleets of fast booting vms at https://fly.io . It’s really the best OSS to come out of Amazon.
I dabbled with an idea similar to Fly.io's Heroku supercharging functionality (Turboku?). One issue I encountered is that the app in question does not benefit from full-page caching. Even if we deployed our app through Fly.io, we'd still have our databases hosted somewhere else. How does Fly.io solve this, or how could we solve this? When I dabbled with this idea, I thought about deploying DB read-only replicas aroun…
You've pretty much nailed the problem. The ?good? news though is that Heroku is really slow, so just running Firecracker VMs on real hardware, doing edge TLS, and adding http2 + brotli is a huge win.
When people use https://fly.io/heroku, we launch VMs in the same region their Heroku app is in so there's no latency hit to the DB. Weirdly, latency between a Fly app a DB on AWS in the same region is sometimes even better than AWS cross-zone latency.
We _also_ give apps a special global Redis cache (https://fly.io/docs/redis). This is sometimes enough to make a full stack app multi-regional, usually people cut way down on DB queries when they use their framework's caching abilities, which can make it pretty nice to run a Rails app + cache in, say, San Jose while the DB is in Virginia.
I know of a couple of devs running Elixir apps on Fly that leave a data service in the region where their DB is and basically rpc to it from other regions, which seems to work well.
Read replicas are a good idea, we'd actually like to try that out at some point. It seems pretty doable to put something like pgbounce/pgpool in front of a read replica and let it handle routing write transactions properly.
Re: Firecracker: Secure and fast microVMs for serverless computing
#53I'm curious about restarts and snapshotting: would it be feasible to reset a microVM for each incoming request?
Snapshot / suspend isn't doable yet, but it sure would be nice.
We've booted some apps in Firecracker in <20ms, so if you build the app properly you can absolutely do a new VM per request or TCP connection even.
Re: Firecracker: Secure and fast microVMs for serverless computing
#54Firecracker is great We use it to run fleets of fast booting vms at https://fly.io . It’s really the best OSS to come out of Amazon.
Hi Kurt,
What are usual cold-start times you see with firecracker?
What other VMMs or Unikernels did you consider before settling on firecracker?
Was the firecracker documentation good enough or did you have to go digging through emails or code to figure out certain things?
What was the hardest part of using firecracker in production?
Thanks.
Re: Firecracker: Secure and fast microVMs for serverless computing
#55I'm curious about restarts and snapshotting: would it be feasible to reset a microVM for each incoming request?
Snapshot / suspend isn't doable yet, but it sure would be nice. We've booted some apps in Firecracker in <20ms, so if you build the app properly you can absolutely do a new VM per request or TCP connection even.
Yeah, this is what AWS does....
Re: Firecracker: Secure and fast microVMs for serverless computing
#56Earlier quoted context omitted.
Intel has been an absolute security shit-show for over 3 years now. The shift is underway but it takes time.
Meltdown effects all the way back to 2nd gen(2010), it's been bad for a decade we just didn't know.
Meltdown goes much further back than that, potentially all the way to the Pentium Pro (1995)
It was also independently implemented in POWER and ARM A75
Re: Firecracker: Secure and fast microVMs for serverless computing
#57I'm curious about restarts and snapshotting: would it be feasible to reset a microVM for each incoming request?
Might consider a pre-emptive approach that always keeps a few warmed up and ready to receive a request. The determination on how many to keep warm could be a function of traffic over the last X period of time.
Re: Firecracker: Secure and fast microVMs for serverless computing
#58Earlier quoted context omitted.
I dabbled with an idea similar to Fly.io's Heroku supercharging functionality (Turboku?). One issue I encountered is that the app in question does not benefit from full-page caching. Even if we deployed our app through Fly.io, we'd still have our databases hosted somewhere else. How does Fly.io solve this, or how could we solve this? When I dabbled with this idea, I thought about deploying DB read-only replicas aroun…
You've pretty much nailed the problem. The ?good? news though is that Heroku is really slow, so just running Firecracker VMs on real hardware, doing edge TLS, and adding http2 + brotli is a huge win. When people use https://fly.io/heroku , we launch VMs in the same region their Heroku app is in so there's no latency hit to the DB. Weirdly, latency between a Fly app a DB on AWS in the same region is sometimes even bet…
> The ?good? news though is that Heroku is really slow
Did you do any measurements and if so, on which dyno types? We found that using Performance-M dynos' gives us a rather large performance boost. Performance-m dyno's are also more stable because they run on dedicatd hardware. They're expensive, but we don't run any apps in production without it.
One thing that worked really well for us is to just put Cloudflare or Cloudfront in front of our app. As I mentioned, we don't do any full-page caching. We cache pretty much everything else, but pages themselves have zero caching (business requirement). I believe Cloudflare and Cloudfront also do edge TLS.
> Read replicas are a good idea, we'd actually like to try that out at some point. It seems pretty doable to put something like pgbounce/pgpool in front of a read replica and let it handle routing write transactions properly.
This is going to be tricky. We weren't able to set up replication from Heroku Postgres databases to hosts outside of Heroku. Another thing to keep in mind that is it might be better to let the app decide what is a read query and what is a write query. We have some parts of the app that we need reading directly from the master, so we let the app handle it. The app receives two database URI's, both pointing to pgbouncer.
Re: Firecracker: Secure and fast microVMs for serverless computing
#59I played with Weave Ignite the other day which is a Docker-like CLI for Firecracker. Sure there were some rough edges but the overall experience was pretty good. If you are familiar with the Docker CLI you will be able to get some virtual machines up and running very quickly. Two questions in case someone from Weave tunes into the discussion: I got the impression that VMs needed an SSH server to be accessible. Is thi…
> I got the impression that VMs needed an SSH server to be accessible. Is this correct and if so will it be possible to implement something similar to docker exec so that I won't need an SSH server on every VM? If you want the capability to exec processes from the host into the VM, I think either Docker API or Kube API is the thing for that, as I understand it. If you could kernel exec processes directly into a VM, t…
i think using an emulated serial connection or similar driver interface could be part of a solution to this.
Re: Firecracker: Secure and fast microVMs for serverless computing
#60Earlier quoted context omitted.
You've pretty much nailed the problem. The ?good? news though is that Heroku is really slow, so just running Firecracker VMs on real hardware, doing edge TLS, and adding http2 + brotli is a huge win. When people use https://fly.io/heroku , we launch VMs in the same region their Heroku app is in so there's no latency hit to the DB. Weirdly, latency between a Fly app a DB on AWS in the same region is sometimes even bet…
> The ?good? news though is that Heroku is really slow Did you do any measurements and if so, on which dyno types? We found that using Performance-M dynos' gives us a rather large performance boost. Performance-m dyno's are also more stable because they run on dedicatd hardware. They're expensive, but we don't run any apps in production without it. One thing that worked really well for us is to just put Cloudflare or…
I tried adding Cloudfront in front of an app hosted on Heroku with page caching off (Vary by Cookie) and it increased latency to 3X, never could figure out why. Would like to do it but seems like way too much of a trade off.