Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rethinking serverless with FLAME

fly.io

Post reply on HN
Print view View original

51–60 of 153 posts

Re: Rethinking serverless with FLAME

#51

Post by chrismccord »

gitgud wrote: post #47

> It then finds or boots a new copy of our entire application and runs the function there. So for each “Flame.call” it begins a whole new app process and copies the execution context in? A very simple solution to scaling, but I’d imagine this would have some disadvantages… Adding 10ms to the app startup time, adds 10ms to every “Flame.call” part of the application too… same with memory I suppose I guess these concern…

The FLAME.Pool discussed later in the post addresses this. Runners are pooled and remain configurable hot for whatever time you want before idling down. Under load you are rarely paying the cold start time because the pool is already hot. We are also adding more sophisticated pool growth techniques to the Elixir library next so you also avoid hitting an at capacity runner and cold starting one.

For hot runners, the only overhead is the latency between the parent and child, which should be the same datacenter so 1ms or sub 1ms.

Re: Rethinking serverless with FLAME

#52

Post by cchance »

bovermyer wrote: post #42

As an alternative to Lambdas I can see this being useful. However, the overhead concerns me. This would only make sense in a situation where the function in question takes long enough that the startup overhead doesn't matter or where the main application is running on hardware that can't handle the resource load of many instances of the function in question. I'm still, I think, in the camp of "monoliths are best in m…

He commented in another post that they use pooling so you don't really pay the cold start penalty as often as you'd think so maybe not a issue?

Re: Rethinking serverless with FLAME

#53

Post by darwin67 »

chrismccord wrote: post #49
tonyhb wrote: post #43

This is great! It reminds me of a (very lightweight) Elixir specific version of what we built at https://www.inngest.com/ . That is, we both make your existing code available to serverless functions by wrapping with something that, essentially, makes the code callable via remote-RPC . Some things to consider, which are called out in the blog post: Often code like this runs in a series of imperative steps. Each of the…

Ingest looks like an awesome service! I talk about job processors/durability/retries in the post. For Elixir specifically for durability, retries, and workflows we reach for Oban, which we'd continue to do here. The Oban job would call into FLAME to handle the elastic execution.

FYI: there's an Elixir SDK for Inngest as well. Haven't fully announced it yet, but plan to post it in ElixirForum some time soon.

https://github.com/inngest/ex_inngest

Re: Rethinking serverless with FLAME

#54

Post by rubenfiszel »

That's great. I agree with the whole thesis.

We took an alternative approach with https://www.windmill.dev which is to consider the unit of abstraction to be at the source code level rather than the container level. We then parse the main function, and imports to extract the args and dependencies, and then run the code as is in the desired runtime (typescript, python, go, bash). Then all the secret sauce is to manage the cache efficiently so that the workers are always hot regardless of your imports

It's not as integrated in the codebase as this, but the audience is different, our users build complex workflows from scratch, cron jobs, or just one-off scripts with the auto-generated UI. Indeed the whole context in FLAME seems to be snapshotted and then rehydrated on the target VM. Another approach would be to introduce syntax to specify what is required context from what is not and only loading the minimally required. That's what we are currently exploring for integrating better Windmill with existing codebase instead of having to rely on http calls.

Re: Rethinking serverless with FLAME

#55

Post by bbkane »

ryanjshaw wrote: post #24
chrismccord wrote: post #13

Author here. I’m excited to get this out and happy to answer any questions. Hopefully I sufficiently nerd sniped some folks to implement the FLAME pattern in js , go, and other langs :)

This looks great. Hopefully Microsoft are paying attention because Azure Functions are way too complicated to secure and deploy, and have weird assumptions about what kind of code you want to run.

I had a lot of problems trying to set up Azure Functions with Terraform a couple of years ago. Wonder if it's gotten better?

https://www.bbkane.com/blog/azure-functions-with-terraform/

Re: Rethinking serverless with FLAME

#56

Post by bo0tzz »

rubenfiszel wrote: post #54

That's great. I agree with the whole thesis. We took an alternative approach with https://www.windmill.dev which is to consider the unit of abstraction to be at the source code level rather than the container level. We then parse the main function, and imports to extract the args and dependencies, and then run the code as is in the desired runtime (typescript, python, go, bash). Then all the secret sauce is to manage…

> Indeed the whole context in FLAME seems to be snapshotted and then rehydrated on the target VM. Another approach would be to introduce syntax to specify what is required context from what is not and only loading the minimally required.

This isn't strictly what is happening. FLAME just uses the BEAM's built in clustering features to call a function on a remote node. That implicitly handles transferring only the context that is necessary. From the article:

> FLAME.call accepts the name of a runner pool, and a function. It then finds or boots a new copy of our entire application and runs the function there. Any variables the function closes over (like our %Video{} struct and interval) are passed along automatically.

Re: Rethinking serverless with FLAME

#57

Post by bo0tzz »

chrismccord wrote: post #51
gitgud wrote: post #47

> It then finds or boots a new copy of our entire application and runs the function there. So for each “Flame.call” it begins a whole new app process and copies the execution context in? A very simple solution to scaling, but I’d imagine this would have some disadvantages… Adding 10ms to the app startup time, adds 10ms to every “Flame.call” part of the application too… same with memory I suppose I guess these concern…

The FLAME.Pool discussed later in the post addresses this. Runners are pooled and remain configurable hot for whatever time you want before idling down. Under load you are rarely paying the cold start time because the pool is already hot. We are also adding more sophisticated pool growth techniques to the Elixir library next so you also avoid hitting an at capacity runner and cold starting one. For hot runners, the o…

Currently the per-runner concurrency is limited by a fixed number. Have you thought about approaches that instead base this on resource usage, so that runners can be used optimally?

Re: Rethinking serverless with FLAME

#58

Post by Nezteb »

rubenfiszel wrote: post #54

That's great. I agree with the whole thesis. We took an alternative approach with https://www.windmill.dev which is to consider the unit of abstraction to be at the source code level rather than the container level. We then parse the main function, and imports to extract the args and dependencies, and then run the code as is in the desired runtime (typescript, python, go, bash). Then all the secret sauce is to manage…

Oops you've got an extra w, here is the URL for anyone looking: https://www.windmill.dev/

I love the project's goals; I'm really hoping Windmill becomes a superior open-source Retool/Airtable alternative!

Re: Rethinking serverless with FLAME

#59

Post by freedomben »

bovermyer wrote: post #42

As an alternative to Lambdas I can see this being useful. However, the overhead concerns me. This would only make sense in a situation where the function in question takes long enough that the startup overhead doesn't matter or where the main application is running on hardware that can't handle the resource load of many instances of the function in question. I'm still, I think, in the camp of "monoliths are best in m…

I don't think this goes against "monoliths are best in most cases" at all. In fact it supports that by letting you code like it's all one monolith, but behind-the-scenes it spins up the instance.

Resource-wise if you had a ton of unbounded concurrency then that would be a concern as you could quickly hit instance limits in the backend, but the pooling strategy discussed lower in the post addresses that pretty well, and gives you a good monitoring point as well.

Post reply on HN