Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rethinking serverless with FLAME

fly.io

Post reply on HN
Print view View original

91–100 of 153 posts

Re: Rethinking serverless with FLAME

#91

Post by chrismccord »

timenova wrote: post #88

I have a question about distributed apps with FLAME. Let's say the app is running in 3 Fly regions, and each region has 2 "parent" servers with LiveViews and everything else. In that case, how should the Flame pools look like? Do they communicate in the same region and share the pools? Or are Flame pools strictly children of each individual parent? Does it make a difference in pricing or anything else to run on hot w…

> Or are Flame pools strictly children of each individual parent?

Confirm. Each parent node runs its own pool. There is no global coordination by design.

> Does it make a difference in pricing or anything else to run on hot workers instead of starting up per parent?

A lot would depend on what you are doing, the size of runner machines you decide to start in your pools (which can be different sizes from the app or other pools), etc. In general Elixir scales well enough that you aren't going to be running your app in every possible region. You'll be in a handful of regions servicing traffic in those regions and the load each region has. You could build in your own global coordination on top, ie try to find processes running on the cluster already (which could be running in a FLAME runner), but you're in distributed systems land and it All Depends™ what you're building the tradeoffs you want.

Re: Rethinking serverless with FLAME

#92

Post by bo0tzz »

hq1 wrote: post #84

So how does it work if there are workers in flight and you redeploy the main application?

The workers get terminated. If the work they were doing is important, it should be getting called from your job queue and so it should just get started up again.

Re: Rethinking serverless with FLAME

#93

Post by timenova »

chrismccord wrote: post #91
timenova wrote: post #88

I have a question about distributed apps with FLAME. Let's say the app is running in 3 Fly regions, and each region has 2 "parent" servers with LiveViews and everything else. In that case, how should the Flame pools look like? Do they communicate in the same region and share the pools? Or are Flame pools strictly children of each individual parent? Does it make a difference in pricing or anything else to run on hot w…

> Or are Flame pools strictly children of each individual parent? Confirm. Each parent node runs its own pool. There is no global coordination by design. > Does it make a difference in pricing or anything else to run on hot workers instead of starting up per parent? A lot would depend on what you are doing, the size of runner machines you decide to start in your pools (which can be different sizes from the app or oth…

Thanks for the reply!

Can I suggest adding some docs to Fly to run Flame apps? To cover the more complex aspects of integrating with Fly, such as running Flame machines with a different size compared to the parent nodes, what kind of fly.toml config works and doesn't work with Flame, such as the auto_start and auto_stop configurations on the parent based on the number of requests, and anything else particularly important to remember with Fly.

Re: Rethinking serverless with FLAME

#94

Post by aidos »

I used a service years ago that did effectively this. PiCloud were sadly absorbed into Dropbox but before that they had exactly this model of fanning out tasks to workers transparently. They would effectively bundle your code and execute it on a worker.

There’s an example here. You’ll see it’s exactly the same model.

https://github.com/picloud/basic-examples/blob/master/exampl...

I’ve not worked with Elixer but I used Erlang a couple of decades back and it appears BEAM hasn’t changed much (fundamentally). My suspicion is that it’s much better suited for this work since it’s a core part of the design. Still, not a totally free lunch because presumably there a chance the primary process crashes while waiting?

Re: Rethinking serverless with FLAME

#95

Post by chrismccord »

hq1 wrote: post #84

So how does it work if there are workers in flight and you redeploy the main application?

If you're talking about inflight work that is running on the runner, there is a Terminator process on the runner that will see the parent go away, then block on application shutdown for the configured `:shutdown_timeout` as long as active work is being done. So active processes/calls/casts are given a configurable amount of time to finish and no more work is accepted by the runner.

If you're talking about a FLAME.call at app shutdown that hasn't yet reached the runner, it will follow the same app shutdown flows of the rest of your code and eventually drop into the ether like any other code path you have. If you want durability you'd reach for your job queue (like Oban in Elixir) under the same considerations as regular app code. Make sense?

Re: Rethinking serverless with FLAME

#96

Post by icedchai »

imafish wrote: post #75

Having dealt with the pain and complexity of a 100+ lambda function app for the last 4 years, I must say this post definitely hits the spot wrt. the downsides of FaaS serverless architectures. When starting out, these downsides are not really that visible. On the contrary, there is a very clear upside, which is that everything is free when you have low usage, and you have little to no maintenance. It is only later, w…

I couldn't even stand having a dozen lambdas. The app was originally built by someone who didn't think much about maintenance or deployment. Code was copy-pasted all over the place. Eventually, we moved to a "fat lambda" monolith where a single lambda serves multiple endpoints.

Re: Rethinking serverless with FLAME

#97

Post by viraptor »

imafish wrote: post #75

Having dealt with the pain and complexity of a 100+ lambda function app for the last 4 years, I must say this post definitely hits the spot wrt. the downsides of FaaS serverless architectures. When starting out, these downsides are not really that visible. On the contrary, there is a very clear upside, which is that everything is free when you have low usage, and you have little to no maintenance. It is only later, w…

> that you wish you had just gone the monolith route

Going from hundreds of lambdas to a monolith is overreacting to one extreme by going the other one. There's a whole spectrum of possible ways to split a project in useful ways, which simplify development and maintenance.

Re: Rethinking serverless with FLAME

#98

Post by thefourthchime »

I created something similar at my work, which I call "Long Lamda", the idea is that what if a lambda could run more than 15 minutes? Then do everything in a Lambda. An advantage of our system as is you can also run everything locally and debug it. I didn't see that with the FLAME but maybe I missed it.

We use it for our media supply chain which processes a few hundred videos daily using various systems.

Most other teams drank the AWS Step Koolaid and have thousands of lambas deployed, with insane development friction and surprisingly higher costs. I just found out today that we spend 6k a month on "Step Transitions", really?!

Re: Rethinking serverless with FLAME

#99

Post by conradfr »

solatic wrote: post #77
chrismccord wrote: post #51

Earlier quoted context omitted.

The FLAME.Pool discussed later in the post addresses this. Runners are pooled and remain configurable hot for whatever time you want before idling down. Under load you are rarely paying the cold start time because the pool is already hot. We are also adding more sophisticated pool growth techniques to the Elixir library next so you also avoid hitting an at capacity runner and cold starting one. For hot runners, the o…

Cold start time is the issue with most serverless runtimes. Your own mission statement states: "We want on-demand, granular elastic scale of specific parts of our app code." Doing that correctly is fundamentally a question of how long you need to wait for cold starts, because if you have a traffic spike, the spiked part of the traffic is simply not being served until the cold start period elapses. If you're running h…

In a Elixir/Phoenix app I don't think this will be really used for web traffic and more for background/async jobs.

Re: Rethinking serverless with FLAME

#100

Post by seabrookmx »

I'm firmly in the "I prefer explicit lambda functions for off-request work" camp, with the recognition that you need a lot of operational and organizational maturity to keep a fleet of functions maintainable. I get that isn't everyone's cup of tea or a good fit for every org.

That said, I don't understand this bit:

> Leaning on your worker queue purely for offloaded execution means writing all the glue code to get the data into and out of the job, and back to the caller or end-user’s device somehow

I assumed by "worker queue" they were talking about something akin to Celery in python land, but it actually does handle all this glue. As far as I can tell, Celery provides a very similar developer experience to FLAME but has the added benefit that if you do want durability those knobs are there. The only real downside seems you need redis or rabbit to facilitate it? I don't have any experience with them but I'd assume it's the same story with other languages/frameworks (eg ruby+sidekiq)?

Maybe I'm missing something.

Post reply on HN