Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rethinking serverless with FLAME

fly.io

Post reply on HN
Print view View original

41–50 of 153 posts

Re: Rethinking serverless with FLAME

#41

Post by agundy »

chrismccord wrote: post #28
agundy wrote: post #23

Looks like a great integrated take on carving out serverless work. Curious to see how it handles the server parts of serverless like environment variables, db connection counts, etc. One potential gotcha I'm curious if there is a good story for is if it can guard against code that depends on other processes in the local supervision tree. I'm assuming since it's talking about Ecto inserts it brings over and starts the…

It starts your whole app, including the whole supervision tree, but you can turn on/off services based on whatever logic you want. I talk a bit about this in the screencast. For example, no need to start the phoenix endpoint (webserver) since we aren't serving web traffic. For the DB pool, you'd set a lower pool size or single connection in your runtime configuration based on the presence of FLAME parent or not.

Oh cool! Thanks for the reply, haven't had time to watch the screencast yet. Looking forward to it.

Re: Rethinking serverless with FLAME

#42

Post by bovermyer »

As an alternative to Lambdas I can see this being useful.

However, the overhead concerns me. This would only make sense in a situation where the function in question takes long enough that the startup overhead doesn't matter or where the main application is running on hardware that can't handle the resource load of many instances of the function in question.

I'm still, I think, in the camp of "monoliths are best in most cases." It's nice to have this in the toolbox, though, for those edge cases.

Re: Rethinking serverless with FLAME

#43

Post by tonyhb »

This is great! It reminds me of a (very lightweight) Elixir specific version of what we built at https://www.inngest.com/.

That is, we both make your existing code available to serverless functions by wrapping with something that, essentially, makes the code callable via remote-RPC .

Some things to consider, which are called out in the blog post:

Often code like this runs in a series of imperative steps. Each of these steps can run in series or parallel as additional lambdas. However, there's implicit state captured in variables between steps. This means that functions become workflows. In the Inngest model, Inngest captures this state and injects it back into the function so that things are durable.

On the note of durability, these processes should also be backed by a queue. The good thing about this model is that queues are cheap. When you make queues cheap (eg. one line of code) everything becomes easy: any developer can write reliable code without worrying about infra.

Monitoring and observability, as called out, is critical. Dead letter queues suck absolute major heaving amounts of nauseous air, and being able to manage and replay failing functions or steps is critical.

A couple differences wrt. FLAME and Inngest. Inngest is queue backed, event-driven, and servable via HTTP across any language. Because Inngest backs your state externally, you can write a workflow in Elixir, rewrite it in Typescript, redeploy, and running functions live migrate across backend languages, similar to CRIU.

Being event-driven allows you to manage flow control: everything from debounce to batching to throttling to fan-out, across any runtime or language (eg. one Elixir app on Fly can send an event over to run functions on TypeScript + Lambda).

I'm excited where FLAME goes. I think there are similar goals!

Re: Rethinking serverless with FLAME

#44

Post by sgarland »

quaunaut wrote: post #11
sgarland wrote: post #9

Earlier quoted context omitted.

Because they are not concerned with those things, the lack of knowledge in those areas creeps in. If you don’t understand how the OS works at a very fundamental level, you will write sub-optimal code. This is usually fine, because people are smart, compilers are good, and hardware is absurdly fast. However, the fact remains that it’s probably giving up some bits here and there. More germane to my point, though, the a…

All true, but you've got a few issues: - Flame still assumes you have a server somewhere- this is purely for elastic workloads. - Dynamically scaling horizontally is not something "linux" can do for you: It's a legitimately hard problem because of all the usual distributed systems difficulties. - Often, there are workloads that are only needed occasionally but also have very different resource needs than the rest of…

> Dynamically scaling horizontally is not something "linux" can do for you

It can, depending on your definition of horizontal, and whether you accept that tools like HAProxy fall under the realm of “Linux administration.”

systemd can spawn N services from a single definition file, which can then register with the load balancer. A small shell script could monitor those service cgroups for basic metrics like memory or CPU utilization, and then command systemd to scale up/down. This is horizontal from the app’s perspective.

I’m not suggesting the above is a great idea, and you’d probably be better off with container orchestration of some kind – but tbf, all that’s doing is encapsulating cgroups and monitoring consumption to command scaling.

Re: Rethinking serverless with FLAME

#45

Post by tonyhb »

davidjfelix wrote: post #39
chrismccord wrote: post #31

Earlier quoted context omitted.

Thanks for the thoughts – hopefully I can make this more clear: > * Pretending that starting a fly machine doesn't cost the same as triggering via s3 seems disingenuous. You're going to be paying for resources wherever you decide to run your code. I don't think this needs to be spelled out. The point about costs is rather than paying to run "my app", I'm paying at multiple layers to run a full solution to my problem.…

> This is exactly my point. You bolt on ever more Serverless offerings to accomplish any actual goal of your application. SNS notifications is exactly the kind of thing I don't want to think about, code around, and pay for. I have Phoenix.PubSub.broadcast and I continue shipping features. It's already running on all my nodes and I pay nothing for it because it's already baked into the price of what I'm running – my a…

You're literally describing what we've built at https://www.inngest.com/. I don't want to talk about us much in this post, but it's so relevant it's hard not to bring it up. (Huge disclaimer here, I'm the co-founder).

In this case, we give you global event streams with a durable workflow engine that any language (currently Typescript, Python, Go, Elixir) can hook into. Each step (or invocation) is backed by a lightweight queue, so queues are cheap and are basically a 1LOC wrapper around your existing code. Steps run as atomic "transactions" which must commit or be retried within a function, and are as close to exactly once as you could get.

Re: Rethinking serverless with FLAME

#46

Post by sgarland »

shortrounddev2 wrote: post #10
sgarland wrote: post #9

Earlier quoted context omitted.

Because they are not concerned with those things, the lack of knowledge in those areas creeps in. If you don’t understand how the OS works at a very fundamental level, you will write sub-optimal code. This is usually fine, because people are smart, compilers are good, and hardware is absurdly fast. However, the fact remains that it’s probably giving up some bits here and there. More germane to my point, though, the a…

The cost you are trying to save is not infrastructure, or compute time, it's labor. It's easier to find a junior javascript developer and then sandbox them inside docker or some serverless service than it is to find someone qualified to lock down a Linux box

Until their code breaks the DB that it’s almost certainly reaching out to.

Everything has trade-offs. You don’t generally get good, fast, and cheap.

Re: Rethinking serverless with FLAME

#47

Post by gitgud »

> It then finds or boots a new copy of our entire application and runs the function there.

So for each “Flame.call” it begins a whole new app process and copies the execution context in?

A very simple solution to scaling, but I’d imagine this would have some disadvantages…

Adding 10ms to the app startup time, adds 10ms to every “Flame.call” part of the application too… same with memory I suppose

I guess these concerns just need to be consider when using this system

Re: Rethinking serverless with FLAME

#49

Post by chrismccord »

tonyhb wrote: post #43

This is great! It reminds me of a (very lightweight) Elixir specific version of what we built at https://www.inngest.com/ . That is, we both make your existing code available to serverless functions by wrapping with something that, essentially, makes the code callable via remote-RPC . Some things to consider, which are called out in the blog post: Often code like this runs in a series of imperative steps. Each of the…

Ingest looks like an awesome service! I talk about job processors/durability/retries in the post. For Elixir specifically for durability, retries, and workflows we reach for Oban, which we'd continue to do here. The Oban job would call into FLAME to handle the elastic execution.

Re: Rethinking serverless with FLAME

#50

Post by [deleted] »

tonyhb wrote: post #43

This is great! It reminds me of a (very lightweight) Elixir specific version of what we built at https://www.inngest.com/ . That is, we both make your existing code available to serverless functions by wrapping with something that, essentially, makes the code callable via remote-RPC . Some things to consider, which are called out in the blog post: Often code like this runs in a series of imperative steps. Each of the…

[deleted]
Post reply on HN