Rethinking serverless with FLAME

A pattern I see over and over, which has graduated to somewhere between a theorem and a law, is that motivated developers can make just about any process or architecture work for about 18 months.

By the time things get bad, it's almost time to find a new job, especially if the process was something you introduced a year or more into your tenure and are now regretting. I've seen it with a handful of bad bosses, at least half a dozen times with (shitty) 'unit testing', scrum, you name it.

But what I don't know is how many people are mentally aware of the sources of discomfort they feel at work, instead of a more nebulous "it's time to move on". I certainly get a lot of pushback trying to name uncomfortable things (and have a lot less bad feelings about it now that I've read Good to Great). Nobody wants to say, "Oh look, the consequences of my actions."

The people materially responsible for the Rube Goldberg machine I help maintain were among the first to leave. The captain of that ship asked a coworker of mine if he thought it would be a good idea to open source our engine. He responded that nobody would want to use our system when the wheels it reinvented already exist (and are better). That guy was gone within three to four months, under his own steam.

No, because then you're dividing the resources allocated to the function among the existing run + the new run. If you over-allocate ahead of time to accommodate for this, you might as well just run ordinary VMs, which always have excess allocation locally; the core idea of scaling granularly is that you only allocate the resources you need for that single execution (paying a premium compared to a VM but less overall for spiky workloads since less overhead will be wasted).

azure functions don't fit common definition of serverless, I've had a few convos with them over several years.. but there is really a mismatch owing to the original origin at azure, and real lack of understanding of the space, ie origin is as built on top of web apps.. ie.. azure functions is built on a hack for to try and enter the marketing in the serverless space at its origins. how many websites do you need to run... ie you can't run more than 50 functions, or the 16 cell table on different runtime options (ie. provision servers for your server less)... consumption is better, but the origins in web apps means its just a different product.. hey every function has a url by default :shrug: azure needs a radical rethink of what serverless is, I haven't seen any evidence they got the memo. in aws, lambda originated out of s3, re bring compute to storage.

One thing I'm not following how this would work with IAM etc. The power of Lambda to me is that it's also easy to deal with authorization to a whole bunch of AWS services. If I fire off a flame to a worker in a pool and it depends on say accessing DynamoDB, how do I make sure that that unit of work has the right IAM role to do what it needs to do?

Similarly how does authorization/authentication/encryption work between the host and the forked of work? How is this all secured with minimal permissions?

> Also thanks to Fly infrastructure, we can guarantee the FLAME runners are started in the same region as the parent.

If customers think this is a feature and not a bug, then I have a very different understanding about what serverless/FaaS is meant to be used for. My division is pretty much only looking at edge networking scenarios. Can I redirect you to a CDN asset in Boston instead of going clear across the country to us-west-1? We would definitely NOT run Lamba out of us-west-1 for this work.

There are a number of common ways that people who don't understand concurrency think they can 'easily' or 'efficiently' solve a problem that provably do not work, and sometimes tragicomically so. This feels very similar and I worry that fly is Enabling people here.

Particularly in Elixir, where splitting off services is already partially handled for you.