Cloud development environments tame complexity by reducing state • Hacker News Board

Re: Cloud development environments tame complexity by reducing state

#31

Post by quickthrower2 » Thu, Nov 03, 2022, 10:53 AM UTC

buscoquadnary wrote: post #7

drunkenmagician wrote: post #5

TL;DR Use cloud based development environments to avoid machine (developer) specific, non reproducible problems, and increase productivity (speculatively) Probably ok for certain needs, but not something I would use. My productivity comes from a localised and personalised setup. Issues with the local or remote environments IMHO, indicate a problems that 1) others may also encounter. 2) should at least, if possible be…

Isn't that what the promise of Docker was, you can distribute the docker image and everyone is running and building the same thing?

Yeah but what is the debugging experience like? Does that work? I think it adds a layer of complication. Also images can take a while to build. Although most would be cached hopefully with just copying your code changing. Maybe a snappy startup can fix?

Re: Cloud development environments tame complexity by reducing state

#32

Post by quickthrower2 » Thu, Nov 03, 2022, 10:56 AM UTC

dmitriid wrote: post #26

No. No. No. No. Solve local development first and then maybe cloud dev. Reasons, just off the top of my head: - always-on internet connection is not always viable - intermittent and bad connections are a thing - which local environment do you support for accessing your cloud? An ever changing combination of "latest only" browsers, apps, and select IDE integrations? - how stable is your cloud environment? which featur…

Plus latency. And cost. And slowness of a browser vs native console.

Re: Cloud development environments tame complexity by reducing state

#33

Post by robertlagrant » Thu, Nov 03, 2022, 11:14 AM UTC

dmitriid wrote: post #26

No. No. No. No. Solve local development first and then maybe cloud dev. Reasons, just off the top of my head: - always-on internet connection is not always viable - intermittent and bad connections are a thing - which local environment do you support for accessing your cloud? An ever changing combination of "latest only" browsers, apps, and select IDE integrations? - how stable is your cloud environment? which featur…

> Solve local development first and then maybe cloud dev

I think this is it. Progressive enhancement beats graceful degradation.

Re: Cloud development environments tame complexity by reducing state

#34

Post by pbalau » Thu, Nov 03, 2022, 1:05 PM UTC

taeric wrote: post #16

nordsieck wrote: post #14

Earlier quoted context omitted.

> Rigidly defined development environments breed fragile systems. They are enticing, for often rapid onboarding of new developers. However, not having the churn of letting new people onboard their setup to the system loses on an anti fragile mechanism. > This is ignoring the removal of control from your developers. Autonomy of tool choice is an amazing boon to job satisfaction. > Which is not to say that a cloud envi…

I think letting people do this is awesome. I think standardizing on it would be a huge mistake. In particular, I think local deployment is important. And having freedom in all local deploys leads to finding the small details that matter. Especially since you probably do not own the full prod tech stack, you are behooved by having proof that you can change it. And, by all means, make the best attempt you can. I am arg…

> I think letting people do this is awesome. I think standardizing on it would be a huge mistake.

There is a hidden issue with not standardizing. Most people either don't care about local vs container (matching prod) development OR are terrible not equipped to make an informed choice. Thus, as a devops, I have to support both methods. This is a huge drain on my resources, especially since devopsing is just part of what I need to do.

The root cause of this is the myth that all software engineers will understand source control, will understand databases, will understand how systems interact with each other and on top of it, will understand the business so they can actually do the work required. This is how you end up with "it works on my machine". This is how you end up with "but it works in repl, therefore your prod environment is broken".

Standardizing doesn't mean this is the only way of doing things. It means, this is the standard way of doing things and if you get stuck, you get support. It means this is the way we consider to be most efficient of doing this stuff. You are still free to do things your way if you chose to, but if you get stuck, it's on you to get unstuck.

Re: Cloud development environments tame complexity by reducing state

#35

Post by taeric » Thu, Nov 03, 2022, 1:05 PM UTC

busterarm wrote: post #25

taeric wrote: post #12

Rigidly defined development environments breed fragile systems. They are enticing, for often rapid onboarding of new developers. However, not having the churn of letting new people onboard their setup to the system loses on an anti fragile mechanism. This is ignoring the removal of control from your developers. Autonomy of tool choice is an amazing boon to job satisfaction. Which is not to say that a cloud environmen…

No offense, but it's not my responsibility to make sure you have job satisfaction. It's my responsibility to keep the lights on and your paychecks coming in. This is a public company and I have compliance requirements to meet or I am personally liable or face jail time. And with thousands/tens-of-thousands of engineers, your tool choice and lack of streamlined repeatable systems is costing us millions of dollars in p…

I'm not sure why you think I am questioning your responsibility?

My observation has been that teams that don't have tool choice almost always lost it because they had an overly specified build environment. Often under the guise of "streamlining" the setup. And this is why they have that particular build server still sitting around that only the head developer is allowed to modify.

In what should be a straw man argument, I've seen this go so far as folks not knowing how to even try to build their software on anything other than the sanctioned IDE with them having to use the mouse to click the "now build" button.

Which is all to say, my argument is intended to be that you do need a repeatable and streamlined build system that is tested and run in a variety of environments. The more diversity in where/how it is run, the more trust I have that the team actually understands how it works. Especially as a whole.

For the personal control, letting the devs accomplish onboarding in a way that they control empowers them to make the changes to the setup that help make that happen, and gives more autonomy to them. Giving them autonomy means they can fix things they see. And you want that, as they will see far far more of what is happening than board level management will.

Does this mean you can't/don't meet compliance? Of course not. Most compliance can be tested. And it should be tested. Again, this testing benefits from diversity of execution, as well, such that a test that can pass on a variety of environments and toolsets is far more trustworthy than one that can only pass on a particularly sanctioned setup.

Now, pulling this back from a straw man argument. I accept there are situations where this level of freedom is harder to achieve. If you take my argument as absolute requiring that the entire team supports every possible variation of tool that could be thrown at them, please walk it back some. That isn't my intent. I just get scared by efforts to unify it all down to one, as well.

Re: Cloud development environments tame complexity by reducing state

#36

Post by KronisLV » Thu, Nov 03, 2022, 2:34 PM UTC

quickthrower2 wrote: post #31

buscoquadnary wrote: post #7

Earlier quoted context omitted.

Isn't that what the promise of Docker was, you can distribute the docker image and everyone is running and building the same thing?

Yeah but what is the debugging experience like? Does that work? I think it adds a layer of complication. Also images can take a while to build. Although most would be cached hopefully with just copying your code changing. Maybe a snappy startup can fix?

> Yeah but what is the debugging experience like? Does that work?

Sort of. In certain stacks, you essentially set up remote debugging like you would for an app running in a remote environment (which is your local container with an exposed port) and your IDE just works.

It's relatively carefree when it works as expected, but a bit of a pain to set up sometimes. Admittedly, something like CPU flame graphs or tools like VisualVM that let you easily select from locally running Java processes to instrument might be harder to work with.

But even then, you can have issues with file system permissions and any bind mounts that you might need (e.g. files in a PHP container, where you want to keep developing and testing your app after page reloads, without rebuilding the entire container).

I wrote a bit more about it here: https://blog.kronis.dev/everything%20is%20broken/containers-...

I'm still a proponent of using containers for making applications more consistently managed (configuration, resource limits, port bindings, storage), self-contained (dependencies, running different/multiple versions in parallel) and easier to launch (e.g. Docker Compose file or a fancier variation of YAML instead of Ansible + systemd services), but they definitely can be a leaky abstraction if you don't have *nix as your development machine OS.

Re: Cloud development environments tame complexity by reducing state

#37

Post by KronisLV » Thu, Nov 03, 2022, 2:41 PM UTC

dmitriid wrote: post #26

No. No. No. No. Solve local development first and then maybe cloud dev. Reasons, just off the top of my head: - always-on internet connection is not always viable - intermittent and bad connections are a thing - which local environment do you support for accessing your cloud? An ever changing combination of "latest only" browsers, apps, and select IDE integrations? - how stable is your cloud environment? which featur…

I've also seen "hybrid" setups in certain stacks, where the applications run locally but the database instance is shared amongst multiple developers for the same app version/branch.

In my experience, that can be a way to deal with some resource requirement and licensing constraints, but can also be pretty horrible due to either bandwidth requirements or latency that adds up for many smaller queries that follow one another (e.g. when your app needs to shuffle around lots of data to load a page).

Therefore, I'm tempted to agree that you should be able to relatively easily run as much as possible locally: especially a database with automated schema migrations and either data import or seeding (generating believable test data).

Then again, one can see why something like that would be problematic in situations where you can have dozens of microservices, running all of which locally isn't entirely viable, yet re-architecting everything isn't either. It'd still be nice to be able to launch and test everything you need for a given scenario locally, even while using certain dependencies from a cloud dev environment.

Re: Cloud development environments tame complexity by reducing state

#38

Post by taeric » Thu, Nov 03, 2022, 3:24 PM UTC

pbalau wrote: post #34

taeric wrote: post #16

Earlier quoted context omitted.

I think letting people do this is awesome. I think standardizing on it would be a huge mistake. In particular, I think local deployment is important. And having freedom in all local deploys leads to finding the small details that matter. Especially since you probably do not own the full prod tech stack, you are behooved by having proof that you can change it. And, by all means, make the best attempt you can. I am arg…

> I think letting people do this is awesome. I think standardizing on it would be a huge mistake. There is a hidden issue with not standardizing. Most people either don't care about local vs container (matching prod) development OR are terrible not equipped to make an informed choice. Thus, as a devops, I have to support both methods. This is a huge drain on my resources, especially since devopsing is just part of wh…

I don't view that as a hidden issue. I view that as the cost.

And agreed that conforming to what others are doing will, by definition, get you more support. I just think there is hidden value in that diversity. It sucks, as it can't be forced/codified.

I will further agree that too many will chase the "long game" of "if we support diverse environments, we will have a more robust product," at the expense of short term health. And often that is a false dichotomy, in that often the best long term choice for a team is also the best short term tradeoff. Surviving is important.

Re: Cloud development environments tame complexity by reducing state

#39

Post by nithayakumar » Thu, Nov 03, 2022, 4:04 PM UTC

taeric wrote: post #12

Rigidly defined development environments breed fragile systems. They are enticing, for often rapid onboarding of new developers. However, not having the churn of letting new people onboard their setup to the system loses on an anti fragile mechanism. This is ignoring the removal of control from your developers. Autonomy of tool choice is an amazing boon to job satisfaction. Which is not to say that a cloud environmen…

Extremely biased here. I'm one of the founders of Nimbus (www.usenimbus.com) and we help companies move to cloud dev environments.

Cloud dev environments are a tool like any other. How they're designed matters. A lot of home grown tooling ends up being limited for ease of build out and maintenance cost/effort.

Re: Cloud development environments tame complexity by reducing state

#40

Post by dmitriid » Thu, Nov 03, 2022, 5:56 PM UTC

KronisLV wrote: post #37

dmitriid wrote: post #26

No. No. No. No. Solve local development first and then maybe cloud dev. Reasons, just off the top of my head: - always-on internet connection is not always viable - intermittent and bad connections are a thing - which local environment do you support for accessing your cloud? An ever changing combination of "latest only" browsers, apps, and select IDE integrations? - how stable is your cloud environment? which featur…

I've also seen "hybrid" setups in certain stacks, where the applications run locally but the database instance is shared amongst multiple developers for the same app version/branch. In my experience, that can be a way to deal with some resource requirement and licensing constraints, but can also be pretty horrible due to either bandwidth requirements or latency that adds up for many smaller queries that follow one an…

In the case when there's a staging environment, it would be nice to easily run the microservice you're working on locally. And/or mock some of the external stuff like BigQueries, Dataflows etc.

You can't even test serverless functions properly because... you need the Amazon/Google server to run them on :)