The importance of local development • Hacker News Board

Re: The importance of local development

#51

Post by Washuu » Sun, Oct 06, 2024, 1:15 PM UTC

cletus wrote: post #17

I live by this principle: If it takes you more than 30 seconds to test a change, you're going to have a huge productivity drain. That means writing a test should be easy and running it should be fast (including any compilation steps). As soon as something takes more than 30 seconds, you've lost a lot of people. They've switched tabs. They're on HN or reddit or they've pulled out their phone. You've broken the flow. S…

> That means writing a test should be easy and running it should be fast (including any compilation steps). As soon as something takes more than 30 seconds, you've lost a lot of people. They've switched tabs. They're on HN or reddit or they've pulled out their phone.

ADHD can make this worse since a time delay means getting distracted for much longer.

I have been working on testing a single issue for months. I wrote the fix the first day, but because it can only be tested in the remote development environment. An environment that I have zero insight into how it works and also requires coordinator cross-team members to work together for several hours together to test the fix! ...Because if we need to make a change pushing takes twenty minutes to test and build the new environment.

Re: The importance of local development

#52

Post by viraptor » Sun, Oct 06, 2024, 1:16 PM UTC

andai wrote: post #23

cletus wrote: post #17

I live by this principle: If it takes you more than 30 seconds to test a change, you're going to have a huge productivity drain. That means writing a test should be easy and running it should be fast (including any compilation steps). As soon as something takes more than 30 seconds, you've lost a lot of people. They've switched tabs. They're on HN or reddit or they've pulled out their phone. You've broken the flow. S…

I find it irritating when a program's compile time is perceptible. In compiled languages, TCC (Tiny C Compiler) was the only one that came close, though it doesn't seem to be great for production use. This made me switch back to JS and Python. The frictionlessness of "I made this change and I see the result in the same breath" is very compelling to me. Fortunately there are fast TypeScript compilers now, which make t…

How long it should take to launch VS or the app through VS? I start it once an update, so any time to start is basically amortised to 0. If anything can be sped up later by doing more processing at startup, I'd totally agree to that.

Re: The importance of local development

#53

Post by mananaysiempre » Sun, Oct 06, 2024, 1:19 PM UTC

benreesman wrote: post #11

There are lots of reasons to want a robust local development environment, but a paved path or sometimes even the possibility of "consistency between environments" is not one of them dear sir or madame. The JankStack in which one piles Python environment jank on top of Ubuntu/Darwin jank, and piles Docker jank on top of the previous, and piles Docker Compose jank on the previous, until you finally arrive at Jank-As-A-…

> piles Python environment jank on top of Ubuntu/Darwin jank, and piles Docker jank on top of the previous, and piles Docker Compose jank on the previous, until you finally arrive at Jank-As-A-Service

Don’t demand your language do the job of the operating system and come with a package manager. Your distro has packages that are guaranteed to play well together, use them.

Re: The importance of local development

#54

Post by Dansvidania » Sun, Oct 06, 2024, 1:23 PM UTC

Good local dev-env is vital for productivity, but even more so for dev satisfaction and engagement.

I have been arguing with myself on what a good dev-env is for as long as I have been working in enterprise software dev which by now is about 10 years.

All I know is that attempts to reproduce prod via tilt and similar have not been successful (I have witnessed 3 attempts). The promise of "one dev env for all teams" quickly falls apart.

The main problems with this are IMO: - 1. there is a false sense of encapsulation of complexity: you don't need to know how the components of non-owned services work, until something breaks (and break it will), and then you really do. - 2. #1 + docker + k8s + kafka + graphql etc... make complexity seem very cheap. - 3. add a minimum of deadline pressure on teams, and quickly they will stop caring about keeping the dev-env images up-to-date and/or working.

I would rather have intimate familiarity with what my services depend on, which is much easier to get by running these dependencies somewhat manually, close to natively. You can be sure that your colleagues will come knocking if complexity is not respected.

But this seems hard to package as a product...

Re: The importance of local development

#55

Post by viraptor » Sun, Oct 06, 2024, 1:24 PM UTC

bob1029 wrote: post #19

I'm fighting with a variant of this right now - Attempting to build a multiplayer game with dedicated servers & master server authority. I've got 4 computers involved to run the full dev iteration (2 for 1v1 clients, 1 for master server and 1 for dedicated server host). The clients I am running in my local LAN with both servers in AWS. Attempting to do all of this on the local machine will mostly work, but it fails t…

Have you seen https://github.com/Shopify/toxiproxy ? Maybe you could use this / similar setup to test with more realistic network behaviour.

Re: The importance of local development

#56

Post by benreesman » Sun, Oct 06, 2024, 1:27 PM UTC

the_gipsy wrote: post #44

benreesman wrote: post #11

There are lots of reasons to want a robust local development environment, but a paved path or sometimes even the possibility of "consistency between environments" is not one of them dear sir or madame. The JankStack in which one piles Python environment jank on top of Ubuntu/Darwin jank, and piles Docker jank on top of the previous, and piles Docker Compose jank on the previous, until you finally arrive at Jank-As-A-…

I get the "stacking jank but then it's different in thr cloud", okay, but what's the remedy? Are you talking about Nix, since you mention DLLs?

There are a number of things that I reach for in such situations, and some combination of Nix/Flox/Bazel is often what you opt for if you've got serious requirements around diverse software stacks, particularly in the presence of accelerators like NVIDIA cards.

None of those things is a silver bullet though. They are among the best tools for that class of job, but while Nix and Bazel both have facilities for coping with multi-platform builds with difficult dependencies, you still work hard for it. Nix/NixOS will get you just about perfect builds on a given OS/CPU pair, and Bazel has a lot to offer there too, but when targeting `aarch64-darwin` for local development on Macs and `x86_64-linux` on the deployment target, it's not a free lunch.

The documentation and OSS community support and stuff like that has gotten better (a lot better) in the last year or two for both of those things, but I still can't in good conscience recommend either to a shop that doesn't already have experienced staff (or staff that is passionate about getting that experience): it's a steep learning curve because the problem is fundamentally insane in practice if not in theory.

For folks that want code to run the same in production as in development and can't categorically commit to languages and libraries that religiously eschew diverse native dependencies but still don't want to go through the looking glass with e.g. Nix the best bet in my experience is to provision dev servers adjacent to the production infrastructure and lean on a massively improved remote development experience in e.g. VSCode compared to even a few years ago.

Re: The importance of local development

#57

Post by mtndew4brkfst » Sun, Oct 06, 2024, 1:33 PM UTC

rcxdude wrote: post #9

simonw wrote: post #2

I go back and forth on this. On the one hand, maintaining local development environments that work reliably across larger team if developers is a HUGE cost in terms of time and effort. Developers are amazing at breaking their environments in unexpected ways. It’s common for teams to lose many hours a week to problems related to this. So I love the idea of cloud-based development environments where, if something break…

What about local development precludes having a consistent, repeatable development environment? At worst you can distribute an environment the same way you would on the cloud.

In practice, the number one way to bungle a previously-working local environment at many of my past roles has been "brew upgrade". The number two way was to have MacOS and at least one Linux distribution used for local dev on the same team, but one vastly outnumber the other.

While I am very pro-containers as a deployment target I am also very resistant to containers in the critical feedback loop of local dev, chiefly due to iteration speed and performance/battery overhead, which was much more glaring when the Intel chips were all we had.

I can't endorse Nix in any capacity these days but it did seek to address multi platform environmental consistency, without containerization as your LCD.

Re: The importance of local development

#58

Post by benreesman » Sun, Oct 06, 2024, 1:35 PM UTC

mananaysiempre wrote: post #53

benreesman wrote: post #11

There are lots of reasons to want a robust local development environment, but a paved path or sometimes even the possibility of "consistency between environments" is not one of them dear sir or madame. The JankStack in which one piles Python environment jank on top of Ubuntu/Darwin jank, and piles Docker jank on top of the previous, and piles Docker Compose jank on the previous, until you finally arrive at Jank-As-A-…

> piles Python environment jank on top of Ubuntu/Darwin jank, and piles Docker jank on top of the previous, and piles Docker Compose jank on the previous, until you finally arrive at Jank-As-A-Service Don’t demand your language do the job of the operating system and come with a package manager. Your distro has packages that are guaranteed to play well together, use them.

I use a number of operating systems/distributions/package managers, and they all have their strengths and weaknesses. Most can more or less build most stuff in a way that you can run infrastructure on as long as you're not in a nasty cross-platform setting (MacOS development and Linux deployment comes to mind).

Off the top of my head I can't think of any language other than Python that manages to be a constant and persistent cavalcade of self-inflicted environment and dependency chemical fires on absolutely every single one of them. Pointing this out usually generates a bunch of "pipenv/poetry/uv/whatever works fine for me", and for those folks I am very happy that they don't deal with heavily native extension-backed requirements. Mazel tov.

It's a meme: https://xkcd.com/1987/

Re: The importance of local development

#59

Post by IX-103 » Sun, Oct 06, 2024, 1:36 PM UTC

acdha wrote: post #15

simonw wrote: post #2

I go back and forth on this. On the one hand, maintaining local development environments that work reliably across larger team if developers is a HUGE cost in terms of time and effort. Developers are amazing at breaking their environments in unexpected ways. It’s common for teams to lose many hours a week to problems related to this. So I love the idea of cloud-based development environments where, if something break…

Two thoughts: * in every single case I’ve seen, the developers who broke their local environments that badly had significant skill deficits which affected their general productivity. Investing in training paid dividends far beyond not having to deal with their local environment getting hosed since they also stopped creating massive security and performance problems in production. * building a cloud-based development…

If you've never seen good developers bork their local environments then you clearly haven't worked on a large enough and complex enough project.

I worked on a large project that needed to support development on many platforms. The amount of scripting for the git hooks and build hooks alone required its own development team. When subtly incompatible changes in git hooks rolled out, you would see a few people that ran commands in just the wrong way to bork their repository. These commands are usually fine, but when combined with an otherwise invisible change in the hooks, would just break things. Now the actual breakage rate was low, like 1 in 1000 in a given month, but with as many developers as were working on this project that was always a couple of people that had to nuke their local repo from orbit (it's the only way to be sure).

Re: The importance of local development

#60

Post by komali2 » Sun, Oct 06, 2024, 1:37 PM UTC

If you can't build and run the full environment locally, even in a diminished state, how can you debug the production stack? I'm always advocating for ridiculously fleshed out readmes that describe in detail how to run every part of our stack locally, how to deploy it, and how it's running on the prod servers. If I die I want one of the juniors the next day able to report on how to do anything I do.