Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fragile narrow laggy asynchronous mismatched pipes kill productivity

thume.ca

Post reply on HN
Print view View original

71–80 of 91 posts

Re: Fragile narrow laggy asynchronous mismatched pipes kill productivity

#71

Post by hinkley »

btreecat wrote: post #38

So I have been thinking about software projects lately, and I have come to the conclusion that a lot of these tools/solutions exist to "build houses" when most of us are just throwing together lean-to sheds and dog houses. Software projects today are naturally more complex and have more complex tooling the same way building a house today requires more knowledge and skill than it did 50 years ago. Then there are some…

You have one group building dog houses like the dog will always be a puppy, and another group looking at those people, laughing, and wandering off to build a kennel, just in case you might decide at some point to start raising show dogs for 4 different breeds at once.

What is most likely instead is that you'll get a second dog in a couple years and need to make a better version of what you already have. And if it's nice enough, make a couple more for friends or as a gift.

Re: Fragile narrow laggy asynchronous mismatched pipes kill productivity

#72

Post by hinkley »

m463 wrote: post #65
btreecat wrote: post #38

So I have been thinking about software projects lately, and I have come to the conclusion that a lot of these tools/solutions exist to "build houses" when most of us are just throwing together lean-to sheds and dog houses. Software projects today are naturally more complex and have more complex tooling the same way building a house today requires more knowledge and skill than it did 50 years ago. Then there are some…

Software engineering evolves to get the most out of your hardware. If you sort of turn that idea around in your head, software will always end up trying to barely work on the shittiest hardware. Think about it. Chips work on 30 nm. What's our next step, to make 30 nm more solid and reliable? No, it's to go to 20 nm or 10 nm and get the same software to work. The same goes with systems stuff. Do we work on having much…

I've heard it said that the reason bits of the Roman aqueducts are still standing is that they were overbuilt by at least a factor of 2.

The craftsman often wants to overbuild something. It's part of what gets them out of bed in the morning, or lets them sleep at night.

What if you - or someone else - got your margin of error wrong, like the Citibank building? A little extra to deal with unexpected situations isn't so bad.

Re: Fragile narrow laggy asynchronous mismatched pipes kill productivity

#73

Post by john-tells-all »

MauranKilom wrote: post #10
dylan604 wrote: post #7

Earlier quoted context omitted.

every. damn. time. i'm a freelancer, so there's not a dev team behind me. the client likes what i made, and wants to start using it. realizing this, i've started spending much more time on the UI/UX (even though i'm not one of those guys) to at least make the tool useable in a dogfooding way.

Maybe it's not 100% relevant to your case, and maybe you've already seen it, but just in case: https://www.joelonsoftware.com/2002/02/13/the-iceberg-secret...

This article is spectacular, thank you!

Joel explains that it's hard explaining Dev work to nondevs, e.g. the Business. They think the UI/screenshot is the software, which has a number of consequences.

As a DevOps, I also see a similar challenge explaining Infra/DevOps to Developers :)

Re: Fragile narrow laggy asynchronous mismatched pipes kill productivity

#74

Post by breeny592 »

colechristensen wrote: post #63
btreecat wrote: post #38

So I have been thinking about software projects lately, and I have come to the conclusion that a lot of these tools/solutions exist to "build houses" when most of us are just throwing together lean-to sheds and dog houses. Software projects today are naturally more complex and have more complex tooling the same way building a house today requires more knowledge and skill than it did 50 years ago. Then there are some…

French cooking's mise en place or first order retrievability by Adam Savage or Lean Engineering or whatever. There's a common thread among them, having what you need, when you need it, where you need it, and understanding how to use it. This is what is lacking. The original Unix philosophy build by greybeards and university linguistics and language professors had this at it's core. The "do one thing well" combined wi…

> Complexity. This is the enemy, the second enemy is bad attempts to reduce complexity which often end up adding more complexity than they take away, just harder to find.

This is true at every level of the systems design process - often by trying to make a system "simpler" i.e. less complex for the end user, the complexity is shifted further down the stack into application code, or even to the underlying infrastructure.

It's easy for those of us with technical backgrounds to see the beauty and simplicity in well designed interfaces, but as the realm of computing and computer interaction shifts away from technical to non-technical people, we start to absorb some of that complexity into our systems design to make up for knowledge shortcomings of end users.

Your example of sed being better than the "fancy data tools" I feel is a good one - whilst sed is incredibly powerful for this use case, if the consumer of what needs to be run there only knows how to use excel, it's often required to create these abstraction layers to allow the end user to do their own primary function/role.

Re: Fragile narrow laggy asynchronous mismatched pipes kill productivity

#75

Post by tlarkworthy »

colechristensen wrote: post #63
btreecat wrote: post #38

So I have been thinking about software projects lately, and I have come to the conclusion that a lot of these tools/solutions exist to "build houses" when most of us are just throwing together lean-to sheds and dog houses. Software projects today are naturally more complex and have more complex tooling the same way building a house today requires more knowledge and skill than it did 50 years ago. Then there are some…

French cooking's mise en place or first order retrievability by Adam Savage or Lean Engineering or whatever. There's a common thread among them, having what you need, when you need it, where you need it, and understanding how to use it. This is what is lacking. The original Unix philosophy build by greybeards and university linguistics and language professors had this at it's core. The "do one thing well" combined wi…

BTW Google Secrets Manager is rediculously easy to use. It's not quite at the level of Vault but it does enough for me and is 100x easier to deploy

Re: Fragile narrow laggy asynchronous mismatched pipes kill productivity

#76

Post by RangerScience »

btreecat wrote: post #38

So I have been thinking about software projects lately, and I have come to the conclusion that a lot of these tools/solutions exist to "build houses" when most of us are just throwing together lean-to sheds and dog houses. Software projects today are naturally more complex and have more complex tooling the same way building a house today requires more knowledge and skill than it did 50 years ago. Then there are some…

"The stone mason lives in a wooden house."

Re: Fragile narrow laggy asynchronous mismatched pipes kill productivity

#77

Post by chubot »

Someone wrote: post #47
chubot wrote: post #28

Earlier quoted context omitted.

I don't agree with that framing... When you say "cache coherency" you're implicitly assuming some authoritative state. The point of git is that there isn't a single authoritative state. Not all apps will work with that model, but more apps than you think would. I guess the difference is how fine-grained you want the updates to be. For example something like Figma in the browser (a collaborative photoshop) implements…

“The point of git is that there isn't a single authoritative state.“ I think you’re missing the point of the post you refer to. For git, there _technically_ isn’t a single authoritave state, but for many, if not most, git use cases, there _sociologically_ is. Projects typically have one repo that is _the_repo_: the repo most merges are done to, that releases get built from, whose url you give when you tell people whe…

The presence of useful forks means that there isn't a single authoritative state. For example, I've had a fork of bash-completion for a over a year, and I use it all the time.

How do I fork a Google doc? At best you can copy and modify, but here's no merge.

Merging is a feature, not a bug (maybe not for all applications, but 99% of them).

Re: Fragile narrow laggy asynchronous mismatched pipes kill productivity

#78

Post by lazyjones »

Networking and distributed software are well understood nowadays, the lower productivity of web companies vs. SpaceX etc. comes from all the unstable (both ever-changing and buggy) software they need to use. Most modern software is affected by this, but the web has it worse because of security issues and because the way browsers are evolving (on purpose, one has to add, because it's a cartel of large players on the web trying to stifle competition). SpaceX doesn't get some innovative new alloy they didn't order every couple of weeks and even the games industry has fewer obstructing external dependencies (hardware vendors and their drivers being one).

At least that's my experience from about 20 years of web development (15 professionally).

Re: Fragile narrow laggy asynchronous mismatched pipes kill productivity

#79

Post by m463 »

hinkley wrote: post #72
m463 wrote: post #65

Earlier quoted context omitted.

Software engineering evolves to get the most out of your hardware. If you sort of turn that idea around in your head, software will always end up trying to barely work on the shittiest hardware. Think about it. Chips work on 30 nm. What's our next step, to make 30 nm more solid and reliable? No, it's to go to 20 nm or 10 nm and get the same software to work. The same goes with systems stuff. Do we work on having much…

I've heard it said that the reason bits of the Roman aqueducts are still standing is that they were overbuilt by at least a factor of 2. The craftsman often wants to overbuild something. It's part of what gets them out of bed in the morning, or lets them sleep at night. What if you - or someone else - got your margin of error wrong, like the Citibank building? A little extra to deal with unexpected situations isn't s…

> gets them out of bed in the morning

I always thought you should do it as best as you could too.

It took a bunch of time in the real world before I could actually grasp the concept of over-engineering in a meaningful way.

The first example of over-engineering I could "get" was when someone (jwz?) ranted about over-engineered high-end audio. The deeper you look, having $1000 00-gauge cables just won't make anything sound better. It's marketing.

The second example was japanese motorcycles. I started realizing that there was an envelope. If you made something bigger and stronger, the weight would go up and the acceleration would get worse. If you made it all motor, acceleration would go up, but handling would get worse. Some of the fastest bikes were actually lighter less powerful bikes that could still accelerate, but also go through turns.

as an example, before the s1000rr era, BMW made big strong durable motorcycles... that weighed a lot and handled in a very mediocre fashion.

(as to safety - yes, bringing safety or security into the equation ... you change the equation)

Re: Fragile narrow laggy asynchronous mismatched pipes kill productivity

#80

Post by dimatura »

unqueued wrote: post #62
dimatura wrote: post #21

Earlier quoted context omitted.

I have been interested in "git for binary data" for a while, mostly for ML/computer vision purposes. I've tried quite a few systems. Of course, there's git-lfs (which keeps "pointer" files and blobs in a cache), which I do use sometimes - but it has a quite few things I don't like. It doesn't give you a lot of control on where the files are stored and how the storage is managed on the remote side. The way it works me…

Actually, a lot has updated about git-annex in the last few years. It actually does support git pointer files like git lfs, which makes it easier to when you want to modify binary files. In fact, it can even use git-lfs servers as one of its back-ends. However, I still prefer symlinks mode, because operations on them are faster because it bypasses the smudge filter. Also, git-annex uses reflink copies whenever possib…

Yeah, git-annex has a lot of cool features that I have yet to see in other systems. I still use it for some things. My main pain point on MacOS was that the symlink mode didn't work well with some apps that didn't understand symlinks. Obviously this is not git-annex's fault, but it still made it so I couldn't use it. I think I could try again at some point and see if I could get it to use reflinks -- maybe it's a version issue.

I also had weird conflicts with the line ending (the whole CR/LF annoyance) on some of the metadata files git-annex used which I couldn't fix, no matter how many .gitconfigs I tweaked. Again, this is not really git-annex's fault, I think.

Post reply on HN