Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Wheel Reinventor’s Principles (2024)

tobloef.com

Post reply on HN
Print view View original

81–90 of 101 posts

Re: Wheel Reinventor’s Principles (2024)

#81

Post by mikepurvis »

lelanthran wrote: post #79
mikepurvis wrote: post #59

Earlier quoted context omitted.

It's easy if the fields are all numbers and you have a good handle on whether any of them will be negative, in scientific notation, etc. Once strings are in play, it quickly gets very hairy though, with quoting and escaping that's all over the place. Badly formed, damaged, or truncated files are another caution area— are you allowed to bail, or required to? Is it up to your parser to flag when something looks hinky s…

What do you mean "allowed to bail"? Regardless of the format if you're parsing something and encounter an error there are very few circumstances where the correct action is to return mangled dat.

Maybe? If the dataset is large and the stakes are low, maybe you just drop the affected records, or mark them as incomplete somehow. Or generate a failures spool on the side for manual review after the fact. Certainly in a lot of research settings it could be enough to just call out that 3% of your input records had to be excluded due to data validation issues, and then move on with whatever the analysis is.

It's not usually realistic to force your data source into compliance, nor is manually fixing it in between typically a worthwhile pursuit either.

Re: Wheel Reinventor’s Principles (2024)

#82

Post by Symmetry »

strongpigeon wrote: post #65

> [...] Be wary of abstractions made for fabricated use cases. Very well put and I would argue this applies to general software development. This is one of the biggest difference between my freshly-out-of-college self and me right now and something I try to teach engineer I'm trying to grow into "seniors". Too many time have I seen a lot of wasted efforts on trying to build hyper flexible components ("this can do any…

The concept of Don't Repeat Yourself and the concept of You Ain't Gonna Need It are the yin and yang of software development.

Re: Wheel Reinventor’s Principles (2024)

#83

Post by Symmetry »

hinkley wrote: post #74
strongpigeon wrote: post #65

> [...] Be wary of abstractions made for fabricated use cases. Very well put and I would argue this applies to general software development. This is one of the biggest difference between my freshly-out-of-college self and me right now and something I try to teach engineer I'm trying to grow into "seniors". Too many time have I seen a lot of wasted efforts on trying to build hyper flexible components ("this can do any…

I don’t think there’s an accepted set of concrete criteria for making software that can absorb major design changes later without a great deal of effort and stress. How you write code that can accept an abstraction layer at the last responsible moment. Some people have an intuition for it, but it’s sort of an ineffable quality, buried in Best Practices in a way that is not particularly actionable. So people having be…

I've found that refactoring to fix a lack of abstraction is usually easier than refactoring to fix the wrong abstraction.

Re: Wheel Reinventor’s Principles (2024)

#84

Post by pizlonator »

Great post!

If you're a PL/compiler/GC hacker, then here are wheels you should reinvent in order to even just have a basic idea of WTF is going on in the Big Serious Production Wheels that you might end up being gainfully employed to maintain:

- Invent your own language, and write at least an interpreter for it, to get a feel for what makes a language work at all, or not.

- Invent your own compiler IR. Don't worry if you make a bunch of mistakes. Don't worry about whether you follow my advice for how to do it, or anyone else's advice. Make it your own and learn from your mistakes.

- Invent your own way of doing the major compiler optimizations. Of course there are established ways of doing SSA conversion, CSE, constant prop, regalloc, instruction selection, etc. But you won't know why they are that way unless you try to make your own, and then either succeed because you are smarter than everyone else (it's possible that you are), or succeed because you literally reinvented the wheel (then you understand the compiler's wheels better than your friends because you got there from first principles), or you'll fail (most likely outcome) but then you'll understand why the real wheels work the way that they do better than others.

- Reinvent memory management. Write your own GC or whatever.

That's how I learned the craft. Can't think of a better way to learn.

Re: Wheel Reinventor’s Principles (2024)

#85

Post by dkarl »

aleph_minus_one wrote: post #56
JackFr wrote: post #44

Earlier quoted context omitted.

> I/O of specialized formats comes to mind quickly The classic "I'll write my own csv parser - how hard can it be?"

> The classic "I'll write my own csv parser - how hard can it be?" I did as part of my work. It was easy. To be very clear: the CSV files that are used are outputs from another tool, so they are much more "well-behaved" and "well-defined" (e.g. no escaping in particular for newlines; well-known separators; well-known encoding; ...) than many CSV files that you find on the internet. On the other hand, some columns nee…

> very well-behaved CSV files

You were incredibly lucky. I've never heard of anyone who insisted on integrating via CSV files who was also capable of consistently providing valid CSV files.

Re: Wheel Reinventor’s Principles (2024)

#86

Post by krab »

jofer wrote: post #73
JackFr wrote: post #44

Earlier quoted context omitted.

> I/O of specialized formats comes to mind quickly The classic "I'll write my own csv parser - how hard can it be?"

CSV is _way_ hairier than folks think it is!! And for anyone who's not convinced by CSV, consider parsing XML with a regex. "I don't need a full XML parser, I just need this little piece of data! Let's keep things lightweight. This can just be a regex..." I've said it many times myself and been eventually burned by it each time. I'm not saying it's always wrong, but stop and think whether or not you can _really_ trus…

A plural of regex is regrets...

Re: Wheel Reinventor’s Principles (2024)

#87

Post by switchbak »

strongpigeon wrote: post #65

> [...] Be wary of abstractions made for fabricated use cases. Very well put and I would argue this applies to general software development. This is one of the biggest difference between my freshly-out-of-college self and me right now and something I try to teach engineer I'm trying to grow into "seniors". Too many time have I seen a lot of wasted efforts on trying to build hyper flexible components ("this can do any…

I'm not sure, perhaps this is an issue with how our craft is taught, but I think we're missing something when we talk about the (economic) tradeoffs when making these decisions.

Keeping components simple, decoupled and with minimal dependencies allows for a high degree of optionality. When you pair this with a simple system that is easy to reason about - you're doing huge favours to your future self.

On the other hand, hanging off a bunch of unused features, especially ones that have interacting configuration - that's more like adding lead weights to your future self. It weighs you down and adds friction. And we tend to do a terrible job of predicting our future needs.

Kent Beck does a great job discussing the costs of these tradeoffs in his recent book "Tidy First". It builds upon the YAGNI principle, but adds a level of cost analysis that should allow you to sell these ideas to the managerial level.

Re: Wheel Reinventor’s Principles (2024)

#88

Post by hinkley »

Symmetry wrote: post #83
hinkley wrote: post #74

Earlier quoted context omitted.

I don’t think there’s an accepted set of concrete criteria for making software that can absorb major design changes later without a great deal of effort and stress. How you write code that can accept an abstraction layer at the last responsible moment. Some people have an intuition for it, but it’s sort of an ineffable quality, buried in Best Practices in a way that is not particularly actionable. So people having be…

I've found that refactoring to fix a lack of abstraction is usually easier than refactoring to fix the wrong abstraction.

Definitely. Among other things, this is akin to Work Hardening.

Refactoring tries to avoid this but the slope of that line can still end up being positive and you can’t refactor forever unless you’re very careful. And “very careful” is also not yet quantified.

Re: Wheel Reinventor’s Principles (2024)

#90

Post by strongpigeon »

switchbak wrote: post #87
strongpigeon wrote: post #65

> [...] Be wary of abstractions made for fabricated use cases. Very well put and I would argue this applies to general software development. This is one of the biggest difference between my freshly-out-of-college self and me right now and something I try to teach engineer I'm trying to grow into "seniors". Too many time have I seen a lot of wasted efforts on trying to build hyper flexible components ("this can do any…

I'm not sure, perhaps this is an issue with how our craft is taught, but I think we're missing something when we talk about the (economic) tradeoffs when making these decisions. Keeping components simple, decoupled and with minimal dependencies allows for a high degree of optionality. When you pair this with a simple system that is easy to reason about - you're doing huge favours to your future self. On the other han…

I think some of it comes from a sense of admiration or even awe of complex systems. You’ve just been introduced to some of these tools as a college student and you really want to use them as they seem so neat.

But then as you start dealing with over-engineered system, you become intimately aware of the downsides of poorly abstracted system and you start becoming much more careful in your approach.

At least, that’s my pet theory.

Post reply on HN