We have used too many levels of abstractions • Hacker News Board

Re: We have used too many levels of abstractions

#191

Post by stakhanov » Sat, Oct 21, 2023, 12:44 PM UTC

steeleduncan wrote: post #151

There was a point in the history of aviation where anyone who could fly a plane was also capable of constructing and designing one. I wonder if there were similar concerns at that time about a future where someone could be in a cockpit of a plane without truly understanding how the machine works from first principles? Today that type of concern would seem absurd, and we've gotten used to the idea that flying a plane…

The problem is that it takes a really long time until technology is so good and so reliable that you really don't need to understand it to be able to operate it.

Take, for instance, the "Yes, let's all go back to coding in assembly!" line -- The thing is: For a really really long time after high-level languages had become mainstream, you really did still have to know assembly to be a programmer, even if you did most of your work in, say, C, or Pascal. That's because compilers for high-level languages and their debugging tools were initially a "leaky" abstraction. When your programme failed, you had to know assembly to figure out what went wrong and work your way back from that to what you could do in your high-level language to fix the problem. Nowadays, compilers and debugging tools have become so good, that those days are mostly gone, and you really don't need to know assembly any more (for most practical intents and purposes).

But the problem we have today: We pile on layer upon layer upon layer of leaky abstraction without ever giving it the time it needs to mature. We're designing for shortening the amount of time a developer spends on getting something done, under the completely misguided assumption that the developer will never leave the "happy path" where everything works as designed. This is neglecting the fact that a developer spends most of their time debugging the situations that don't work as designed. Usually, if you make the "happy path" time more productive with a side-effect of making the "unhappy path" time less productive, that amounts to a net-negative, and that's the big problem.

Re: We have used too many levels of abstractions

#192

Post by figassis » Sat, Oct 21, 2023, 12:45 PM UTC

mrkeen wrote: post #22

I think a good first step towards improvement in this space is by separating the terms 'abstraction' and 'indirection'. Programmers too often add indirections which don't provide abstraction: * The programmer wants to POST an object to a web server. * The programmer also wants to think about it at the level of POSTing an object to a web server. * And yet the programmer creates an HttpClient.java and an AbstractClient…

This, a manager and domain manager for everything. One of the reasons I love go is that is inherently encourages being direct. All of the verbosity that people complain about it is infinitely offset by simply doing the thing. Some examples:

- how directory structure is part of the code organization of the language, and how it's something you have no choice about (so have to spend zero brain power on), and works out of the box.

- the concept of exported/unexported fields (no need complex logic/annotations, ineffective _prefixes)

- named returns eliminating the need for explicit, so useful when exiting early

- default values (+ named returns)

Any of these *manager concepts just seem out of place in Go.

Re: We have used too many levels of abstractions

#193

Post by pictureofabear » Sat, Oct 21, 2023, 12:45 PM UTC

bongobingo1 wrote: post #93

cjalmeida wrote: post #64

Earlier quoted context omitted.

I've heard that advanced math has a similar problem, that some subjects are quite deep that only a handful of people really understand the topic and passing the knowledge down to younger mathematicians is not a given.

Are there good historical examples of this happening, where foundational knowledge is completely lost? I know I have read about it in books as a fiction, but I assume it must have already happened here before. Maybe its an oxymoronic question to ask, seeing as if its lost we might not even know its lost, but more in the vein of "We put the lime in the mortar because this is what we have always done", unaware of the a…

I think that's essentially what the medieval dark ages were before the renaissance revived historic cultural and intellectual achievements and began building upon them again.

Re: We have used too many levels of abstractions

#194

Post by victorNicollet » Sat, Oct 21, 2023, 12:45 PM UTC

I think it's wonderful that we have created a tech ecosystem (a techosystem?) that allows people to contribute with only a very narrow knowledge of things—or, in less positive terms, that allows companies to hire cheaper labor because fewer skills are required. I believe there are many cases where a mediocre solution is better than no solution at all.

The internet of 1995-2005 was built on mediocre solutions. It was a blast.

Deep-dive experts are not a dying breed, they're not a limited resource with secret knowledge from a time before abstractions. We have more of them now than ever before. They're not defined by having worked all the way up and down the stack, but by having the curiosity to look into layers that are not their own, because not only are we adding more abstraction layers on top, we're also changing some of the layers down the stack: NVMe, WebGPU, WebAssembly, QUIC, AVX512 !

But deep-dive experts are a luxury that most teams don't need, and one of the most important skills of a technical manager is, I would argue, to know when they absolutely must hire one.

Re: We have used too many levels of abstractions

#195

Post by pjmlp » Sat, Oct 21, 2023, 12:46 PM UTC

EvanAnderson wrote: post #45

onetimeuse92304 wrote: post #12

I think there is one interesting angle to this problem. I am someone who grew up with the technology, as the levels of abstractions were being added. I am now benefiting from all those accumulated decades of knowledge. As the IT / development world was changing, I had enormous privilege and comfort to learn the things at the pace they were happening. Being able to assimilate changes over long decades. Be a witness to…

There will be an increasingly important history component in comp sci., software engineering, infosec, and associated fields. I think this is a place where the IT field has done a bad job and, as a result, we've had unbelievable amounts of "wheel reinvention". I'd love to hear from somebody with experience in scientific or other disciplines and IT who could contrast their fields with ours. I only know that I have had…

My software engineering degree had such information, that is how eventually I went from a UNIX/FOSS zealot, to someone discovering the alternative universe of Xerox PARC, ETHZ and having the seed to research the department library about systems programming languages of the past, since Fortran came to be as first alternative to raw Assembly.

Unfortunately not all degrees, or universities are well prepared to offer this, or have staff that cares about making it happen.

Re: We have used too many levels of abstractions

#196

Post by hef19898 » Sat, Oct 21, 2023, 12:46 PM UTC

>> A big percentage of so-called experts today only know how to configure some kind of hype-tool, but they understand nothing about how things work at the deeper level. This is a real challenge and a big problem for the future.

Oh, I couldn't agree more! And it is not just tools ans systems, it is everything from power generation to infrastructure to society and economics.

When start to work like that, the DevSecOps example used in the article but the same happens basically everywhere else too, it spills over into everything else. And yes, this is dangerous in deed.

Re: We have used too many levels of abstractions

#197

Post by josephg » Sat, Oct 21, 2023, 12:47 PM UTC

BiteCode_dev wrote: post #109

onetimeuse92304 wrote: post #12

I think there is one interesting angle to this problem. I am someone who grew up with the technology, as the levels of abstractions were being added. I am now benefiting from all those accumulated decades of knowledge. As the IT / development world was changing, I had enormous privilege and comfort to learn the things at the pace they were happening. Being able to assimilate changes over long decades. Be a witness to…

You can actually see this with new frontend devs. They know only full SPA frameworks, they have never seen a dump of an HTTP message, headers and verbs are abstract things to them. Hell, many of them don't know you can have fully functional websites with zero JS, including payment, video, login, etc. I started to write an HTMX tutorial ( https://www.bitecode.dev/p/a-little-taste-of-htmx-part-1 ) because I noticed a l…

> they have never seen a dump of an HTTP message, headers and verbs are abstract things to them.

When I was teaching programming, I had a fun party trick whenever we got to HTTP. I'd fire up netcat (in listen mode), then connect from a web browser and "serve" a website by hand. I'd show the students the HTTP request that came in, and just manually type out a simple HTTP response and they'd see it appear live in the browser. Its pretty magical.

And once I'd shown them that, I'd write (by hand) an HTTP request to wikipedia or something to show them how its symmetrical.

Of course, real websites increasingly do HTTP2 over TLS or something. So unfortunately its not as "real" as it once was. But if wide eyed look on my students faces is anything to go by, it was a great lesson.

Re: We have used too many levels of abstractions

#198

Post by notarobot123 » Sat, Oct 21, 2023, 12:49 PM UTC

BiteCode_dev wrote: post #124

ksec wrote: post #120

Earlier quoted context omitted.

>You can actually see this with new frontend devs. They know only full SPA frameworks, ...... Around I guess 2017 or even 2016? I used to think this is some sort internet troll comment about people never seen a dump of an HTTP message, until...... >Because they started with assembly. It wasn't necessary starting with assembly or something low level. ( Although that certainly helped ) We have less entertainment, more…

Having to figure it out is not specific to a generation. My generation had magazines and forums. Then came out google and SO. Now chat gpt. The difference is where you start figuring things out. Before, you needed to figure things out at your level, because it was the immediate area of mystery. Now this level is generally solved, you need to figure things out at a different level: - filter out the mass of irrelevant…

It's fair to say that these are different kinds of "figuring out" though. Learning by trial and error is distinct from learning how to find and synthesize information and each lead to very different outcomes over the long-term.

I'd argue that the "find and synthesize" generation have an advantage within contemporary software paradigms because of their experience but, without deep knowledge of the foundations they are building on, they might be disadvantaged when it comes to imagining/creating new paradigms.

Then again, first-order thinking seems to be easier when you're not marred with the traditions and conventions of the past so maybe this isn't actually a disadvantage.

Re: We have used too many levels of abstractions

#199

Post by pjmlp » Sat, Oct 21, 2023, 12:52 PM UTC

criddell wrote: post #128

EvanAnderson wrote: post #45

Earlier quoted context omitted.

There will be an increasingly important history component in comp sci., software engineering, infosec, and associated fields. I think this is a place where the IT field has done a bad job and, as a result, we've had unbelievable amounts of "wheel reinvention". I'd love to hear from somebody with experience in scientific or other disciplines and IT who could contrast their fields with ours. I only know that I have had…

I couldn’t agree more and it’s something I didn’t understand at all until my final year at university. I was talking with one of my professors and complained about having to learn and use SML and Eiffel for some projects when in the real world , I’d be using something like C (this was 1994). He asked me if I was there for an education or training. I think today it’s probably more true than ever. Schools are under pre…

In some countries, e.g. Portugal, this is sorted how by having two university systems, and technical schools.

One wants to go straight into the work market, with focus on getting the bases, technical school.

One wants to go into the work market, with some more preparation than only the bases, polytechnic institut.

One wants to go into the work market, with overall experience, in a degree certified by the engineering order as qualified for professional title certification exam, university.

Re: We have used too many levels of abstractions

#200

Post by figassis » Sat, Oct 21, 2023, 12:52 PM UTC

warkanlock wrote: post #17

This is an insightful article, although I don't necessarily agree with the view that "everyone" needs to know everything from first principles to be good at their job. Talking about abstractions, during my past month, I was reading nand2tetris, and it's a compelling experience if you understand the exercise you are doing, which is not about building a computer from first principles; it's much more than that. It makes…

I think everyone needs to know everything they do professionally from first principles. And if you are to build abstractions, they also need to be based on first principles in order to be easy to reason about.

Sometimes I'm learning a framework, and I'm in the part of "...you write this easy syntax and it outputs pure HTML!", which is great. But then the next line you read "also, when you use [obscure symbol], it does [obscure thing] in order to respect [obscure concept]". You have no idea what to do other than start googling those 3 new things, none of which are explained properly or at all in the current "super simple framework" documentation. I think even naming should make sense.