Live data from Hacker News

Nobody knows how the whole system works

surfingcomplexity.blog

11–20 of 226 posts

Re: Nobody knows how the whole system works

#11
The claimed connections here fall apart for me pretty quickly.

CPU instructions, caches, memory access, etc. are debated, tested, hardened, and documented to a degree that's orders of magnitude greater than the LLM-generated code we're deploying these days. Those fundamental computing abstractions aren't nearly as leaky or nearly as in need of refactoring tomorrow.

Re: Nobody knows how the whole system works

#12
post #6

But people are expected to understand the part of the system they are responsible for at the level of abstraction they are being paid to operate. This new arrangement would be perfectly fine if they aren't responsible when/if it breaks.

I don't think there is anything new here and the metaphor holds up perfectly. There have always been bugs we don't understand in compilers or libraries or implementations beyond that, that make the path we chose unavailable to us at a certain level. The responsibility is to create a working solution, sure, but there is nothing that would prevent us from getting there by typing "Hey LLM, this is not working, let's try a different approach", even though it might not feel great.

Re: Nobody knows how the whole system works

#13
Perhaps a dose of pragmatism is needed here?

I am no CS major, nor do I fully understand the inner workings of a computer beyond "we tricked a rock into thinking by shocking it."

I'd love to better understand it, and I hope that through my journey of working with computers, i'll better learn about these underlying concepts registers, bus's, memory, assembly etc

Practically however, I write scripts that solve real world problems, be that from automating the coffee machine, to managing infrastructure at scale.

I'm not waiting to pick up a book on x86 assembly first before I write some python however. (I wish it were that easy.)

To the greybeards that do have a grasp of these concepts though? It's your responsibility to share that wealth of knowledge. It's a bitter ask, I know.

I'll hold up my end of the bargain by doing the same when I get to your position and everywhere in between.

Re: Nobody knows how the whole system works

#14

Perhaps a dose of pragmatism is needed here? I am no CS major, nor do I fully understand the inner workings of a computer beyond "we tricked a rock into thinking by shocking it." I'd love to better understand it, and I hope that through my journey of working with computers, i'll better learn about these underlying concepts registers, bus's, memory, assembly etc Practically however, I write scripts that solve real wor…

[dead]

Re: Nobody knows how the whole system works

#15
I take a fairly optimistic view to the adoption of AI assistants in our line of work. We begin to work and reason at a higher level and let the agents worry about the lower level details. Know where else this happens? Any human organization that existed, exists, and will exist. Hierarchies form because no one person can do everything and hold all the details in their mind, especially as the complexity of what they intend to accomplish goes up.

One can continue to perfect and exercise their craft the old school way, and that’s totally fine, but don’t count on that to put food on the table. Some genius probably can, but I certainly am not one.

Re: Nobody knows how the whole system works

#16

  Adam Jacob
  It’s not slop. It’s not forgetting first principles. It’s a shift in how the craft work, and it’s already happened. 
This post just doubled down without presenting any kind of argument.

  Bruce Perens
  Do not underestimate the degree to which mostly-competent programmers are unaware of what goes on inside the compiler and the hardware.
Now take the median dev, compress his lack of knowledge into a lossy model, and rent that out as everyone's new source of truth.

Re: Nobody knows how the whole system works

#17
To be fair, I don't know how a living human individual work, let alone how they actually work in society. I suspect I'm not alone in this case.

So nothing new under the sun, often the practices come first, then only can some theory emerge, from which point it can be leverage on to go further than present practice and so on. Sometime practice and theory are more entengled in how they are created on the go, obviously.

Re: Nobody knows how the whole system works

#18
post #7
post #3

Strange article. The problem isn’t that everyone doesn’t know how everything works, it’s that AI coding could mean there is no one who knows how a system works.

Including the AI, which generated it once and forgot. This is going to be a big problem. How do people using Claude-like code generation systems do this? What artifacts other than the generated code are left behind for reuse when modifications are needed? Comments in the code? The entire history of the inputs and outputs to the LLM? Is there any record of the design?

I for one I save all conversations in the codebase. Includes both human prompts and outputs. But I’m using a modified codex to do so. Not sure why it’s not default as it’s useful to have this info.

Re: Nobody knows how the whole system works

#19
post #7
post #3

Strange article. The problem isn’t that everyone doesn’t know how everything works, it’s that AI coding could mean there is no one who knows how a system works.

Including the AI, which generated it once and forgot. This is going to be a big problem. How do people using Claude-like code generation systems do this? What artifacts other than the generated code are left behind for reuse when modifications are needed? Comments in the code? The entire history of the inputs and outputs to the LLM? Is there any record of the design?

I have experimented with telling Claude Code to keep a historical record of the work it is performing. It did work (though I didn't assess the accuracy of the record) but I decided it was a waste of tokens and now direct it to analyze the history in ~/.claude when necessary. The real problem I was solving was making sure it didn't leave work unfinished between autocompacts (eg crucial parts of the work weren't performed and instead there are only TODO comments). But I ended up solving that with better instructions about how to break down the plan into bite-sized units that are more friendly to the todo list tool.

I have prompting in AGENTS.md that instructs the agent to update the relevant parts of the project documentation for a given change. The project has a spec, and as features get added or reworked the spec gets updated. If you commit after each session then the git history of the spec captures how the design evolves. I do read the spec, and the errors I've seen so far are pretty minor.

Re: Nobody knows how the whole system works

#20
post #5

That's not how things work in practice. I think the concern is not that "people don't know how everything works" - people never needed to know how to "make their own food" by understanding all the cellular mechanisms and all the intricacies of the chemistry & physics involved in cooking. BUT, when you stop understanding the basics - when you no longer know how to fry an egg because you just get it already prepared fr…

At what point is the threshold between fine and concerning? Seems like the one you put is from your point of view. I’m sure not everyone would agree and is subjective.
Post reply on HN