Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nobody knows how the whole system works

surfingcomplexity.blog

Post reply on HN
Print view View original

31–40 of 226 posts

Re: Nobody knows how the whole system works

#31

Post by ahnick »

luckydata wrote: post #22
Animats wrote: post #7

Earlier quoted context omitted.

Including the AI, which generated it once and forgot. This is going to be a big problem. How do people using Claude-like code generation systems do this? What artifacts other than the generated code are left behind for reuse when modifications are needed? Comments in the code? The entire history of the inputs and outputs to the LLM? Is there any record of the design?

Is this an actual problem? Takes minutes for an AI to explore and document a codebase. Sounds like a non problem.

Yes, exactly my point as well. It cuts both ways.

Re: Nobody knows how the whole system works

#32

Post by shevy-java »

ahnick wrote: post #8
virgilp wrote: post #5

That's not how things work in practice. I think the concern is not that "people don't know how everything works" - people never needed to know how to "make their own food" by understanding all the cellular mechanisms and all the intricacies of the chemistry & physics involved in cooking. BUT, when you stop understanding the basics - when you no longer know how to fry an egg because you just get it already prepared fr…

Most people have no idea how to hunt, make a fire, or grow food. If all grocery stores and restaurants run out of food for a long enough time people will starve. This isn't a problem in practice though, because there are so many grocery stores and restaurants and supply chains source from multiple areas that the redundant and decentralized nature makes it not a problem. Thus it is the same with making your own food.…

In Star Trek they just 3D printed everything via light.

Re: Nobody knows how the whole system works

#33

Post by shevy-java »

luckydata wrote: post #22
Animats wrote: post #7

Earlier quoted context omitted.

Including the AI, which generated it once and forgot. This is going to be a big problem. How do people using Claude-like code generation systems do this? What artifacts other than the generated code are left behind for reuse when modifications are needed? Comments in the code? The entire history of the inputs and outputs to the LLM? Is there any record of the design?

Is this an actual problem? Takes minutes for an AI to explore and document a codebase. Sounds like a non problem.

Is that documentation useful? I haven't seen a well-documented codebase by AI so far.

To be fair - humans also fail at that. Just look at the GTK documentation as an example. When you point that out, ebassi may ignore you because criticism is unwanted; and the documentation will never improve, meaning they don't want new developers.

Re: Nobody knows how the whole system works

#36

Post by sciencejerk »

We keep delegating knowledge of the natural, physical world for temporary, rapidly-changing knowledge of abstractions and software tools, which we do not control (now LLM cloud tools).

The lack of comprehensive, practical, multi-disciplinary knowledge creates a DEEP DEPENDENCY on the few multinational companies and countries that UNDERSTAND things and can BUILD things. If you don't understand it, if you can't build it, they OWN you.

Re: Nobody knows how the whole system works

#37

Post by bsza »

fedeb95 wrote: post #35

why does the author imply not knowing everything is a bad thing? If you have clear protocol and interfaces, not knowing everything enables you to make bigger innovations. If everything is a complex mess, then no.

Not knowing everything never "enables" you to do anything. Knowing how something works is always better than not knowing, assuming you want to use it or make changes to it.

Re: Nobody knows how the whole system works

#38

Post by tjchear »

mhog_hn wrote: post #25
tjchear wrote: post #15

I take a fairly optimistic view to the adoption of AI assistants in our line of work. We begin to work and reason at a higher level and let the agents worry about the lower level details. Know where else this happens? Any human organization that existed, exists, and will exist. Hierarchies form because no one person can do everything and hold all the details in their mind, especially as the complexity of what they in…

But what if the AI agent has a 5% chance of adding a bug to that feature? Surely before any feature was completely bug free

Yeah it’s all trade offs. If it means I get to where I want to be faster, even if it’s imperfect, so be it.

Humans aren’t without flaws; prior to coding assistants, I’ve lost count of the times my PM telling me to rush things at the expense of engineering rigor. We validate or falsify the need for a feature sooner and move on to other things. Sometimes it works sometimes a bug blows up in our faces, but things still chug along.

This point will become increasingly moot as AI gets better at generating good code, and faster, too.

Re: Nobody knows how the whole system works

#39

Post by lijok »

virgilp wrote: post #5

That's not how things work in practice. I think the concern is not that "people don't know how everything works" - people never needed to know how to "make their own food" by understanding all the cellular mechanisms and all the intricacies of the chemistry & physics involved in cooking. BUT, when you stop understanding the basics - when you no longer know how to fry an egg because you just get it already prepared fr…

> that's a whole different level of ignorance, that's much more dangerous.

Why? Is it more dangerous to not know how to fry an egg in a teflon pan, or on a stone over a wood fire? Is it acceptable to know the former but not the latter? Do I need to understand materials science so I can understand how to make something nonstick so I’m not dependant on teflon vendors?

Post reply on HN