Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nobody knows how the whole system works

surfingcomplexity.blog

Post reply on HN
Print view View original

121–130 of 226 posts

Re: Nobody knows how the whole system works

#121

Post by markbao »

There’s a difference between abstracting away the network layer and not understanding the business logic. What we are talking about with AI slop is not understanding the business logic. That gets really close to just throwing stuff at the wall and seeing what works instead of a systematic, reliable way to develop things that have predictable results.

It’s like if you are building a production line. You need to use a certain type of steel because it has certain heat properties. You don’t need to know exactly how they make that type of steel. But you need to know to use that steel. AI slop is basically just using whatever steel.

At every layer of abstraction in complexity, the experts at that layer need to have a deep understanding of their layer of complexity. The whole point is that you can rely on certain contracts made by lower layers to build yours.

So no, just slopping your way through the application layer isn’t just on theme with “we have never known how the whole system works”. It’s ignoring that you still have a responsibility to understand the current layer where you’re at, which is the business logic layer. If you don’t understand that, you can’t build reliable software because you aren’t using the system we have in place to predictably and deterministically specify outputs. Which is code.

Re: Nobody knows how the whole system works

#122

Post by raw_anon_1111 »

Spivak wrote: post #117
raw_anon_1111 wrote: post #113

Earlier quoted context omitted.

LLMs are good at turning well defined requirements to code. But even now it’s struggling on a project to understand the correlation between “It is creating Lambda code to do $x meaning it needs to change the corresponding IAM role in CloudFormation to give it permission it needs”

The LLMs are fantastic at writing terraform when you tell it what to do which is a huge timesaver, but good heavens is it terrible at actually knowing what pieces need to be wired up for anything but the simplest cases. Job security for now I guess?

I was able to one shot CDK, Terraform and CloudFormation on my last three projects respectively (different clients - different IAC). But I was really detailed about everything I needed and I fed ChatGPT the diagram.

I guess I could be more detailed in the prompt/md files about every time it changes lambda code, check the permissions in the corresponding IAC and check to see if a new VPC endpoint is needed.

Re: Nobody knows how the whole system works

#123

Post by sceptic123 »

g947o wrote: post #61
sceptic123 wrote: post #57

Earlier quoted context omitted.

I read it more as: We already don't know how everything works, AI is steering us towards a destination where there is more of the everything. I would also add it's also possible it will reduce the number people that are _capable_ of understanding the parts it is responsible for.

Who's "we"? I am sure engineers collectively understand how the entire stack works. With LLM generated output, nobody understands how anything works, including the very model you just interacted with -- evident in "you are absolutely correct"

Even as a collective whole, engineers will likely only understand the parts of system that are engineering problems and solutions. Even if they could understand it all, there is still no _one_ person who understands how everything works.

Re: Nobody knows how the whole system works

#124

Post by stetrain »

idiotsecant wrote: post #63
ahnick wrote: post #8

Earlier quoted context omitted.

Most people have no idea how to hunt, make a fire, or grow food. If all grocery stores and restaurants run out of food for a long enough time people will starve. This isn't a problem in practice though, because there are so many grocery stores and restaurants and supply chains source from multiple areas that the redundant and decentralized nature makes it not a problem. Thus it is the same with making your own food.…

Ok, poof. Now everyone knows how to hunt, farm, and cook. What problem does this solve? In the event of breakdown of society there is nowhere near enough game or arable land near, for example, New York City to prevent mass starvation if the supply chain breaks down totally. This is a common prepper trope, but it doesn't make any sense. The actual valuable skill is trade connections and community. A group of people yo…

I don't think that comment is advocating for most people to be able to do these things or stating that this is a problem.

In fact it says "This isn't a problem in practice though"

Re: Nobody knows how the whole system works

#125

Post by erelong »

Reminds me of a short writing "I, Pencil"

The problem is education, and maybe ironically AI can assist in improving that

I've read a lot about programming and it all feels pretty disorganized; the post about programmers being ignorant about how compilers work doesn't sound surprising (go to a bunch of educational programming resources and see if they cover any of that)

It sounds like we need more comprehensive and detailed lists

For example, with objections to "vibe coding", couldn't we just make a list of people's concerns and then work at improving AI's outputs which would reflect the concerns people raise? (Things like security, designs to minimize tech debt, outputting for rradability if someone does need to manually review the code in the future, etc.?)

Incidentally this also reminds me of political or religious stances against technology, like the Amish take for example, as the kind of ignorance of and dependence on processes out of our control discussed seem to be inherent qualities of technological systems as they grow and become more complex.

Re: Nobody knows how the whole system works

#126

Post by Jtsummers »

toyg wrote: post #115
scottLobster wrote: post #100

Earlier quoted context omitted.

The irony is "ownership" is a common management talking point, but when you actually try to take ownership you inevitably run into walls of access, a lack of information, and generally a "why are you here?" mentality. Granted one person can't know/do everything, but large companies in particular seem allergic to granting you any visibility whatsoever. It's particularly annoying when you're given a deadline, bust your…

Ownership doesn't imply FULL ownership. You get handed ownership of a slice, and expected to be responsible for that bit of land; but you'll never own the farm, and will likely never be consulted on whether that land should become a car park from Tuesday. That's just how capitalism work.

> and will likely never be consulted on whether that land should become a car park from Tuesday.

Then you don't have ownership. What you have is responsibility without ownership or authority if this rug pull can be performed.

Re: Nobody knows how the whole system works

#127

Post by devin »

ffsm8 wrote: post #94
rorylaitila wrote: post #71

There are many layers to this. But there is one style of programming that concerns me. Where you neither understand the layer above you (why the product exists and what the goal of the system is) nor the layer below (how to actually implement the behavior). In the past, many developers barely understood the business case, but at least they understood how to translate into code, and could put backpressure on the busin…

I am still missing something like Claude code that's less "hands-off" and optimizes for small edits instead of full feature development Like you're sitting in your ide, select a few rows, press (for example) caps lock to activate speech and then just say a short line what it should adjust or similar - which is then staged for the next adjustments to be done with the same UX Like saying "okay, I need a new usecase her…

The level of exposition required for a lot of edits you might want to make is what stops this from being a primary method of interaction. If I have to express >= AND <= AND NOT == OR ... then I may as well write the thing myself.

Re: Nobody knows how the whole system works

#128

Post by sceptic123 »

pixl97 wrote: post #68
sceptic123 wrote: post #56

Earlier quoted context omitted.

I don't think they're saying _you_ can't do those things, just that most people can't which I have to agree with. They're not saying people can't learn those things either, but that's the practice you're talking about here. The real question is, can you learn to do it before you starve or freeze to death? Or perhaps poison yourself because you ate something you shouldn't or cooked it badly.

Can you list a situation where this matters that you know this personally? Maybe if you end up alone and lost in a huge forest or the Outback, but this is a highly unlikely scenario. If society falls apart cooking isn’t something you need to be that worried about unless you survive the first few weeks. Getting people to work together with different skills is going to be far more beneficial.

The existential crisis part for me is that no-one (or not enough people) have the skills or knowledge required to do these things. Getting people to work together only works if some people have those skills to begin with.

I also wasn't putting the focus is on cooking, the ability to hunt/gather/grow enough food and keep yourself warm are far more important.

And you are far more optimistic about people than me if you think people working together is the likely scenario here.

Re: Nobody knows how the whole system works

#129

Post by MobiusHorizons »

There will always be many gaps in peoples knowledge. You start with what you need to understand, and typically dive deeper only when it is necessary. Where it starts to be a problem in my mind is when people have no curiosity about what’s going on underneath, or even worse, start to get superstitious about avoiding holes in the abstraction without the willingness to dig a little and find out why.

Re: Nobody knows how the whole system works

#130

Post by ltbarcly3 »

I mean the quotes in this article aren't even disagreeing except on vague value judgements with no practical consequences.

Yes you can make better and more perfect solutions with a deep understanding of every consequence of every design decision. Also you can make some real world situation thousands of times better without a deep understanding of things. These two statements don't disagree at all.

The MIPS image rendering example is perfect here. Notice he didn't say "there was some obscure attempt to load images on MIPS and nobody used it because it was so slow so they used the readily available fast one instead". There was some apparently widely used routine to load images that was popular enough it got the attention of one of the few people who deeply understands how the system worked, and they fixed it up.

PHP is an awful trash language and like half the internet was built on it and lots of people had a lot of fun and got a lot more work done because people wrote a lot of websites in PHP. Sure, PHP is still trash, but it's better to have trash than wait around for someone to 'do it right', and maybe nobody ever gets around to it.

Worse is better. https://en.wikipedia.org/wiki/Worse_is_better

Post reply on HN