what a well written article. That's actually a problem. Time will come and hit the same way it has done to aqueduct, like lost technology that no one knows how they have worked in details. Maybe it is just how engineering evolution works?
Nobody knows how the whole system works
51–60 of 226 posts
Re: Nobody knows how the whole system works
#52Get enough people in the room and they can describe "the system". Everything OP lists (QAM, QPSK, WPA whatever) can be read about and learned. Literally no one understands generative models, and there isn't a way for us to learn about their workings. These things are entirely new beasts.
Re: Nobody knows how the whole system works
#53It is not about having infinite width and depth of knowledge. Is about abstracting at the right level for the components are relevant enough and can assume correctness outside the focus of what you are solving.
Systems include people, that make their own decisions that affect how they work and we don’t go down to biology and chemistry to understand how they make choices. But that doesn’t mean that people decisions should be fully ignored in our analysis, just that there is a right abstraction level for that.
And sometimes a side or abstracted component deserves to be seen or understood with more detail because some of the sub components or its fine behavior makes a difference for what we are solving. Can we do that?
Re: Nobody knows how the whole system works
#54That's not how things work in practice. I think the concern is not that "people don't know how everything works" - people never needed to know how to "make their own food" by understanding all the cellular mechanisms and all the intricacies of the chemistry & physics involved in cooking. BUT, when you stop understanding the basics - when you no longer know how to fry an egg because you just get it already prepared fr…
> that's a whole different level of ignorance, that's much more dangerous. Why? Is it more dangerous to not know how to fry an egg in a teflon pan, or on a stone over a wood fire? Is it acceptable to know the former but not the latter? Do I need to understand materials science so I can understand how to make something nonstick so I’m not dependant on teflon vendors?
It's relative, not absolute. It's definitely more dangerous to not know how to make your own food than to know something about it - you _need_ food, so lacking that skill is more dangerous than having it.
That was my point, really - that you probably don't need to know "materials science" to declare yourself competent enough in cooking so that you can make your own food. Even if you only cooked eggs in teflon pans, you will likely be able to improvise if need arises. But once you become so ignorant that you don't even know what food is unless you see it on a plate in a restaurant, already prepared - then you're in a lot poorer position to survive, should your access to restaurants be suddenly restricted. But perhaps more importantly - you lose the ability to evaluate food by anything other than aspect & taste, and have to completely rely on others to understand what food might be good or bad for you(*).
(*) even now, you can't really "do your own research", that's not how the world works. We stand on shoulders of giants - the reason we have so much is because we trust/take for granted a lot of knowledge that ancestors built up for us. But it's one thing to know /prove everything in detail up until the basic axioms/atoms/etc; nobody does that. And it's a completely different different thing to have your "thoughts" and "conclusions" already delivered to you in final form by something (be it Fox News, ChatGPT, New York Times or anything really) and just take them for granted, without having a framework that allows to do some minimal "understanding" and "critical thinking" of your own.
Re: Nobody knows how the whole system works
#55That's not how things work in practice. I think the concern is not that "people don't know how everything works" - people never needed to know how to "make their own food" by understanding all the cellular mechanisms and all the intricacies of the chemistry & physics involved in cooking. BUT, when you stop understanding the basics - when you no longer know how to fry an egg because you just get it already prepared fr…
Most people have no idea how to hunt, make a fire, or grow food. If all grocery stores and restaurants run out of food for a long enough time people will starve. This isn't a problem in practice though, because there are so many grocery stores and restaurants and supply chains source from multiple areas that the redundant and decentralized nature makes it not a problem. Thus it is the same with making your own food.…
> Most people have no idea how to hunt, make a fire, or grow food
That's a bizarre claim, confidently stated.
Of course I can make a fire, cook and my own food. You can, too. When it comes to hunting, skinning and the cutting of animals, that takes a bit more practice but anyone can manage something even if the result isn't pretty.
If stores ran out of food we would have devastating problems but because of specialization, just because we live in cities now you simply can't go out hunting even if you wanted to. Plus there is probably much more pressing problems to take care of, such as the lack of water and fuel.
If most people actually couldn't cook their own food, should they need, that would be a huge problem. Which makes the comparison with IT apt.
Re: Nobody knows how the whole system works
#56Earlier quoted context omitted.
Most people have no idea how to hunt, make a fire, or grow food. If all grocery stores and restaurants run out of food for a long enough time people will starve. This isn't a problem in practice though, because there are so many grocery stores and restaurants and supply chains source from multiple areas that the redundant and decentralized nature makes it not a problem. Thus it is the same with making your own food.…
> Most people have no idea how to hunt, make a fire, or grow food That's a bizarre claim, confidently stated. Of course I can make a fire, cook and my own food. You can, too. When it comes to hunting, skinning and the cutting of animals, that takes a bit more practice but anyone can manage something even if the result isn't pretty. If stores ran out of food we would have devastating problems but because of specializa…
I don't think they're saying _you_ can't do those things, just that most people can't which I have to agree with.
They're not saying people can't learn those things either, but that's the practice you're talking about here. The real question is, can you learn to do it before you starve or freeze to death? Or perhaps poison yourself because you ate something you shouldn't or cooked it badly.
Re: Nobody knows how the whole system works
#57Strange article. The problem isn’t that everyone doesn’t know how everything works, it’s that AI coding could mean there is no one who knows how a system works.
I read it more as:
We already don't know how everything works, AI is steering us towards a destination where there is more of the everything.
I would also add it's also possible it will reduce the number people that are _capable_ of understanding the parts it is responsible for.
Re: Nobody knows how the whole system works
#58Strange article. The problem isn’t that everyone doesn’t know how everything works, it’s that AI coding could mean there is no one who knows how a system works.
No I think the problem is AI coding removes intentionality . And that introduces artifacts and connections and dependencies that shouldn’t be there if one had designed the system with intent. And that makes it eventually harder to reason about. There is a difference in qualia in it happens to work and it was made for a purpose . Business logic will strive more for it happens to work as a good enough .
The core problem is irresponsibility. Things that happen to work may stop working, or be revealed to have terrible flaws. Who is responsible? What is their duty of care?
Re: Nobody knows how the whole system works
#59Strange article. The problem isn’t that everyone doesn’t know how everything works, it’s that AI coding could mean there is no one who knows how a system works.
It's that no one knows if a system works.
Re: Nobody knows how the whole system works
#60This also applies to other things. No one person knows how to make a pencil.
Three minute video by Milton Friedman: https://youtu.be/67tHtpac5ws?si=nFOLok7o87b8UXxY