Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Tower Keeps Rising

lucumr.pocoo.org

Post reply on HN
Print view View original

211–220 of 280 posts

Re: The Tower Keeps Rising

#211

Post by hahahaa »

Mikhail_Edoshin wrote: post #187
_superposition_ wrote: post #164

Earlier quoted context omitted.

I can't prove it but I have strong beliefs that the logic and intuition required for abstracting for future changes is not possible in a stream of predicted tokens. Mental models aren't built from text. Something's missing that can't be measured.

The real understanding builds on not knowing. Once I realise I don't know something I cease thinking and start watching the thing. Then somehow I understand. We do have a mechanism similar to LLMs but it provides existing knowledge; it's a search mechanism, basically. New knowledge happens when we turn that mechanism off.

LLMs can learn but only by copy paste into that tiny context. Actually theoretically they could self fine-tune but it would be very expensive/slow.

Re: The Tower Keeps Rising

#212

Post by permalac »

jambalaya8 wrote: post #42
stavros wrote: post #40

Earlier quoted context omitted.

Please no.

Lemme guess, you were in favour of monolithic systemd too?

All have its space. Small team, do monolith or you'll stop moving. Big teams, be wary of Conway's law, and don't fight it, probably some services will emerge.

Re: The Tower Keeps Rising

#213

Post by permalac »

peter_retief wrote: post #210

It is a great analogy. Also quite like Linus Torvalds likening of AI to a compiler. He argues that calling AI the "author" of code is the equivalent of saying a compiler wrote your code. He views AI as an incredibly powerful productivity layer, much like the historic progression from machine code to assemblers, and then to higher-level compilers!

Shall we be concerned that not many are even able to read assembler code? I'm not talking about reading machine code, produced by the assembler, but the assembler code itself.

Machine code was hard, but one could make it pretty efficient. Not efficiently.

Assembler was still pretty performant, for today's standard it is tip top.

So, moving on 10, 20 years from now, can someone read c++? Even html?

Re: The Tower Keeps Rising

#214

Post by jdw64 »

slopinthebag wrote: post #207
jdw64 wrote: post #76

Since GPT 5.2, AI has been writing code much better than I can. In 5.6, it beat Tourlist in competitive coding. After seeing that, I only practice about an hour a day just to keep the feel of coding. Honestly, rather than pointless debates about whether human coding is bad or AI coding is bad, I just think it's good to build tools that help me understand the world. I don't really care whether it's hand-coded or bad c…

For a lot of people, LLM's are actually doing a better job than what they are capable of. I do worry that they are the people first up for replacement, since they don't really provide much value that the LLM itself can't provide. There isn't much point in paying a human to prompt an LLM if you could do it yourself, the value is in someone with the skill and judgement to both prompt it correctly and validate the outpu…

I see that point differently.

Typing code and using AI well are distinctly different things. I've dealt with Chinese subcontracting managers before. But managing people is different from doing code reviews and writing code. I think these two are fundamentally similar but different. Of course, it's undeniable that you need a certain level of skill to verify the output, which is why I practice typing for an hour every day. But that aside, while it's true that good coders may have a head start in using AI well, I don't think they're essentially the same thing.

The reason I say this is simple: AI has already learned more than I have in many areas. Good code often involves a lot of error handling. But humans can't cover all of those edge cases. Even something like UAF (Use-After-Free) can happen depending on timing, and pointers may not release properly. But AI catches these things surprisingly well.

So I think this way: on average, it may be true that people with higher coding skills use AI better, but I think it's hard to say that means they're better at using AI. People who are good at using AI are usually those who break functions down into more readable pieces and set boundaries better, rather than those who are good at LeetCode-style problems. That's similar to being a good coder, but it's a slightly different skill set.

I respect your opinion, but I think an additional layer will emerge.

Re: The Tower Keeps Rising

#215

Post by [deleted] »

radarsat1 wrote: post #126
ashdksnndck wrote: post #46

Earlier quoted context omitted.

Same issue happens in models trained by organizations who aren’t selling tokens. I believe it’s because being parsimonious is simply harder. Achieving the task at hand independently and declaring the job done is easier than building an abstraction and reconciling between every use case.

Agreed. I've noticed this happens without fail in any project driven primarily by "tickets" and "sprints". Finishing the immediate task ends up taking precedence over future concerns that cover a wider scope. On the other end of the spectrum it's easy to spend too much time at the beginning humming and hawing about the best, most future proof design, because you just know you're not going to be able to easily fix it…

[deleted]

Re: The Tower Keeps Rising

#216

Post by zeafoamrun »

marcus_holmes wrote: post #185
radarsat1 wrote: post #126

Earlier quoted context omitted.

Agreed. I've noticed this happens without fail in any project driven primarily by "tickets" and "sprints". Finishing the immediate task ends up taking precedence over future concerns that cover a wider scope. On the other end of the spectrum it's easy to spend too much time at the beginning humming and hawing about the best, most future proof design, because you just know you're not going to be able to easily fix it…

I have a vaguely-relevant war story. 1998 - Huge business, re-writing some vital piece of the platform in the middle of Y2K. Contract coders are expensive but also the only available people to throw at this. The architect had mapped out the entire system down to class/method level. They'd produced a huge list of classes and methods that needed to be built. So the company hired a bunch of contract coders to build said…

Crazy to think that if it was specced out so well, a modern LLM would make short work of the whole thing. The things we used to get paid for!

Re: The Tower Keeps Rising

#217

Post by sylye »

SoftTalker wrote: post #60
overgard wrote: post #36

Earlier quoted context omitted.

I feel like with software, things have gotten way too complicated (just layer's upon layers upon layers). But to deal with that complexity, now we're using something that just creates WAY more complexity. I've been coding for a while, and I remember the 90s and early 00s where people could make pretty powerful applications with like visual basic or php with essentially no formal training. Those technologies weren't g…

And what's ironic is that a lot of those layers and complexity were added with the stated goal of making it easier for average developers to build applications.

Yes. The computer software trend now is heading to the trend of 'more and more easy', but it cause it more and more complex. Due to the vast money poring in to the software industry for the past 10 years, bosses want fast to market, so easier to produce software. But in the same time it makes it more complex to debug and understand.

We should advocate hard but simple instead, but that will take more time to market, which the current trend of hot money driven market would not want, sigh. There are some good article discussing about this: https://tomauger.gitlab.io/posts/2021-06-08-simple-vs-easy/ https://samwho.dev/blog/simple-complex-easy-hard/

Re: The Tower Keeps Rising

#218

Post by psychoslave »

conartist6 wrote: post #157
conrs wrote: post #152

Earlier quoted context omitted.

Fair, my mistake. I've built a lot with LLM's, my experience sort of but not really tracks that. I've had to course correct a few bad abstractions but the larger the code base becomes the better it seems to be at reusing things. Maybe this is because of types, or spec-first development (with OpenAPI), or black box integration testing - but also maybe not. But generally I have to think about the abstractions and let t…

I built a web-OS, a graphical IDE, and a version control system to replace git, all in about 40,000 lines of highly abstracted Javascript. If you're thinking about how important it is to be able to maintain million-line codebases, I suspect you might have substituted a metric for the actual end goal.

That looks like a nice feat, can you share a repo?

That said, reality at scale always come with details that will break the model, and the main roads when it happens are to ignore/reject any change proposal in the model, go in the mystic quest to reach a model that will fit it all including these new cases with an elegant simple solution, or accommodate special cases on the side until it grows too big or just percolate too fast in the main part to let it be sustainable.

Re: The Tower Keeps Rising

#219

Post by RevEng »

fooqux wrote: post #106
Animats wrote: post #13

Earlier quoted context omitted.

The upper bound on program complexity used to be the power of the human mind. "Vibe coding" can break through that barrier. But not because the problem being solved needs that complexity. Because the process does not drive itself towards compact abstractions. It's the AI-powered version of the scaling problem Brooks described back in "The Mythical Man-Month". The combinatoric problems get worse with scale. Concretely…

> The upper bound on program complexity used to be the power of the human mind. Maybe for simple one-person projects. We've long since developed methods and models to allow us to make things bigger than ourselves. Linux, SAP, etc. These software projects are not held in the mind of a single developer. But we use structure, rules, and other tools so that the pieces still fit together.

They said "the" human kind, not "one" human mind. Even with structures, rules, and other tools, we are limited in the amount we can abstract, comprehend, and communicate. As projects get larger, we inevitably see a friction develop where coordination breaks down and we turn to isolation and responding to failures at the boundaries. This is where complexity in design of an individual system ends. Larger ecosystems aren't designed or coordinated - they emerge organically as each tries to adapt to the other. If only we could handle such complexity, we could design efficient systems that don't suffer from the same waste.

Re: The Tower Keeps Rising

#220

Post by psychoslave »

ColdStream wrote: post #158
Animats wrote: post #13

Earlier quoted context omitted.

The upper bound on program complexity used to be the power of the human mind. "Vibe coding" can break through that barrier. But not because the problem being solved needs that complexity. Because the process does not drive itself towards compact abstractions. It's the AI-powered version of the scaling problem Brooks described back in "The Mythical Man-Month". The combinatoric problems get worse with scale. Concretely…

The question is that if it can go beyond the upper limit of human mind complexity, would we be able to understand it or use it? I cannot remember who it was but there was an author who was traveling with their dog. They noticed that their dog would always pee on various tree to mark them as their territory. On their travels they ended up need some giant Red wood trees and figured, "I want to give my dog the ultimate…

If we would be able to act that consciously, humanity would be flourishing without anyone sent to war or starvation.

It doesn't look like we are that bright, at least collectively. And if they are individual which are really above everyone else on that matter and the rest, like maybe you but definitely not me, then their individual power seems to be unable to move us all away from our collective ill habits.

Post reply on HN