Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What if useful AI is a fantasy?

lzon.ca

Post reply on HN
Print view View original

51–60 of 66 posts

Re: What if useful AI is a fantasy?

#51

Post by teravor »

to maintain a mental model of LLM generated code I resort to exclusively DOD (data oriented design) principles. I spend most of my time engineering structs and describing transitions in English then have the LLM execute on the transitions and whatever scaffolding (eg. async runtimes, external libraries) are required.

it usually helps if you know how something is to be done, you just want to save time not doing it yourself. this way you can give the LLM intermediate structs and transitions to work with, and constrain it this way not to do something stupid.

Re: What if useful AI is a fantasy?

#52

Post by kennyadam »

chrisjj wrote: post #15
kennyadam wrote: post #9

Anecdotally, my 81 year old father was able to point his phone at the boiler and ask Gemini what the error message meant. It correctly identified the boiler model, identified the error code being displayed on the screen, explained what it meant, provided him with a way to confirm the problem by checking the water pressure guage and then asked to be shown the underside. It again correctly identified and described to h…

> That kind of thing is no fantasy and was amazing to witness. How about the case where the "AI" mis ided the error code? Did you test that? Result could be even more amazing... > He was able to do all this instead of spending hundreds on getting a plumber out at night ... or consulting the manual.

I edited in that i’d already looked up the code in the manual seconds after submitting the post, so you clearly just ignored that bit to rage.

Re: What if useful AI is a fantasy?

#53

Post by hermitShell »

nomel wrote: post #37
latentsea wrote: post #35

Earlier quoted context omitted.

Those layers are deterministic though...

Determinism is NOT a requirement, as proven by your existence of software devs. Again, shift the perspective to the people you work for, or to your fellow dev. They get along fine with you , who are also not deterministic. They check your output by setting requirements that you can prove are true in a way that satisfies them. You yourself guarantee correctness by tests , because you learned long ago that you yourself…

Do you think this necessarily sends us as a society (as a species) towards a future where we eventually stop understanding technology and science itself?

Like in Asimov's Foundation or Liu Cixin's 'Taking Care of God'.

Maybe we need two tiers of technology. Human in the loop up to the point of AI, and then the AI layer which depends on the human stack.

Re: What if useful AI is a fantasy?

#54

Post by bluegatty »

twister2920 wrote: post #46
rspeele wrote: post #27

Earlier quoted context omitted.

Even if you have a moral opposition to a single line of LLM-generated code marring your codebase (not even unit tests!) it can be incredibly useful to assist with reading code. Take a complex, sophisticated codebase you wrote, that only you have deep understanding of, and throw Codex or Claude at it and ask it questions. How does the Foo work? What happens if the user tries to Frozzle a Bar? Give me a guided tour of…

> Try to pretend that's not a useful tool to use when you have to fix bugs or add features to a large codebase you aren't already familiar with. how will I know the answers are accurate?

The same way you would know the answers of a junior developer would be.

Honestly, this reads a bit like you have not tried it, because the answer is 'kind of clear' if very nuanced.

You get a sense pretty quickly for the 'boundaries' of the tech - and though it will make errors for sure - you can sense what kinds of errors it's inclined to make.

Re: What if useful AI is a fantasy?

#55

Post by bluegatty »

win311fwg wrote: post #43
bluegatty wrote: post #39

Earlier quoted context omitted.

For bespoke things you 100% have to read the code, and for very straight forward SaaS stacks you still have to read. It depends on how much 'rails' there are. More rails, then there is a lot less to examine.

Why? I agree if you have chosen an ill-fitted technology stack where those aforementioned developers who didn't care about quality before LLMs tainted the training data then you do have to carefully read the code to watch for the LLM going crazy, because they will, but if you have chosen a suitable technology stack where the training data is overwhelmingly of high quality, then what need is there to read it? You alre…

" but the software will be fine if you don't."

" You already know that the quality will be on par with what you would have written yourself."

No - I totally don't agree.

I've almost never seen a situation where one does not have to read, other than for bug fixes.

In almost every case where I have 'gone under the good' to AI created system - it was bad.

The AI is terrible at grasping and coding concepts. It names things poorly, and over iterations, it will turn 'state' into a hairball.

Try asking it to do an DOM event adapter to filter and map events, like doing 'ime end even guaranted' - it will do write something and write tests that all pass, but there will be holes.

The logic will be inconsistent and won't follow an hard spec, and even getting it to write a hard, clean spec won't work.

Let me put it differently: it's not the 'code' that's the problem - it's the concepts that it is trying to represent, which it screws up.

Only where those 'concepts' are already clean, such as in a really good code-base that you only need to tweak - does it work well. And amazingly well.

But for doing anything from scratch - you have to read the code.

I don't think it's going to get better any time soon because I think there is a 'noisy channel' problem here, in that it can only get so good without proper specification, and specification is 'the developers job' that cannot be replaced, at best we can get tools to support us there.

Re: What if useful AI is a fantasy?

#56

Post by bluegatty »

AnimalMuppet wrote: post #36
bluegatty wrote: post #19

Earlier quoted context omitted.

It's delusional for someone to contemplate that millions of developers out there are just churning out 'non usable stuff' and don't care about quality and that somehow you have some 'magic insight' they don't have. And lacking in self awareness to publicize that. Obviously some people are using it recklessly, it's the same with everything. Do you think it's more likely that 'the entire world is wrong and you are righ…

> Do you think it's more likely that 'the entire world is wrong and you are right' ... OR ... that you just personally have not figured out how to use the technology in way that makes sense and/or you have a bad sense of of what quality means in most contexts. Yeah, see, I've seen this movie before. I've seen it with Lisp. "You just haven't used it enough to reach enlightenment! We all have! Keep going; you'll see it…

No - that is the wrong analogy.

'Lisp' is an arcane tool and it's highly debatable in the objective sense if 'Lisp' is better.

The analogy is someone is saying that 'cars are useless' and that everyone should just keep riding horses.

I'm saying that we can debate the merits of cars, but it's absurd to suggest that they are not useful, and it's reasonable to conclude that someone who cannot 'find value' in cars, has not learned to drive them.

Then debate the merits.

Re: What if useful AI is a fantasy?

#57

Post by win311fwg »

bluegatty wrote: post #55
win311fwg wrote: post #43

Earlier quoted context omitted.

Why? I agree if you have chosen an ill-fitted technology stack where those aforementioned developers who didn't care about quality before LLMs tainted the training data then you do have to carefully read the code to watch for the LLM going crazy, because they will, but if you have chosen a suitable technology stack where the training data is overwhelmingly of high quality, then what need is there to read it? You alre…

" but the software will be fine if you don't." " You already know that the quality will be on par with what you would have written yourself." No - I totally don't agree. I've almost never seen a situation where one does not have to read, other than for bug fixes. In almost every case where I have 'gone under the good' to AI created system - it was bad. The AI is terrible at grasping and coding concepts. It names thin…

> Try asking it to do an DOM event adapter to filter and map events

Why? While you can technically build a DOM event system in any language, you wouldn't, so this sounds like you are talking about Javascript (or Typescript, but same thing)... Which we already know LLMs cannot handle because the training data was full of poor quality code and thus LLMs also produce poor quality code. So, yes, you have to watch Javascript like a hawk here because the LLMs will go to crazy town if you don't pay attention. But that was already acknowledged.

Perhaps you accidentally chose the wrong example?

Re: What if useful AI is a fantasy?

#58

Post by bluegatty »

win311fwg wrote: post #57
bluegatty wrote: post #55

Earlier quoted context omitted.

" but the software will be fine if you don't." " You already know that the quality will be on par with what you would have written yourself." No - I totally don't agree. I've almost never seen a situation where one does not have to read, other than for bug fixes. In almost every case where I have 'gone under the good' to AI created system - it was bad. The AI is terrible at grasping and coding concepts. It names thin…

> Try asking it to do an DOM event adapter to filter and map events Why? While you can technically build a DOM event system in any language, you wouldn't, so this sounds like you are talking about Javascript (or Typescript, but same thing)... Which we already know LLMs cannot handle because the training data was full of poor quality code and thus LLMs also produce poor quality code. So, yes, you have to watch Javascr…

First - it's an examples, it's not about 'why do that?' - that's another question entirely.

Second - it is very wrong to imply 'the AI has bad training data for JS / TS' - that's generally not how it works.

Third - the main point is not the code, it's information distillation as I suggested. It has nothing to do with the programming language. Yes - it might plausibly work better in some than others, but that's not the crux of the issue.

As to to your question the 'DOM event filtering' is just a good example because there are ambiguities in the problem space.

The spec is odd, browsers have different stated behaviours, different actual behaviours - this creates difficult problem space surface. It doesn't matter what 'la

Re: What if useful AI is a fantasy?

#59

Post by win311fwg »

bluegatty wrote: post #58
win311fwg wrote: post #57

Earlier quoted context omitted.

> Try asking it to do an DOM event adapter to filter and map events Why? While you can technically build a DOM event system in any language, you wouldn't, so this sounds like you are talking about Javascript (or Typescript, but same thing)... Which we already know LLMs cannot handle because the training data was full of poor quality code and thus LLMs also produce poor quality code. So, yes, you have to watch Javascr…

First - it's an examples, it's not about 'why do that?' - that's another question entirely. Second - it is very wrong to imply 'the AI has bad training data for JS / TS' - that's generally not how it works. Third - the main point is not the code, it's information distillation as I suggested. It has nothing to do with the programming language. Yes - it might plausibly work better in some than others, but that's not th…

> it's an examples

It is an example, but an example that doesn't seem to fit the discussion.

> it is very wrong to imply 'the AI has bad training data for JS / TS' - that's generally not how it works.

Where would it have found good training data? The popularity of JS/TS means that the vast majority of the code that was written for it prior to LLMs is pretty lousy. Worse, even among the codebases that could be considered good, there is very little consistency between them so there is real good way for an AI to converge on what a good codebase even looks like in that ecosystem.

Regardless, no matter what explanation you want to choose, it remains abundantly clear that LLMs show time and time again that they struggle with JS/TS where there is no such struggle in certain other ecosystems. If you want to leverage LLMs to their full potential you are simply not going to be able to choose JS/TS (among others).

> As to to your question the 'DOM event filtering' is just a good example because there are ambiguities in the problem space.

Perhaps the disconnect here is that you have forgotten, but the discussion was about code quality. What about DOM event filtering in particular would lead an LLM that otherwise produces quality code to suddenly not? I have seen nothing to suggest that, in ecosystems that LLMs are suited for, that code quality declines when the problem space becomes more difficult. Code quality and problem space difficulty are orthogonal.

Or perhaps I am misunderstanding you, but my read here is that you point out that when a problem isn't well constrained an LLM may provide a solution that doesn't meet your expectations. That is true, but code quality and meeting user expectations are not one and the same. In fact, in the startup realm you often hear things like "users don't care about your code quality, just release something!" to really drive home that they are in entirely different dimensions.

Re: What if useful AI is a fantasy?

#60

Post by FloorEgg »

bluefirebrand wrote: post #47
FloorEgg wrote: post #30

Earlier quoted context omitted.

As someone who cooks most of my own meals (and almost never eats out) I agree. I also know quite a few people who don't know how to cook at all and the thought of cooking stresses them out. So far it seems that we can't all be good/capable at everything. Maybe you write all your own code and cook your own meals, but do you do your own taxes? would you defend yourself in court? would you fix your car yourself? Some pe…

> does the person want to develop or practice a skill, or do they just want it done I'm probably biased, but I think when it comes to professional work it should always, 100% of the time be the former. People who "just want it done" are the reason every piece of commercial software is becoming dogshit > So far it seems that we can't all be good/capable at everything Sure, but then stay the fuck out of the area I'm go…

Whether or not it's a shitty deal is ultimately up to you.

For all of history the environment has put evolutionary pressure on living things. If you choose to make a deal with the universe that you get what you want without having to adapt to changing circumstances then you're going to get a shitty deal. If you embrace what's outside your control and adapt gracefully you will get a good deal.

What that means in this particular case with AI and your skills as an engineer I can't say. I don't mean anything specific by it. My point is more so that a big source of unnecessary suffering is wanting the universe to be something that it's not.

Post reply on HN