Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Human coders are still better than LLMs

antirez.com

Post reply on HN
Print view View original

711–720 of 762 posts

Re: Human coders are still better than LLMs

#711

Post by cushychicken »

zahlman wrote: post #709
cushychicken wrote: post #629

Earlier quoted context omitted.

What kind of things is it doing? Wrote me: - a SPI deserializer that sets a bit after 12 bits read in, to trigger a prefetch - an SDC constraints file for the deserializer that correctly identified the SPI clock and bus clock as separate domains requiring their own statement - a test bench that validated both that the prefetch bit was being set, and that it was being set at the proper time relative to the SPI clock -…

> I personally believe that most of the doubt and cynicism is due to: > 1) a pretty big collective identity crisis among software professionals, and > 2) a suspicion that LLMs make it so that anyone who is good at articulating the problem precisely no longer needs a software engineer as a translation specialist from specs to code. ... But "articulating the problem precisely" is a huge part of what software engineers…

I have a mountain of professional experience that indicates many software engineers are not very good at it either.

Why would I add a subpar translation layer into the process of achieving my goals? There’s no inherent value in that.

Re: Human coders are still better than LLMs

#712

Post by cyjackx »

ZoomZoomZoom wrote: post #596
cyjackx wrote: post #456

Earlier quoted context omitted.

I remember when I first learned Java, having to just accept "public static void main(String[] args)" before I understood what any of it was. All I knew was that went on top around the block and I did the code inside it. Should people really understand every syntax there before learning simpler commands like printing, ifs, and loops? I think it would yes, be a nicer learning experience, but I'm not sure it's actually…

If you need to learn "public static void main(String[] args)" just to print to a screen or use a loop, means you're using the wrong language. When it's time to learn Java you're supposed to be past the basics. Old-school intros to programming starts with flowcharts for a reason. You can learn either way, of course, but with one, people get tied up to a particular language-specific model and then have all kinds of dis…

I mean, I didn't need to learn those things, they were just in whatever web GUI I originally learned on; all I knew was that I could ignore it for now, a la the topic. Should the UI have masked that from me until I was ready? I suppose so, but even then I was doing things in an IDE not really knowing what those things were for until much later.

Re: Human coders are still better than LLMs

#713

Post by KTibow »

lazyasciiart wrote: post #580
materiallie wrote: post #552

Earlier quoted context omitted.

This is my experience, too. As a concrete example, I'll need to write a mapper function to convert between a protobuf type and Go type. The types are mirror reflections of each other, and I feed the complete APIs of both in my prompt. I've yet to find an LLM that can reliability generate mapping code between proto.Foo{ID string} to gomodel.Foo{ID string}. It still saves me time, because even 50% accuracy is still hal…

An LLM ability to do a task is roughly correlated to the number of times that task has been done on the internet before. If you want to see the hype version, you need to write a todo web app in typescript or similar. So it's probably not something you can fix with prompts, but having a model with more focus on relevant training data might help.

These days, they'll sometimes also RL on a task if it's easy to validate outputs and if it seems worth the effort.

Re: Human coders are still better than LLMs

#714

Post by JoshCole »

gitaarik wrote: post #710
JoshCole wrote: post #688

Earlier quoted context omitted.

> Only if you create new reward models for C, the output for D will improve, and so on. Again, tons of false claims. One is that 'you' have to create the reward model. Another that it has to be human-curated at all. Yet another is that you even need to do that at all: you can instead have the model build a bigger model of itself, train using its existing resources or more of them, then synthesize itself back down. An…

Ok, so you claim that LLMs can get smarter without human validation. So why do they hallucinate at all? And why are all reward models currently curated by humans? Or are you claiming they aren't?

I don't find it reasonable that you didn't understand my corrections, because current AI already do. So I'm exiting the conversation.

https://chatgpt.com/share/683a3c88-62a8-8008-92ef-df16ce2e8a...

Re: Human coders are still better than LLMs

#715

Post by sandspar »

layer8 wrote: post #654
skydhash wrote: post #650

Earlier quoted context omitted.

If you have something that generated 20 lines of assembly that takes 100x times more than the 2 lines of clever instructions you know, you'd have the same stance even if the higher level was easier to use. Then those kind of performance tricks ceases to matter. But reliability still do. And the reasons we use higher and higher level of programming languages is because they increase reliability and simplicity (at the…

You are arguing based on the merits of the technology, which is fine, but wasn't my point. I was arguing that derision tends to happen no matter what, and thus doesn't indicate much about the merits of the technology one way or the other.

If the idea has staying power, then derision tends to be followed by hatred or anger. We're already seeing quite a bit of hatred and anger toward AI. I think even the people who are ridiculing AI currently would agree with the statement, "There will be more anger toward AI in the near future." As in, they already know it's here to stay, whether they admit it to themselves or not.

Re: Human coders are still better than LLMs

#716

Post by drob518 »

johnb231 wrote: post #521
drob518 wrote: post #479

Earlier quoted context omitted.

It’s only a matter of time before Google merges search with Gemini. I don’t think you’ll have to wait long.

Already happened. Google search includes an AI generated response. Gemini prompts return Google search results.

See. They saw my comment and got it done. Dang, that was quick.

Re: Human coders are still better than LLMs

#717

Post by zahlman »

insane_dreamer wrote: post #527

Coders may want to look at translators for an idea of what might happen. Translation software has been around for a couple of decades. It was pretty shitty. But about 10 years ago it started to get to the point where it could translate relatively accurately. However, it couldn't produce text that sounded like it was written by a human. A good translator (and there are plenty of bad ones) could easy outperform a machi…

> When LLMs came out, and they now produce text that sounded like it was written by a native speaker (in major languages).

But they still often get things completely wrong, especially in high-context languages such as Japanese. There often isn't a way to convey the necessary context in text. For example, a Japanese live-streamer who says "配信来てくれてありがとう" means "Thank you for coming to (watch) the stream", but DeepL will give "Thanks for coming to the delivery." - because 配信 actually does mean "delivery" in ordinary circumstances, and that's just the word they idiomatically use to refer to a stream. No matter how much you add from the transcript before or after that, it doesn't communicate the essential fact that the text is a transcript of a livestream.

(And going the other way around, DeepL will insert a に which is grammatically correct but rarely actually uttered by livestreamers who are speaking informally and colloquially; and if you put "stream" in the English, it will be rendered as a loanword ストリーム which is just not what they actually say. Although I guess it should get credit for realizing that you don't mean a small river, which would be 小川.)

(Also, DeepL comes up with complete incomprehensible nonsense for 歌枠, where even basic dictionaries like Jisho will get it right.)

More obviously, they get pronouns and roles wrong all the time - they can't reliably tell whether someone is saying that "I did X" or "you did X" because that may depend on facts of the natural world outside of the actual text (which wouldn't include more information than the equivalent of "did X"). A human translator for, say, a video game cutscene may be able to fix these mistakes by observing what happened in the cutscene. The LLM cannot; no matter how good its model of how Japanese is spoken, it lacks this input channel.

Re: Human coders are still better than LLMs

#718

Post by zahlman »

cushychicken wrote: post #711
zahlman wrote: post #709

Earlier quoted context omitted.

> I personally believe that most of the doubt and cynicism is due to: > 1) a pretty big collective identity crisis among software professionals, and > 2) a suspicion that LLMs make it so that anyone who is good at articulating the problem precisely no longer needs a software engineer as a translation specialist from specs to code. ... But "articulating the problem precisely" is a huge part of what software engineers…

I have a mountain of professional experience that indicates many software engineers are not very good at it either. Why would I add a subpar translation layer into the process of achieving my goals? There’s no inherent value in that.

> Why would I add a subpar translation layer into the process of achieving my goals?

Because you don't have a choice. Your thoughts are not code.

Re: Human coders are still better than LLMs

#719

Post by lanstin »

cogogo wrote: post #619
lanstin wrote: post #480

Earlier quoted context omitted.

I think you can learn a lot from debugging, and all the code I've put into prod from LLM has needed debugging (rather more than it should from the LOC count).

I agree and that’s definitely part of my current learning process. But I think someone dependent on a LLM from day one might struggle to debug their LLM generated code. Probably just feed it back to the LLM and their mileage is definitely going to vary with that approach.

Maybe, but if I recall (from long long ago) in learning how to program, the process of debugging ones code was almost more enlightening than writing it initially - so many loops of not understanding the implications of the code and then smacking my forehead - and remembering it for ever. Like being able to type code but not debug is pretty worthless.

Re: Human coders are still better than LLMs

#720

Post by codr7 »

sanderjd wrote: post #538
codr7 wrote: post #497

Earlier quoted context omitted.

And you will keep dreading it for as long as you use them, since you learn nothing from solutions served on a silver plate.

The point is that I don't dread it anymore, because now there are tools that make it a lot easier the one or two times a year I have some reason to use it.

Just wait until they're not there anymore, that's when you realize what you sacrificed.
Post reply on HN