Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I Don't Want to Code with LLM's

blaines-blog.com

Post reply on HN
Print view View original

11–20 of 48 posts

Re: I Don't Want to Code with LLM's

#11

Post by Our_Benefactors »

lmf4lol wrote: post #7
Our_Benefactors wrote: post #4

Earlier quoted context omitted.

The data showed llms are better. This put debate to rest. Now we are post-debate.

give me one seriously peer reviewed study please with proper controls i wait

Go ahead and move the goalposts now... This took about 2 minutes of research to support the conclusions I know to be true. You can waste time as long as you choose in academia attempting to prove any point, while normal people make real contributions using LLMs.

### An Empirical Evaluation of Using Large Language Models for Automated Unit Test Generation We evaluate TESTPILOT using OpenAI’s gpt3.5-turbo LLM on 25 npm packages with a total of 1,684 API functions. The generated tests achieve a median statement coverage of 70.2% and branch coverage of 52.8%. In contrast, the state-of-the feedback-directed JavaScript test generation technique, Nessie, achieves only 51.3% statement coverage and 25.6% branch coverage. - *Link:* [An Empirical Evaluation of Using Large Language Models for Automated Unit Test Generation (arXiv)](https://arxiv.org/abs/2302.06527)

---

### Field Experiment – CodeFuse (12-week deployment) - Productivity (measured by the number of lines of code produced) increased by 55% for the group using the LLM. Approximately one third of this increase was directly attributable to code generated by the LLM. - *Link:* [CodeFuse: Generative AI for Code Productivity in the Workplace (BIS Working Paper 1208)](https://www.bis.org/publ/work1208.htm)

Re: I Don't Want to Code with LLM's

#12

Post by thomas_moon »

The only viewpoint I really agree with in this article is the "use it or lose it" mentality. Skills are developed and maintained by practicing them, but if all the author really wants to do is write code, then LLMs are literally an answer to their prayers!

You can enable virtually free test driven development. Write the test names down and let the LLM implement them for you. You save 50% of your time and you get to go to town on implementation and or optimizations.

You can have the LLM take the non-tech-counterparts description of a bug and have it point you at precise lines of code to investigate rather than grepping around a codebase you might not know well.

You can onboard to new languages, frameworks, repositories extremely fast by having a partner (the LLM) explain implementation patterns and approaches on demand! You don't even need to talk to another human being! Get your questions answered in seconds and start coding!

You can rapidly prototype. You can get immediate code reviews. You can rubber duck. You can visualize business/logic flows and code branching to better understand existing implementations. You can even have the LLM write an implementation plan for you then write the code yourself!

If you cant find a way to write more code with LLMs, its either an imagination or skill issue.

Re: I Don't Want to Code with LLM's

#13

Post by Refreeze5224 »

Our_Benefactors wrote: post #2

I can’t take this article seriously, and neither should you. Being anti AI/anti LLM is solidly in the Luddite camp; there’s really no more debates to be had. Every serious inquiry shows productivity gains by using ai. It’s anyone’s prerogative to continue to advocate for the horse and buggy over the automobile, but most people won’t bother to take the discussion seriously.

Then put me solidly in the Luddite camp. I think you should look into the history of the Luddites though. They were not against technology; they were against technology that destroyed jobs.

AI is about destroying working-class jobs so that corporations and the owning class can profit. It's not about writing code or summarizing articles. Those are just things workers can do with it. That's not what it's actually for. Its purpose is to reduce payroll costs for companies by replacing workers.

Re: I Don't Want to Code with LLM's

#14

Post by AllegedAlec »

Our_Benefactors wrote: post #2

I can’t take this article seriously, and neither should you. Being anti AI/anti LLM is solidly in the Luddite camp; there’s really no more debates to be had. Every serious inquiry shows productivity gains by using ai. It’s anyone’s prerogative to continue to advocate for the horse and buggy over the automobile, but most people won’t bother to take the discussion seriously.

> Being anti AI/anti LLM is solidly in the Luddite camp; there’s really no more debates to be had. Every serious inquiry shows productivity gains by using ai.

"Guys this debate is so stupid. Every serious inquiry shows productivity gains when we take away all senses, jack workers into the matrix and feed them a steady diet of speed intravenously. This put debate to rest. Now we are post-debate"

Something can increase productivity and still not be good.

Re: I Don't Want to Code with LLM's

#15

Post by xg15 »

Our_Benefactors wrote: post #2

I can’t take this article seriously, and neither should you. Being anti AI/anti LLM is solidly in the Luddite camp; there’s really no more debates to be had. Every serious inquiry shows productivity gains by using ai. It’s anyone’s prerogative to continue to advocate for the horse and buggy over the automobile, but most people won’t bother to take the discussion seriously.

You mean like the study that found a 20% productivity drop instead of gain?

Re: I Don't Want to Code with LLM's

#16

Post by Tade0 »

We can't uninvent LLMs. They're here already and the best course of action for everyone is to learn to live with them.

That being said I noticed that the more opinionated a language/framework/library is, the worse off one is using LLMs.

I was surprised by this, but then I put a particularly fishy line into GitHub's search box. What I saw were piles upon piles of bad practices and incorrect usages. There's a lot of bad code there and LLMs are learning from it.

Re: I Don't Want to Code with LLM's

#17

Post by pavel_lishin »

thomas_moon wrote: post #12

The only viewpoint I really agree with in this article is the "use it or lose it" mentality. Skills are developed and maintained by practicing them, but if all the author really wants to do is write code, then LLMs are literally an answer to their prayers! You can enable virtually free test driven development. Write the test names down and let the LLM implement them for you. You save 50% of your time and you get to g…

> You can enable virtually free test driven development. Write the test names down and let the LLM implement them for you. You save 50% of your time and you get to go to town on implementation and or optimizations.

That's assuming that it writes good tests, and that you don't care to take the time to verify the tests it wrote, no?

Re: I Don't Want to Code with LLM's

#18

Post by truetraveller »

thomas_moon wrote: post #12

The only viewpoint I really agree with in this article is the "use it or lose it" mentality. Skills are developed and maintained by practicing them, but if all the author really wants to do is write code, then LLMs are literally an answer to their prayers! You can enable virtually free test driven development. Write the test names down and let the LLM implement them for you. You save 50% of your time and you get to g…

No. There's a difference between writing code, and getting code written. LLMs are the second.

Re: I Don't Want to Code with LLM's

#19

Post by iLemming »

What I don't understand in all that noise from the LLM critics - they keep talking about how LLMs are so horrendously bad at writing code as if that's the only thing we're trying to use them for. As if they're not even genuine programmers, working on real projects, touching code every day.

Software crafting is so much more than merely writing code. There's a significant amount of reading code that goes into it. Code written by you. Code written by someone else. Someone else's code that you butchered with your edits, your own code butchered by someone else, and everything intertwined in between. Code that can't easily be explained by looking at it - sometimes you have to find relevant PRs, tickets, documentation, related online communication, some loosely-related code sitting someplace else, etc.

LLMs absolutely can help you read code, just as they are very capable of helping someone study a book or an academic paper. Denying that fact simply is ignorance. Of course, LLMs are absolutely capable of leading you in the wrong direction, confusing you, and giving you incorrect facts, even when you're studying text in plain English, just like it's possible to end up at the bottom of a lake when driving a car. Everyone needs to exercise caution and "know what the fuck they're doing" when using a model. But calling LLMs "bullshit generators" and "magic 8 balls" is so stupid. Sure, if you use it to perform bullshit stuff, it will generate nothing but bullshit.

Re: I Don't Want to Code with LLM's

#20

Post by steve_adams_86 »

pavel_lishin wrote: post #17
thomas_moon wrote: post #12

The only viewpoint I really agree with in this article is the "use it or lose it" mentality. Skills are developed and maintained by practicing them, but if all the author really wants to do is write code, then LLMs are literally an answer to their prayers! You can enable virtually free test driven development. Write the test names down and let the LLM implement them for you. You save 50% of your time and you get to g…

> You can enable virtually free test driven development. Write the test names down and let the LLM implement them for you. You save 50% of your time and you get to go to town on implementation and or optimizations. That's assuming that it writes good tests, and that you don't care to take the time to verify the tests it wrote, no?

I do find LLMs useful for scaffolding this stuff, but yeah, good test writing still seems to require a lot of hand-holding. I don't mind. I'm happy with my tests and they get written faster. Hand-holding and verifying is still faster than how I used to do it, and the LLMs admittedly capture more cases than I did without them. They will try to create test cases that make no sense too, but it's worth having to delete if it means it also comes up with test cases that I didn't.
Post reply on HN