Earlier quoted context omitted.
It's an observation, not an argument, but note that this is a paper about LLM mistakes . But if the language bothers you and you view LLMs as the solution, you're certainly free to feed it into an LLM yourself. To be frank, I find it a little off-putting to suggest being a non-native English speaker is "no longer valid."
Just a grammar check or a proofreading pass through an LLM is all it needs. It's an English-language paper, so we e.g. separate words by spaces. Hate this modern kneejerk reaction to get preachy and supercilious whenever anything vaguely cultural comes up.
A study on robustness and reliability of large language model code generation
221–229 of 229 posts
Re: A study on robustness and reliability of large language model code generation
#222Earlier quoted context omitted.
It's an observation, not an argument, but note that this is a paper about LLM mistakes . But if the language bothers you and you view LLMs as the solution, you're certainly free to feed it into an LLM yourself. To be frank, I find it a little off-putting to suggest being a non-native English speaker is "no longer valid."
Just a grammar check or a proofreading pass through an LLM is all it needs. It's an English-language paper, so we e.g. separate words by spaces. Hate this modern kneejerk reaction to get preachy and supercilious whenever anything vaguely cultural comes up.
Re: A study on robustness and reliability of large language model code generation
#223Earlier quoted context omitted.
I agree with the majority of your well considered comment. I'm okay with LLMs being really unreliable though. I think if they actually worked, it would be an unmitigated disaster for workers globally.
I think there’s arguments to be made both ways. I‘m generally of the belief that technology that works is a net win for humanity. But I think we need to stop being vague about what it means for an LLM to work. For me, I just want to tell an LLM to replace all the hard coded strings in my app with translation tags. Sadly, this isn’t possible to do reliably with what we have today. I’m not sure who’s job this would eli…
Re: A study on robustness and reliability of large language model code generation
#224Earlier quoted context omitted.
Just a grammar check or a proofreading pass through an LLM is all it needs. It's an English-language paper, so we e.g. separate words by spaces. Hate this modern kneejerk reaction to get preachy and supercilious whenever anything vaguely cultural comes up.
There might be preaching going on here, but it's not coming from me. :)
Re: A study on robustness and reliability of large language model code generation
#225Earlier quoted context omitted.
It's definitely a marketing strategy. "Our technology is so good it might destroy the world. So we're letting you use it for 19.99 a month." It's just a completely inconsistent position. Who benefits from the government saying only openAI and a few other companies can make this stuff? Would openAI rather talk about the (non-existent) existential threat of AGI or actual problems with their technology like data privacy…
Yeah, I think that's well said. I think the doomerism is more of a sell to investors than users sorta situation but it cuts both ways, which makes me sad for the tech industry having just made this same mistake with crypto. Who benefits from this? Anyone invested who doesn’t want to disappoint an investor (LPS, VCs, and founders) and/or anyone who’s ego is invested in the space and/or anyone who is raising money in t…
Re: A study on robustness and reliability of large language model code generation
#226I understand the need to stop people from editorializing in the titles, but I saw this within the first hour it was posted and just spent 15 minutes looking for it again because the title change didn't contain the two keywords I remembered - API and GPT.
Old Title - 62% of code generated by GPT-4 contains API misuses
Re: A study on robustness and reliability of large language model code generation
#227Earlier quoted context omitted.
Now hopefully I won't get horribly dinged for mistakes and poor advice here. What I am trying to say is that I too read "even senior devs don't understand the race conditions they create downstream." And I thought - oh God, don't I. But five minutes thought can help you walk through most issues. For most applications most of the time you can reason your way through without fear, and when you do encounter gnarly probl…
Or, sidestep it by creating an empty log file (if it doesn't exist) during initialization before creating multiple threads.
There probably is a "design pattern" for that but darned if I can draw it in UML
Re: A study on robustness and reliability of large language model code generation
#228Earlier quoted context omitted.
>My believe is that there are exactly zero such jobs. If your claim is that we're at least at 1%, then you're claiming that there are at least 269k fully automatable programmer jobs [1]. It shouldn't be hard for you to find at least one to start. If I built an entire car but I am missing the key. I cannot drive the car but the car is 99% complete. It's just missing the key. So what happens is, it can replace 0% of hu…
Your "1%" is a fantasy measure. It is purely subjective. It creates a false sense of linearity when the progress curve is not only not linear, it may not be possible to complete. You just handwave away any criticism with a "clearly" when it isn't clear at all. Mine is clearly measurable. You wanted to talk about job replacement; I'm measuring jobs replaced. You believe that we're at 48% complete (or 38% or whatever)…
unfortunately the hard topics and the topics that matter are the questions that matter most are often qualitative.
Measurable answers are easy. Nobody cares about those things because it's obvious. Has ChatGPT replaced any jobs for programmers right at this moment? Overall no. But why argue a point everybody already knows?
Will chatgpt replace you in the future as the technology develops? Is it a precursor to the machine that will replace you? Qualitative trend-lines point to an unknown artificial entity that does replace all of us. It is useful to speculate and extend the trendline in that direction.
You may wish to shield yourself to reality and only look at things with quantitative numbers and lengths that are measurable by rulers but the shield is an illusion what you are doing is blindness.
Natural selection and natural history is built from qualitative understanding. Without subjective analysis on qualitative data we would not be able to understand natural selection from the macro perspective. You would be ignorant of the concept of evolution and natural selection and not much different than a creationist.
Obviously you aren't like that, you're just using a tactic to win a discussion. Always use hard data works when it works. But ultimately this strategy has failed.