Earlier quoted context omitted.
Everyone is a typist now, so I don't think it is farfetched that everyone is a SWE in the future.
Very few people are typist. Most people can use a keyboard, but the majority of non-technical people type at a speed which is orders of magnitude less than a professional typist. Another comment here mentions how they used colab while not being a SWE, but that is already miles ahead of what average people do with computers. There's people who have used computers for decades and wouldn't be able to do a sum in a sprea…
OpenAI Researchers Find That AI Is Unable to Solve Most Coding Problems
81–90 of 174 posts
Re: OpenAI Researchers Find That AI Is Unable to Solve Most Coding Problems
#82I’ve got 15 years of coding experience at some of the biggest tech companies. My personal opinion is that most people have no clue how good these AI coding systems already are. If you use something like RepoPrompt, where you selectively choose which files to include in the prompt, and then also provide a clear description of what changes you want to make—along with a significant portion of the source code—a model lik…
Re: OpenAI Researchers Find That AI Is Unable to Solve Most Coding Problems
#83Re: OpenAI Researchers Find That AI Is Unable to Solve Most Coding Problems
#84> The models weren't allowed to access the internet How many software developers could solve most even simple programming problems (except 'Hello world') with zero shot style (you write in notepad then can compile only once and execute once) without access to internet (stackoverflow, google search, documentation), tools (terminal, debugger, linter, cli)? I think then it's not the best comparison to make any judgement…
> How many software developers could solve most even simple programming problems (except 'Hello world') with zero shot style (you write in notepad then can compile only once and execute once) without access to internet (stackoverflow, google search, documentation), tools (terminal, debugger, linter, cli)? Many, there was a time when SO did not exist and people were able to solve non trivial problems. There was a time…
Re: OpenAI Researchers Find That AI Is Unable to Solve Most Coding Problems
#85Half of the work is specification and iteration. I think there’s a focus on full SWE replacement because it’s sensational, but we’ll more end up with SWE able to focus on the less patterned or ambiguous work and made way more productive with the LLM handling subtasks more efficiently. I don’t see how full SWE replacement can happen unless non-SWE people using LLMs become technical enough to get what they need out of…
> unless non-SWE people using LLMs become technical enough to get what they need out of them Non-SWE person here. In the past year I've been able to use LLMs to do several tasks for which I previously would have paid a freelancer on Fiverr. The most complex one, done last spring, involved writing a Python program that I ran on Google Colab to grab the OCR transcriptions of dozens of 19th-century books off the Interne…
I think this is the nuance most miss when they think about how AI models will displace work.
Most seem to think “if it can’t fully replace a SWE then it’s not going to happen”
When in reality, it starts by lowering the threshold for someone who’s technical but not a SWE, to jump in and do the work themselves. Or it makes the job of an existing engineer more efficient. Each hour less work needed spread across many tasks that would have otherwise gone to an engineer eventually sum up to a full time worth of an engineer. If it’s a Fiverr dev you eliminated the work of, that means the Fiverr dev will eventually go after the work that’s remaining, putting supply pressure on other devs
It’s the same mistake many had about self driving cars not happening because they couldn’t handle every road. No, they just need to start with 1 road, master that, and then keep expanding to more roads. Until they can do all of SF, and then more and more cities
Re: OpenAI Researchers Find That AI Is Unable to Solve Most Coding Problems
#86> The models weren't allowed to access the internet How many software developers could solve most even simple programming problems (except 'Hello world') with zero shot style (you write in notepad then can compile only once and execute once) without access to internet (stackoverflow, google search, documentation), tools (terminal, debugger, linter, cli)? I think then it's not the best comparison to make any judgement…
Re: OpenAI Researchers Find That AI Is Unable to Solve Most Coding Problems
#87> The models weren't allowed to access the internet How many software developers could solve most even simple programming problems (except 'Hello world') with zero shot style (you write in notepad then can compile only once and execute once) without access to internet (stackoverflow, google search, documentation), tools (terminal, debugger, linter, cli)? I think then it's not the best comparison to make any judgement…
You sound like someone who never used punch cards. I think most developers could do that if they trained. As someone who learned how to program before the internet, its just a different mindset and would take some time to adjust. I am doing that now where changes take a day to make it to staging and no local environment. You roll with it.
I hope HN never changes.
Re: OpenAI Researchers Find That AI Is Unable to Solve Most Coding Problems
#88> The models weren't allowed to access the internet How many software developers could solve most even simple programming problems (except 'Hello world') with zero shot style (you write in notepad then can compile only once and execute once) without access to internet (stackoverflow, google search, documentation), tools (terminal, debugger, linter, cli)? I think then it's not the best comparison to make any judgement…
I agree with your point, though. The "LLM" model just isn't a good fit for some tasks, in fact many tasks. It is good for creative writing, but even then only really because our standards for creative writing are pretty low. It doesn't write with any real creativity or flair in the writing. It can make things up and stay on topic. It is poor for anything where accuracy matters. It can't edit what it produces! Nobody writes things in one shot in reality, not even creative writing, but especially not code or technical writing. It needs to be able to do a whole suite of other things: move blocks of output around, rewrite chunks, expand chunks, condense chunks, check chunks against external sources or proper knowledge banks, compare chunks for internal consistency, and more. That is how we operate: at the level of functions or blocks of code, at the level of paragraphs and sentences and sections.
[1]: Yes, the opposite of the problem people here usually have with it, which is things being closed as duplicates. I think more duplicates should be deleted and redirected to a canonical answer, which is then a focus of improvement. Too often google searches give me barely answered or unanswered duplicates and I have to click around in the site to find the result Google clearly should have given me in the first place (better keyword matches, not closed, higher score, etc). I think StackOverflow do this intentionally so people have to click on more pages and see more ads.
Re: OpenAI Researchers Find That AI Is Unable to Solve Most Coding Problems
#89Earlier quoted context omitted.
I mean it’s been a couple of years! It may or may not happen but “scam” means intentional deceit. I don’t think anyone actually knows where LLMs are going with enough certainty to use that pejorative.
Is it intentional deceit to tell everyone it's leading to something when, as you correctly point out, nobody actually knows if it will?
Re: OpenAI Researchers Find That AI Is Unable to Solve Most Coding Problems
#90> The models weren't allowed to access the internet How many software developers could solve most even simple programming problems (except 'Hello world') with zero shot style (you write in notepad then can compile only once and execute once) without access to internet (stackoverflow, google search, documentation), tools (terminal, debugger, linter, cli)? I think then it's not the best comparison to make any judgement…
As one who organises competitive programming contests on a regular basis for university students, I would say almost every single one.