OpenAI thinks their code transformer can already do 37% of all coding “tasks”? Buzz off.
On a dataset of 164 programming problems: https://arxiv.org/pdf/2107.03374.pdf (Section 2.2). The dataset is here: https://github.com/openai/human-eval
A new way to predict when software jobs will become automated
101–110 of 129 posts
Re: A new way to predict when software jobs will become automated
#102Earlier quoted context omitted.
Self-checkout at supermarkets does 90% of the work. The 10% of work remaining is done by a human supervising the self-checkout stations. And same will happen to development. Lots of tasks will be automated and less people will be needed as a result.
Self-checkout isn't a great example since it's mainly just shifting the work from one human (the employee) to another (the customer), rather than true automation. As a side note, we were told decades ago that visual programming would similarly displace developers; however, I'm doing perfectly fine
The best thing about self checkout is the parallel stations, so that getting behind a single black swan customer won't set me back by half an hour.
In some cases, shifting work to the customer is a net positive, especially if the customer can estimate their own time and effort better than they can estimate and manage someone else doing it. Nearly a half century after The Mythical Man Month, managing software development is still widely agreed to be an unsolved problem.
The cost of software development is not just the time and effort, but the existential risk to your own job or business from its unpredictability. This is why I do all of my own programming, and also most of my own home repairs... and use the self checkout at the supermarket.
It didn't displace developers because the industry has grown to absorb them. In my own case, I use my programs to get my own job done, then hand the whole thing over to the project team, and the developers write the production code that ends up in the product... but not at my expense.
Re: A new way to predict when software jobs will become automated
#103> Once an AI gets just 1% efficient at automation, it achieves 100% efficiency quickly thereafter This is really optimistic and completely overlooks the 80/20 problems that have plagued both AI automation and traditional automation. The AI sweeps up the easy cases and humans are left to fix, or paper over with heuristics, the remainder. Example? Speech-to-text. It's pretty good and the spread of automated closed capt…
"If you can play it slowly, you can play it quickly"
Re: A new way to predict when software jobs will become automated
#104Earlier quoted context omitted.
I think this is thinking small. An AI doesn't need to use human readable programming languages or frameworks to solve problems. Also software engineers are expensive. Corporate greed will drive this faster.
So the ai will make solutions that are mostly correct but then can't be modified by humans to make them work? That is the problem with GAN generated art. It is pretty good but then you don't have a psd to modify it so it is useless for most purposes.
My idea as well. Let's create an application which will work, but won't be able to pass QC because testers are able to crash it with unexpected input.
But hey, we got code generated in 5 minutes. Can't bugfix it tho...
Re: A new way to predict when software jobs will become automated
#105Earlier quoted context omitted.
> Since the end goal of AI will result in a crisis for human, isn't it time for governments to step in and regulate? Your crisis has been added to the queue and will be dealt with in due course. Your crisis is .. 1,734 in the crisis queue. (seriously I think this is best left alone until it clarifies or subsumed into the general "regulation of harmful doing things with computers", like privacy laws. Self-driving cars…
I think that we are neglecting the possiblity that AI advancement and growth may proceed at an exponential pace as the author mentioned. The more people join AI industry, the faster AI advances, the more money invested in AI and causing more people to join the AI industry. Many people may join AI industry out of fear of losing their current jobs to AI. This may lead to a sudden avalanche of AI advances(domino effect)…
> overnight collapse of economy
How many decades is "overnight" supposed to represent here? Or do you really mean overnight? The WarGames scenario where the AI decides the only logical option is a pre-emptive nuclear strike?
Collapses, even in seriously banana republic countries, do not happen overnight. They are slow and agonising, and present options for .. kinetic solutions.
Re: A new way to predict when software jobs will become automated
#106Let me summarize the state of AI as a layman who has been following the main trends: 1) Cyc and reasoning - stuck for decades but making slow progress 2) GANNs - can detect a whole lot of things by being trained on data 3) MTCS - this is AlphaGo and AlphaZero. Adversarial networks that play each other. Have nearly solved any turn based game that can be expressed this way, and can be used for StarCraft etc. 4) Data fi…
> What scares me most, though, is deepfakes and swarming. You don’t need GREAT AI to take over political discussions
This. If an AI does take over the world it will be as a cult leader, with the willing help of millions of humans who've "done their own research" on youtube. The AI will of course be at the top of all search rankings and auto-suggestions by other "AI"s; that's table stakes for that kind of scenario.
Re: A new way to predict when software jobs will become automated
#107> Once an AI gets just 1% efficient at automation, it achieves 100% efficiency quickly thereafter This is really optimistic and completely overlooks the 80/20 problems that have plagued both AI automation and traditional automation. The AI sweeps up the easy cases and humans are left to fix, or paper over with heuristics, the remainder. Example? Speech-to-text. It's pretty good and the spread of automated closed capt…
The thing people forget about whether it's 80/20 (and the last 20 takes infinite time) or whether it's 80/20 (and it happens super-linearly) is the nature of the 20. One of two things tends to happen. 1) The 20 is a fundamentally different problem, but is itself solved by applying different new developments (e.g. most breakthrough devices: semiconductor manufacture and design, cellular phones, iPhone) 2) The 80 is so…
> If AI is "better than human in a fundamental way," it will be adopted overnight.
Are there any examples of this and how would we qualify them? Do plenty of pedestrian control systems also satisfy that criterion? After all, the SpaceX landing system is a feat of piloting that would probably be impossible for a human , but I don't believe the system is "AI" rather than an extremely well tuned more conventional thing of Kalman filters and feedback loops.
Re: A new way to predict when software jobs will become automated
#108Earlier quoted context omitted.
The thing people forget about whether it's 80/20 (and the last 20 takes infinite time) or whether it's 80/20 (and it happens super-linearly) is the nature of the 20. One of two things tends to happen. 1) The 20 is a fundamentally different problem, but is itself solved by applying different new developments (e.g. most breakthrough devices: semiconductor manufacture and design, cellular phones, iPhone) 2) The 80 is so…
I would argue that horse drawn carriages were a 100% solution for something like 8000 years of human history. It’s been barely over a hundred years where there has been something that could efficiently replace them and the ‘self driving’ modern replacement have maybe 0.01% the smarts of a horse. I mean, go get drunk and pass out on the back of your horse and the chances are real good it’s going to take you home witho…
A good discussion, previously on HN: https://acoup.blog/2022/05/13/collections-ancient-tanks-char... ; the carriage or similar appears to be four thousand years old.
Re: A new way to predict when software jobs will become automated
#109OpenAI thinks their code transformer can already do 37% of all coding “tasks”? Buzz off.
On a dataset of 164 programming problems: https://arxiv.org/pdf/2107.03374.pdf (Section 2.2). The dataset is here: https://github.com/openai/human-eval
Ah, they seem to think that 100% of programming problems literally amount to Easy leetcodes. I see no glue-code questions there, nor anything brownfield.
Re: A new way to predict when software jobs will become automated
#110Earlier quoted context omitted.
On a dataset of 164 programming problems: https://arxiv.org/pdf/2107.03374.pdf (Section 2.2). The dataset is here: https://github.com/openai/human-eval
Ah, they seem to think that 100% of programming problems literally amount to Easy leetcodes. I see no glue-code questions there, nor anything brownfield.
Yeah I don't know where benreesman pulled that statement from, neither the blog post nor the original OpenAI paper claims anything of the sort.
That aside, I'm not sure how impressive is an AI that could do 37% of "all coding tasks". It's like a self-driving car that only works 37% of the time.