Why are technologists trying so hard to make themselves redundant? This is like the Luddites themselves creating milling machines, eager for the foreman to show them the door. What gives?
If you don't, someone else will
AutoDev: Automated AI-driven development by Microsoft
121–130 of 232 posts
Re: AutoDev: Automated AI-driven development by Microsoft
#122I'm currently looking for a new job in development... should I be looking for an exit instead?
Re: AutoDev: Automated AI-driven development by Microsoft
#123I just finished a 30 page system prompt to do the same thing. I did not use Docker though. I can’t wait to see what they have done in detail. I’m sure like 20% of the people here have tried this too, right?
Re: AutoDev: Automated AI-driven development by Microsoft
#124I'm a software and product guy; what I don't get, is how we're going to replace us lowly engineers and not a substantial percentage of a company's Human stack. Close to engineering, what about the SCRUM masters? The testers. What about the product owners? What about devops? Further from us, what about the people signing up on our vacation? Or the ones signing up on our daily budgets when traveling, or hell, even the…
I feel like no matter how productive AI becomes, it will always be more productive when paired with humans. Since all companies will have AI, and since they will continue to play the same game as before--sell a good or service that provides a better value than the competition--that means that humans will still be the differentiator.
This is what Kasparov said about chess. He was wrong. The “centaur” phase lasted for a while but after a few years humans only make things worse. In fact that is the famous last words of humans in every area where computers have surpassed them.
Re: AutoDev: Automated AI-driven development by Microsoft
#125I guess I'd be interested to see how this performs against the same benchmark Devin was using. It's hard to deny that this isn't impressive. But I think there's two interesting parts to it. Claude 3 Opus already scored around 85-86% on these benchmarks, without an "AutoDev" style agentic approach. And all the same problems with HumanEval remain, the limitations in terms of what style of problems are chosen, and real…
I’ve been using Github copilot daily for two years and ChatGPT for 1 year now. And I think the tide lifts all the boats. I’ve seen a (perceived) 2-3x productivity increase. I think these tools slightly favor people in front of the learning curve of a particular field. I’ve been dabbling in all sorts of things so if you’re a focused expert (who doesn’t need to explore but just exploit) you probably get less than a 2x…
Humans will likely be able to contribute until full AGI, that is. I believe that for pure cognitive tasks, it's plausible the history of centaur chess/Go might be repeated on a much grander scale.
A key requirement is the AGI will need the autonomy, like a human expert, to collect data and perform experiments it needs; but it seems several companies are set on doing precisely that.
My advice and personal strategy is to broaden one's scope beyond pure cognitive tasks.
"if you value intelligence above all other human qualities, you’re gonna have a bad time" -- Ilya Sutskever, OpenAI's Chief Scientist, Oct 7, 2023.
-----
Exchanges in the link below seem informative:
"I don't know about chess, but in the similar game Go, the very best centaur teams were at a similar or maybe even slightly higher level than engines until recently. This was due to cheese strategies, details of the rulesets and better extrapolation of intermediate results. However, this changed a few years ago, when engines learned many of the tricks that the human could contribute. Since then, I believe pure engines are stronger in all practical applications.
Source: am national champion in centaur Go and worked on modern Go engines" " -- mafuy on May 18, 2021
Re: AutoDev: Automated AI-driven development by Microsoft
#126I don't think so AI will replace SW jobs. It will definitely change the way engineers work.
Software development involves:
1. Understanding the requirement
2. Solving the problem with given constraints (and thereby innovate)
3. Talking to stakeholders
4. Code (& unit tests)
5. Review #4
6. Troubleshoot in testing
7. Troubleshoot in prod, both perf & subtle issues (this is hard)
8. Take the input from #6 & #7 and use it as a feedback back to #2
9. Answering questions from users/support which involve suggesting workarounds and not just factual answers. Suggesting a workaround itself is a mini-problem solving which is an intersection of domain knowledge, knowing code at hand, understanding customer's situation, etc.
Coding is hardly taxing and time consuming when there is clarity what needs to be done and how it needs to be done.
Point #4 itself has sub-dimensions like performance, maintainability, test-ability, security, etc and it involves lot of subjective calls. Sometimes you have to deal with undocumented behavior of an API which is a tribal knowledge.
To troubleshoot in prod (esp subtle issues) require deep knowledge of the code at hand. This itself is a challenge when you are dealing with generated code and something you have not written yourself. Think about a human dealing with an existing large code-base when joining a team.
I understand all of the above is a spectrum and there are jobs in SWE which do not require so much rigor.
Key ability for breakthrough and the rest will fall in place: Code generated by AI is consistently put to production without human intervention for a sufficiently complex problem considering all good attributes (like backward compatibility, performance, etc)
Re: AutoDev: Automated AI-driven development by Microsoft
#127I'm a software and product guy; what I don't get, is how we're going to replace us lowly engineers and not a substantial percentage of a company's Human stack. Close to engineering, what about the SCRUM masters? The testers. What about the product owners? What about devops? Further from us, what about the people signing up on our vacation? Or the ones signing up on our daily budgets when traveling, or hell, even the…
See Appendix D in https://arxiv.org/pdf/2303.10130.pdf OpenAI’s “GPTs are GPTs: An early look at the labor market impact potential of large language models”
It’s not just construction workers. Still safe to be a pro athlete and a short order cook, too!
Re: AutoDev: Automated AI-driven development by Microsoft
#128Imagine we had this technology 20 years ago and we switched to fully automated web design. I bet all web design would still be deeply nested tables. I doubt that something like React, Vue, Svelte would exist.
As I see it, the issue with using tables for layout is that they made it difficult for humans to maintain and iterate over time, but I don't have a problem with an LLM working at that level. Equivalently, I don't care that the machine code that my system gets compiled into is full of GOTOs if I don't ever need to concern myself with that level.
Re: AutoDev: Automated AI-driven development by Microsoft
#129Earlier quoted context omitted.
I’ve been using Github copilot daily for two years and ChatGPT for 1 year now. And I think the tide lifts all the boats. I’ve seen a (perceived) 2-3x productivity increase. I think these tools slightly favor people in front of the learning curve of a particular field. I’ve been dabbling in all sorts of things so if you’re a focused expert (who doesn’t need to explore but just exploit) you probably get less than a 2x…
But how many experts do you need? Most dev jobs are mostly repetitive plumbing and those might disappear very fast because 1 dev + LLM >= 5 devs without. So what we'll see is an increase in company margins and an elimination of a large swathe of the middle class. The alternative theory is that if everyone can now quickly create systems multiple companies and competing products will pop off which will drive down the m…
People are conflating Copilot with evaporating demand.
Why pay for a CRUD interface when the chat interface does everything for you?
It’s App Store 2009 out there. Few are taking Assistants and “GPTs” seriously.
Programmers who do front end work can adapt. All that CRUD stuff hardly makes sense in isolation - it’s meant to make other people productive, usually admins. If the chatbot can do the admin’s job, which is a lot easier and more tenuous than the programmer’s, well that’s what’s going to happen.
Re: AutoDev: Automated AI-driven development by Microsoft
#130Earlier quoted context omitted.
Only if you provide an alternative way for the newly unemployed to earn a living. Otherwise you just get crime, hunger and eventually war.
When you consider how early we are in the evolution of software and the role it can play in our professional and personal lives, this seems like one of the easier problems to solve though.
Political problems are much harder to solve than technical ones.