Live data from Hacker News

AutoDev: Automated AI-driven development by Microsoft

arxiv.org

31–40 of 232 posts

Re: AutoDev: Automated AI-driven development by Microsoft

#33

I can see this being used for code much like ChatGPT is used for prose - to generate large amounts of meaningless output of low quality and dubious utility, serving to fulfill pointless metrics at best, and enabling bad actors at worst. As with prose, good human crafted content will always be highly valued and rewarded.

Counterexample for the sake of argument: photography almost entirely replaced human painting.

Photography replaced painting for creating a realistic likeness, but painting as art had moved beyond that already. Photography became a new art form itself, alongside (instead of replacing) painting.

Re: AutoDev: Automated AI-driven development by Microsoft

#34
I guess I'd be interested to see how this performs against the same benchmark Devin was using. It's hard to deny that this isn't impressive. But I think there's two interesting parts to it.

Claude 3 Opus already scored around 85-86% on these benchmarks, without an "AutoDev" style agentic approach.

And all the same problems with HumanEval remain, the limitations in terms of what style of problems are chosen, and real world relevance.

I hate writing these styles of comments because I'm acutely aware that a part of me is just worried. Worried about the speed of progress and worried about a changing landscape.

But I still wonder how much of this stuff is going to be transferrable to a real life software context.

Re: AutoDev: Automated AI-driven development by Microsoft

#35
post #20
post #4

Earlier quoted context omitted.

It would be like steering offshore teams, with the difference the team are now AI agents. Better look for architecture and management jobs, only a few select ones would still be allowed into the magic tower. Naturally after a heavy round of letcode interviews that have nothing to do with writing AI algorithms.

Management ? I'm not sure, if the AI makes engineers 10-20x more efficient, you will need way less engineers, and way less managers.

There is always a spot for more managers, welcome to big corps.

Re: AutoDev: Automated AI-driven development by Microsoft

#36
post #29
post #8

Earlier quoted context omitted.

Because test define the behaviour. The hardest part of requirements is describing exactly what you want. Tests are great way of doing that. I think AI software development is going to involve writing tests, which the AI agents then get passing. Or some other requirements language that allows for exactness where English can fall down.

So we get to do the shitty part.

It's shitty for the same reasons why AI will be bad at it. It involves talking to people, and trying to resolve ambiguous requirements.

Re: AutoDev: Automated AI-driven development by Microsoft

#37
post #3
post #2

If LLM/Agents contribute to software development, how does the role of software engineers evolve? SE should focus on : - System design - Integrations - Project management etc or the job will disappear in 10 years ?

Prompt Engineering! I'm kidding. Software engineering roles of the "glue things together" kind will evolve into a 'technical product' role, defining how systems work, especially at the edges where systems interact. Architects and staff engineers already do this. Lower down the food chain engineers who actually invent new things will do the same, only quicker if they have an AI tool assisting with the grunt work of te…

Good take! I think one specific development we might see would be an end of the SaaS micro tool era. Right now, most companies I advise have an absolute _jungle_ of different tools and a bunch of interns manually bridging the gap between them. If software development becomes a lot cheaper, wouldn't companies be more likely to just roll their own versions of what they need, to meet their needs exactly and have everything fit together?

Sounds like the job of a programmer wouldn't go away very near term in such a scenario. It might well become far less glamorous, less "technical", and not as well paid though. Comparable to an accountant maybe.

As someone who started their career right after the dotcom bubble burst, various people asked me "Why are you doing this? There's no money in software anymore". Turns out there sure was. Back then I didn't know that, and I didn't care. I didn't get into it for the money, I got into it because I had a passion for it. Maybe it'll become more like back then again. Doesn't seem so bad to me.

Re: AutoDev: Automated AI-driven development by Microsoft

#38

Earlier quoted context omitted.

Counterexample for the sake of argument: photography almost entirely replaced human painting.

Photography replaced painting for creating a realistic likeness, but painting as art had moved beyond that already. Photography became a new art form itself, alongside (instead of replacing) painting.

Yeah, I do not buy the idea that Photography replaced painting. Especially since capturing abstract forms, and less realistic interpretations is still something hard to accomplish in photos.

It did absolutely replace the need to commission someone to draw you, I guess. But that is a small subsection of what painting is and was.

Re: AutoDev: Automated AI-driven development by Microsoft

#39

I can see this being used for code much like ChatGPT is used for prose - to generate large amounts of meaningless output of low quality and dubious utility, serving to fulfill pointless metrics at best, and enabling bad actors at worst. As with prose, good human crafted content will always be highly valued and rewarded.

Counterexample for the sake of argument: photography almost entirely replaced human painting.

Did it? I’ve not seen people paying for photographs on the ceiling and have not seen feet long framing of portraits inside someone home. It just became possible to capture snapshots of moments as they happened

Re: AutoDev: Automated AI-driven development by Microsoft

#40
It's really depressing to see that big tech is essentially universally pushing Snake Oil. AI is a lie. LLMs are a legit tech that have some purpose, but LLMs will never evolve into anything remotely resembling the actual definition of AI.

They are flowery language generators and they will never be able to reason, understand, debate and criticize. They know nothing and therefore embody nothing. No matter how much computer power you waste on them the end result will always be bullshit. Nothing more. Nothing less.

To Big Tech I say: Prove me wrong.

Post reply on HN