Earlier quoted context omitted.
How is code not just exactly that special requirements language you are looking for?
It's more about defining outcomes exactly rather than a sequence of exact steps.
Aka logic programming.
AutoDev: Automated AI-driven development by Microsoft
41–50 of 232 posts
Re: AutoDev: Automated AI-driven development by Microsoft
#42Re: AutoDev: Automated AI-driven development by Microsoft
#43Re: AutoDev: Automated AI-driven development by Microsoft
#44[flagged]
Isn't this just wrapping existing agents with a "secure" coding environment? Does this represent a meaningful improvement in AI?
> Isn't this just wrapping existing agents with a "secure" coding environment?
It is. The paper has this paragraph where autodev requires rules for it's setup.
> The user initiates the process by configuring rules and actions through yaml files. These files define the available commands (actions) that AI agents can perform. Users can leverage default settings or fine-grained permissions by enabling/disabling specific commands, tailoring AutoDev to their specific needs. This configuration step allows for precise control over the AI agents’ capabilities. At this stage the user can define the number and behavior of the AI agents, assigning specific responsibilities, permissions, and available actions. For example, the user could define a ”Developer” agent and a ”Reviewer” agent, that collaboratively work towards an objective.
Their example is very simple as well, and the code generated does not cover the usecase where the sentence might have `I'm` or `I've`. I feel that the tool would generate a template with missing logic and then the overhead falls on the developer to figure it out and then add them (since this entire process is automated).
The original comment is just farming upvotes from the `AI hype` people on HN. It's become quite common with gangs of users upvoting each other and downvoting any other narrative that does not fit into their `AI` world.
> Does this represent a meaningful improvement in AI?
It seem more like setting up a factory using a LLM and then making it generate other factories.
Re: AutoDev: Automated AI-driven development by Microsoft
#45Earlier quoted context omitted.
Yeah. The shocking speed of progress on AI - especially for coding - tells me quite clearly what the future holds for much of software engineering. Despite the human factors (yes, requirements gathering and specification is messy) our profession is intrinsically automatable at the end of the day. Being in denial is pointless.
I think software engineering will just go up the stack to requirements engineering. Those requirements will be specified as tests or in a very high level language that doesn't allow for ambiguous meanings. Which is essentially another programming language just more high level.
I agree, and I’m thrilled about it. I love programming but while I’ve had bouts of enjoyment coding “in the small”, it’s my least favorite part. I love systems and software architecture, and seeing things come to life above all else. ChatGPT is greatly improving my enjoyment of it all.
I also doubt traditional programming languages are going anywhere in the short term, seeing there are trillions of lines of code for critical systems in the wild. Good luck replacing that in a few years only, or fully delegating its maintenance to Devin.
Re: AutoDev: Automated AI-driven development by Microsoft
#46[flagged]
Yeah. The shocking speed of progress on AI - especially for coding - tells me quite clearly what the future holds for much of software engineering. Despite the human factors (yes, requirements gathering and specification is messy) our profession is intrinsically automatable at the end of the day. Being in denial is pointless.
"The shocking speed of progress on AI"
Not sure what is shocking. Some of us used Codex in 2021 and nothing has changed since that has revolutionised software development by any capacity. Feels like people are just saying whatever some tech CEO or worse, some guru says.
Progress isn't some infinite exponential either where you can just keep increasing the capacity of models and expect better results. Also not sure where they are going to get better training data when they already polluted the internet with AI generated data.
Re: AutoDev: Automated AI-driven development by Microsoft
#47I guess I'd be interested to see how this performs against the same benchmark Devin was using. It's hard to deny that this isn't impressive. But I think there's two interesting parts to it. Claude 3 Opus already scored around 85-86% on these benchmarks, without an "AutoDev" style agentic approach. And all the same problems with HumanEval remain, the limitations in terms of what style of problems are chosen, and real…
I agree that these benchmarks don’t mean as much anymore because it’s highly likely they were already present in the training set, but also believe it’s likely these tools will be significantly better in a few research cycles
Re: AutoDev: Automated AI-driven development by Microsoft
#48Earlier quoted context omitted.
Prompt Engineering! I'm kidding. Software engineering roles of the "glue things together" kind will evolve into a 'technical product' role, defining how systems work, especially at the edges where systems interact. Architects and staff engineers already do this. Lower down the food chain engineers who actually invent new things will do the same, only quicker if they have an AI tool assisting with the grunt work of te…
Good take! I think one specific development we might see would be an end of the SaaS micro tool era. Right now, most companies I advise have an absolute _jungle_ of different tools and a bunch of interns manually bridging the gap between them. If software development becomes a lot cheaper, wouldn't companies be more likely to just roll their own versions of what they need, to meet their needs exactly and have everyth…
I think one specific development we might see would be an end of the SaaS micro tool era.
Possibly, but if SaaS tool providers make APIs so AI tools can consume the data, that would open up a huge ecosystem of AI 'plugins' that would be immensely valuable. I think OpenAI have something like that already.
Re: AutoDev: Automated AI-driven development by Microsoft
#49Earlier quoted context omitted.
Photography replaced painting for creating a realistic likeness, but painting as art had moved beyond that already. Photography became a new art form itself, alongside (instead of replacing) painting.
Yeah, I do not buy the idea that Photography replaced painting. Especially since capturing abstract forms, and less realistic interpretations is still something hard to accomplish in photos. It did absolutely replace the need to commission someone to draw you, I guess. But that is a small subsection of what painting is and was.
There's a simple test: Do we have less of more illustrators now than we had in 1816 (quick Wikipedia lookup for when the camera was invented)? We should probably adjust this for total population, i.e. compare the percentage of illustrators. I don't have the numbers, but my gut feeling would be it's a lot higher now than it was back then.
The fallacy is thinking that humans only do work somebody needs, and that our needs are static. Turns out they evolve, sometimes in quite unexpected and even hilarious ways. Humanity seems to have a predisposition for keeping everyone busy.
Re: AutoDev: Automated AI-driven development by Microsoft
#50It's really depressing to see that big tech is essentially universally pushing Snake Oil. AI is a lie. LLMs are a legit tech that have some purpose, but LLMs will never evolve into anything remotely resembling the actual definition of AI. They are flowery language generators and they will never be able to reason, understand, debate and criticize. They know nothing and therefore embody nothing. No matter how much comp…
> LLMs are a legit tech that have some purpose, but LLMs will never evolve into anything remotely resembling the actual definition of AI.
It doesn't need to satisfy some definition of AGI to be useful to big tech.
AI just needs to make workers more efficient so companies can either cut costs (fire people) or produce more with the same workforce.