Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Devin: AI Software Engineer

cognition-labs.com

Post reply on HN
Print view View original

561–570 of 604 posts

Re: Devin: AI Software Engineer

#561

Post by mercenario »

MSFT_Edging wrote: post #67

Humans seek work that provides satisfaction and meaning in their life. For every technological advancement, artisans are the first to be made obsolete. Sure we have landfills full of unworn textiles, the market says its good, but overall, we keep destroying what allows humans to seek meaning. Our governments and society have made it clear, if you don't produce value, you don't deserve dignity. We have outsourced art…

> What will be left? Where will we be? Just stranded without dignity or purpose, left to rot when we no longer produce value.

Look at the unemployment rate history. Look at poverty rate history. Your argument is completely nonsense.

Re: Devin: AI Software Engineer

#562

Post by observationist »

LunaSea wrote: post #519
observationist wrote: post #86

Earlier quoted context omitted.

Software engineering still needs a human in the loop, just like art still needs to be prompted and tweaked by a human that can do composition, or writing that needs to be edited and refined, and so on. AI can't do 100% of the jobs, but it seems like we're somewhere past the 100x capabilities point. AI should be able to make a good employee able to do 100x the output at the same level of quality, and AI only gets more…

Lawyers famously don't care about productivity because they are paid by the hour.

Lawyers will usually bill at a 1 hour minimum, occasionally 30 minutes.

Lawyer A cares about productivity, and is paid by the hour, and so charges 1 hour of doc review at $4000, using the latest greatest technology, and is able to bill 1 hour of doc review to 100 clients. He begins to charge $3500/hr to undercut competition, and intersperses each week of work with training his paralegals and associates to amplify their capabilities.

Law Firm B doesn't care about productivity, and bills 1 hour of doc review at $4000 to 100 clients, but have to pay 100 lawyers in turn. Several weeks later, their clients have left for the cheaper, higher quality competition.

Being paid hourly doesn't mean you benefit from inefficiency. Small hungry shops who can rapidly embrace and integrate new technology are going to eat giant firms alive.

When you can set up agents to replace employees, consultants, contractors, and so on, even if the AI is only 90% as good as the average human, that's a staggering advantage for fast movers.

Re: Devin: AI Software Engineer

#563

Post by LunaSea »

observationist wrote: post #562
LunaSea wrote: post #519

Earlier quoted context omitted.

Lawyers famously don't care about productivity because they are paid by the hour.

Lawyers will usually bill at a 1 hour minimum, occasionally 30 minutes. Lawyer A cares about productivity, and is paid by the hour, and so charges 1 hour of doc review at $4000, using the latest greatest technology, and is able to bill 1 hour of doc review to 100 clients. He begins to charge $3500/hr to undercut competition, and intersperses each week of work with training his paralegals and associates to amplify the…

> Being paid hourly doesn't mean you benefit from inefficiency.

That is exactly what it means and what happened.

Lawyers are one of the last professions to move to the digital world.

For ages work was still done on paper and even right now most law firms don't use any non-LLM content search, autocompletes, content organisation, that could be considered "modern".

Re: Devin: AI Software Engineer

#564

Post by falcor84 »

HarHarVeryFunny wrote: post #524
mgummelt wrote: post #518

Earlier quoted context omitted.

You can't compare the accuracy of speech recognition to LLM task completion rates. A nearly-there yet incomplete solution to a Github issue is still valuable to an engineer who knows how to debug it.

Sure, and no doubt people paying for speech recognition 25 years ago were finding uses for it too. It depends on your use case. A 13% success rate is both wildly impressive and also WAY below the level where I would personally find something like this useful. I can't even see reaching for a tool that I knew would fail 90% of the time, unless I was desperate and out of ideas.

I disagree. I think about this a bit as having a developer intern, on whom I can't rely to take much of a workload, and definitely nothing on the critical path, but I could say to them "Take a look at these particular well-defined tasks on the backlog and see which ones you could make some progress on" - I feel there's good value in that.

And the nice thing about an AI here is that I think it will actually find a different subset of these tasks to be easy than a human would.

Re: Devin: AI Software Engineer

#565

Post by DinoDad13 »

PheonixPharts wrote: post #288
ThalesX wrote: post #73

As a developer but also product person, I keep trying to use AI to code for me. I keep failing, because of context length, because of shit output from the model, because of lack of any kind of architecture etc etc etc. I'm probably dumb as hell, because I just can't get it to do anything remotely useful, more than helping me with leetcode. Just yesterday I tried to feed it a simple HTML page to extract a selector, I…

It's worth pointing out that on their eval set for "issues resolved" they are getting 13.86%. While visually this looks impressive compared to the others, anything that only really works 13.86% of the time, when the verification of the work takes nearly as much time as the work would have anyway, isn't useful. The problem with this entire space is that we have VC hype for work that should ultimately still be being do…

That 13.68% is given with no context. Which issues, specifically did Devin solve? Any 3rd party verification?

Re: Devin: AI Software Engineer

#568

Post by falcor84 »

joshuahutt wrote: post #266
steve_adams_86 wrote: post #87

Earlier quoted context omitted.

No, I'm past the denial stage (I was certainly there, though... GPT threw me hard and I spent a good 2 months processing what was happening) but I don't, in this specific case, see this agent displacing many jobs yet. Well, not my kind of jobs. I'm already very worried for the entry level of our industry... I'm not sure what it means yet, but I don't think we will have many entrants into software careers within 5 yea…

I feel the same way. I think the immediate future is bright. Some will try to cram this technology into enterprise. It will do well. Those jobs will die. Others will leverage it alongside the more creative engineering tasks—they will thrive. Eventually, what we call software will change from what it is today into something much more accessible to these types of tools. The plateau we've landed on is just a compromise…

> Eventually, what we call software will change from what it is today into something much more accessible to these types of tools.

That's a very good point. There's that tongue-in-check assertion that Java as a language isn't a human-facing but rather a tool-facing programming language. And now I'm wondering, what would a high-level language that is actually designed from the ground up for manipulation by AIs rather than humans look like. And in contrast, I think that the human-facing interface to it might end up much more visual and graph-oriented, possibly similar to Unreal Engine's blueprints.

Re: Devin: AI Software Engineer

#569

Post by HarHarVeryFunny »

falcor84 wrote: post #564
HarHarVeryFunny wrote: post #524

Earlier quoted context omitted.

Sure, and no doubt people paying for speech recognition 25 years ago were finding uses for it too. It depends on your use case. A 13% success rate is both wildly impressive and also WAY below the level where I would personally find something like this useful. I can't even see reaching for a tool that I knew would fail 90% of the time, unless I was desperate and out of ideas.

I disagree. I think about this a bit as having a developer intern, on whom I can't rely to take much of a workload, and definitely nothing on the critical path, but I could say to them "Take a look at these particular well-defined tasks on the backlog and see which ones you could make some progress on" - I feel there's good value in that. And the nice thing about an AI here is that I think it will actually find a dif…

Yeah, but a developer intern already has human-level AGI to support the on-the-job developer training you are going to help give them. Any LLM available today, or probably in next 5-10 years for that matter, has neither AGI nor the ability to learn on the job.

My experience of working with interns, or low-skill developers, is that the benefit normally flows one way. You are taking time out from completing the project to help them learn. Someone/something of low capability isn't going to be relieving you of the large or complex tasks that would actually be useful, and be a time saver - they are going to try to do the small/simple tasks you could have breezed through, and suck up a lot of your time having to find out and explain to them how they messed up. Of course Devin doesn't even have online learning, so he'd be making the same mistakes over and over.

Re: Devin: AI Software Engineer

#570

Post by joshuahutt »

falcor84 wrote: post #568
joshuahutt wrote: post #266

Earlier quoted context omitted.

I feel the same way. I think the immediate future is bright. Some will try to cram this technology into enterprise. It will do well. Those jobs will die. Others will leverage it alongside the more creative engineering tasks—they will thrive. Eventually, what we call software will change from what it is today into something much more accessible to these types of tools. The plateau we've landed on is just a compromise…

> Eventually, what we call software will change from what it is today into something much more accessible to these types of tools. That's a very good point. There's that tongue-in-check assertion that Java as a language isn't a human-facing but rather a tool-facing programming language. And now I'm wondering, what would a high-level language that is actually designed from the ground up for manipulation by AIs rather…

Good question.

I expect that we're moving into a phase of AIs talking to AIs, and initially it'll be wasteful (because it'll be mostly English), but eventually, they'll derive their own language and seamlessly upgrade protocols when they determine they're talking to an AI. No clue how that will come about or what that language will look like, but honestly, it's kind of exciting.

Really interesting to think about how they might handle context, as well. Even though we have much bigger context windows (and they'll only get larger), context management is still a resource-management issue, which we'll probably continue to refine, as well. Imagine different strategies for managing both what is brought into the context of each request, as well as what form it could take (level of detail, additional references or commentary on it, etc). Things could get really unreadable even in English, and still be very interpretable for an LLM.

W.r.t. the graph-oriented interfaces, are you thinking something like Node-RED [1]? I'm seeing more and more people mention having LLMs produce non-text or structured outputs, like JSON, UI, and other things. Easy to imagine an LLM that wires together various open-source platforms, on-demand. Something like Node-RED for pipelines/functions, some UI tools for visualization/interactivity, other platforms for messaging, etc...

[1] https://nodered.org/

Post reply on HN