Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Thoughts on a month with Devin

answer.ai

Post reply on HN
Print view View original

181–190 of 194 posts

Re: Thoughts on a month with Devin

#181

Post by dimitri-vs »

a1j9o94 wrote: post #82
csomar wrote: post #20

Earlier quoted context omitted.

> One thing that surprised me a little is that there doesn't seem to be an "ask for help" escape hatch in it - it would work away for literally days on a task where any human would admit they were stuck? You are over-estimating the sophistication of their platform and infrastructure. Everyone was talking about Cursor (or maybe was it astroturfing?) but once I checked it out, it was not far from avante on neovim.

Cursor isn't designed to do long running tasks. As someone mentioned in another comment it's closer to a function call than a process like Devin. It will only do one task at a time that it's asked to do.

...for now.

They are pushing in this direction with the Composer Agent mode which can carry out a sequence of multi-file changes without you having to specify the files. It's pretty decent. If you're feeling brave there is also a beta "YOLO" mode that will auto approve these changes and run console commands.

Re: Thoughts on a month with Devin

#182

Post by dimitri-vs »

npilk wrote: post #115

This feels a bit like AI image generation in 2022. The fact that it works at all is pretty mindblowing, and sometimes it produces something really good, but most of the time there are obvious mistakes, errors, etc. Of course, it only took a couple more years to get photorealistic image outputs. A lot of commenters here seem very quick to write off Devin / similar ideas permanently. But I'd guess in a few years the pr…

If were comparing Devin to image generation, then Devin would be a version of Midjourney where you have no prompting skills, you only get one image and if you want something different you can only use the remix feature to make changes, oh and with each change the image resolution goes up and you get more jpeg artifacts.

Re: Thoughts on a month with Devin

#183

Post by hatefulmoron »

dimitri-vs wrote: post #180
nyarlathotep_ wrote: post #130

Earlier quoted context omitted.

> I wonder how many of us, here, understand that many jobs are going away if/when this works out for the companies. And the usual coping mechanism, "it will only be for low hanging fruit", "it will never happen to me because my $SKILL is not replaceable", will eventually not save you. Sure, if you are a unique expert on a unique field, but many of us don't have that luxury. And it'll be used to drive down salaries, t…

Is spending 4 years of your life on education that will likely only be 10-20% applicable to your job any less bizarre? It's just another hoop employers want to see you capable of jumping. If you ignore the syntax programming is just writing detailed instructions. Just because AI is able to translate English to code doesn't mean the 100s of decisions that need to be made go away. Someone still needs to write very deta…

> Just because AI is able to translate English to code doesn't mean the 100s of decisions that need to be made go away. Someone still needs to write very detailed instructions even if they are in English and it sure isn't going to be the people sitting in meetings all day.

What makes you think it will be you? The machines seem increasingly capable of converting English into different English, and if we take it as a given that they can convert English into code.. what are you there for? The people sitting in meetings might as well talk to the machine, to the extent they're willing to talk to you.

To be clear, the professional "meeting participants" are as much on the chopping block as we are, although that's not commonly pointed out.

Re: Thoughts on a month with Devin

#184

Post by barrell »

bufferoverflow wrote: post #152
rbren wrote: post #76

I'm one of the creators of OpenHands (fka OpenDevin). I agree with most of what's been said here, wrt to software agents in general. We are not even close to the point where AI can "replace" a software engineer. Their code still needs to be reviewed and tested, at least as much as you'd scrutinize the code of a brand new engineer just out of boot camp. I've talked to companies who went all in on AI engineers, only to…

We've seen exponential improvements in LLM's coding abilities. Went from almost useless to somewhat useful in like two years. Claude 3.5 is not bad really. I wanted to do a side project that has been on my mind for a few years, and Claude coded it in like 30 seconds. So to say "we are not even close" seems strange. If LLMs continue to improve, they will be comparable to mid level developers in 2-3 years, senior devel…

I think the most you could say is we’ve had improvements - from gpt 4 to whatever the current model is has definitely not been exponential improvements.

My experience is acctually they’ve become dramatically less helpful over the past two years (past year in particular). Claude seems not to have backslid much but it’s progression also has not been very fast at all (I’ve noticed no difference since the 3.5 launch despite several updates).

Everything grows sinusoidally and I feel we’re well past the tipping point into diminishing rate of improvement

Re: Thoughts on a month with Devin

#185

Post by xboxnolifes »

datadrivenangel wrote: post #138
marcyb5st wrote: post #11

Earlier quoted context omitted.

I completely agree with you. More precisely, I feel they are useful when you have specific tasks with limited scope. For instance, just yesterday I was battling with a complex SQL query and I got halfway there. I gave our bot the query and an half assed description of what I wanted/what was missing and it got it right on the first try.

Are you sure that your SQL query is correct?

Can they be sure even if they wrote it theirself?

Re: Thoughts on a month with Devin

#186

Post by qouteall »

huijzer wrote: post #32

I’ve used Cursor a lot and the conclusion doesn’t surprise me. I feel like I’m the one *forcing* the system in a certain direction and sometimes an LLM gives a small snippet of useful code. Sometimes it goes in the wrong direction and I have to abort the suggestion and force it into another direction. For me, the main benefit is having a typing assistant which can save me from typing one line here and there. Especial…

For moving argument order and removing parameter is already doable by a mature IDE and they are more predictable than AI (Jetbrains IDEs support them well. In VSCode it may need extensions.)

But adding parameter is not well supported by IDE as it requires knowing which value to pass. This is where Cursor can shine.

Re: Thoughts on a month with Devin

#187

Post by gtsop »

energy123 wrote: post #174
gtsop wrote: post #166

Earlier quoted context omitted.

o1 is way to slow to keep up with my flow of thinking in order to be of any help in the scenario i am describing

How are you using LLMs? With o1 I've switched to spelling out in lots of details what I want, then asking it it to one shot the full file, so with this approach the wait time has been acceptable.

I'm using it to orient me when tackling something new. For instance, the other day i was making a web driver client in shell and i asked it something apong the lines of "is there an http endpoint to the webdriver to get the class name of an element?"

These are the sort of questions i mostly do. "What is the best practice to read output from a device file in C", "Is there a cli tool to find dead typescript interface fields?"

Re: Thoughts on a month with Devin

#189

Post by sureglymop »

rbren wrote: post #76

I'm one of the creators of OpenHands (fka OpenDevin). I agree with most of what's been said here, wrt to software agents in general. We are not even close to the point where AI can "replace" a software engineer. Their code still needs to be reviewed and tested, at least as much as you'd scrutinize the code of a brand new engineer just out of boot camp. I've talked to companies who went all in on AI engineers, only to…

My biggest issue is just how often these agents make subtle, hard to spot mistakes.

It can seem great for certain tasks at first. Yesterday I had to add license headers to the top of a lot of source code files. The reason why I let the AI try is because the repository contained lots of different programming languages.

It was able to do this but I then realized that it just removed the last sentence of the text it was supposed to add.

Re: Thoughts on a month with Devin

#190

Post by IanCal »

verdverm wrote: post #163
rob137 wrote: post #142

Earlier quoted context omitted.

Can you say more?

pricing model, downtime, model support, pricing model, trying to take over the experience rather than assist within my experience. This last one is big, because Cursor wants to "reimagine" how developers work. The problem is the AIs are so far from being competent, they need to be kept on the sidelines and sub'd in occasionally, not be the quarterback. Oh, did I mention pricing model?

It's a cost per month, supports the top models (tbh sonnet 3.5 is the key model) and is VS code + some more UI.

> trying to take over the experience rather than assist within my experience

I'm not sure I understand. It's got autocomplete, chat, asking it to change files, but it's vscode plus some stuff. What's it "taking over"?

Post reply on HN