121–130 of 194 posts
Most the problems you mentioned will likely be solved with the next iterations of Devin or similar product. I can say that because I work daily with Claude as an agent over mcp, and the problems you mentioned feel very familiar. Based on the type of the issues you mentioned, Devin isn't likely using o1 yet. A workflow like o1 for planning, Claude for Coding, o1 for review, etc., would work better. The problems you me…
> Even more concerning was Devin’s tendency to press forward with tasks that weren’t actually possible. (...) > Devin spent over a day attempting various approaches and hallucinating features that didn’t exist. One of the big problems of GenAI is its inability to know what they don't know. Because of that, they don't ask clarifying questions. Humans, in the same situation, would spend a lot of time learning before th…
Earlier quoted context omitted.
Is that true? I'd like to think my commits are less burdensome to review than a fresh out of boot camp junior dev especially if all that's being done is fixing linter issues. Perhaps there's a small benefit, but doesn't seem like a major productivity boost.
A junior dev is not a good approximation of the strengths and weaknesses of these models.
As someone who uses AI coding tools daily and has done a fair amount of experimentation with different approaches (though not Devin), I feel like this tracks pretty well. The problem is that Devin and other "agentic" approaches take on more than they can handle. The best AI coders are positioned as tools for developers, rather than replacements for them. Github Copilot is "a better tab complete". Sure, it's a neat de…
I agree with this. However, we must not delude ourselves and understand that corporate is pushing for replacement. So there will be a big push to improve on tools like Devin. This is not a conspiracy theory, in many companies (my wife's, for example) they are openly stating this: we are going to reduce (aka "lay off") the engineering staff and use as much AI solutions as possible.
I wonder how many of us, here, understand that many jobs are going away if/when this works out for the companies. And the usual coping mechanism, "it will only be for low hanging fruit", "it will never happen to me because my $SKILL is not replaceable", will eventually not save you. Sure, if you are a unique expert on a unique field, but many of us don't have that luxury. Not everyone can be a top of the cream specialist. And it'll be used to drive down salaries, too.
As someone who uses AI coding tools daily and has done a fair amount of experimentation with different approaches (though not Devin), I feel like this tracks pretty well. The problem is that Devin and other "agentic" approaches take on more than they can handle. The best AI coders are positioned as tools for developers, rather than replacements for them. Github Copilot is "a better tab complete". Sure, it's a neat de…
That's whatever model you chose to use with it. Aider can use any.l model you like.
This feels a bit like AI image generation in 2022. The fact that it works at all is pretty mindblowing, and sometimes it produces something really good, but most of the time there are obvious mistakes, errors, etc. Of course, it only took a couple more years to get photorealistic image outputs. A lot of commenters here seem very quick to write off Devin / similar ideas permanently. But I'd guess in a few years the pr…
"Photorealistic" is a pretty subjective judgement, whereas "does this code produce the correct outputs" is an objective judgement. A blurry background character with three arms might not impact one's view of a "photorealistic" image, but a minor utility function returning the wrong thing will break a whole program.
Earlier quoted context omitted.
And when working with people it's fairly easy to intervene and improve when needed. I think the current working model with LLMs is definitely suboptimal when we cannot confine their solution space AND where they should apply a solution precisely, and timely.
It’s also often possible to know what a human will be bad at before they start. This allows you to delegate tasks better or vary the level of pre-work you do before getting started. This is pretty unpredictable with LLMs still.
Earlier quoted context omitted.
There's often a lot of small fixes that not time efficient to do, but a solution is not much code and is quick to verify. If the cost is small to setting a coding agent (e.g. aider) on a task, seeing if it reaches a quick solution, and just aborting if it spins out, you can solve a subset of these types of issues very quickly, instead of leaving them in issue tracking to grow stale. That lets you up the polish on you…
Thanks - this all makes sense - I still don't feel like this would constitute a massive productivity boost in most cases, since it's not fixing time consuming major issues. But I can see how it's nice to have.
Merge conflicts are probably the biggest one for me. I put up a PR and move onto a new task. Someone approves, but now there are conflicts. I could switch off my task, spend 5-10 min remembering the intent of this PR and fixing the issues. Or I could just say "@openhands fix the merge conflicts" and move back to my new task.
Earlier quoted context omitted.
Is that true? I'd like to think my commits are less burdensome to review than a fresh out of boot camp junior dev especially if all that's being done is fixing linter issues. Perhaps there's a small benefit, but doesn't seem like a major productivity boost.
A junior dev is not a good approximation of the strengths and weaknesses of these models.
I think this is why people struggle so much with agents--they see the agent perform magic, then assume it can be trusted with a larger task, where it completely falls down.
As someone who uses AI coding tools daily and has done a fair amount of experimentation with different approaches (though not Devin), I feel like this tracks pretty well. The problem is that Devin and other "agentic" approaches take on more than they can handle. The best AI coders are positioned as tools for developers, rather than replacements for them. Github Copilot is "a better tab complete". Sure, it's a neat de…
> The best AI coders are positioned as tools for developers, rather than replacements for them. I agree with this. However, we must not delude ourselves and understand that corporate is pushing for replacement . So there will be a big push to improve on tools like Devin. This is not a conspiracy theory, in many companies (my wife's, for example) they are openly stating this: we are going to reduce (aka "lay off") the…
Yeah it's maddening.
The cope is bizarre too: "writing code is the least important part of the job"
Ok then why does nearly every company make people write code for interviews or do take home programming projects?
Why do people list programming languages on their resumes if it's "least important"?
Also bizarre to see people cheering on their replacements as they use all this stuff.