Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Thoughts on a month with Devin

answer.ai

Post reply on HN
Print view View original

51–60 of 194 posts

Re: Thoughts on a month with Devin

#51

Post by Xmd5a »

freddref wrote: post #49
noodletheworld wrote: post #17

Those “how I feel about Devin after using it” comments at the bottom are damning, when you compare them to the user testimonials of people using cursor. Seems to me that agents just aren’t the answer people want them to be, just a hype wave obscuring real progress in other areas (eg. MCST) because they’re easy to implement. …but really, if things are easy to implement, at this point, you have to ask why they haven’t…

How is Devin different from cursor? I recently used cursor and it has felt very capable in implementing tasks across files. I get that cursor is an IDE but it's ai functionality feels very agentic.. where do you draw the line?

I had to look up MCST: it means Model-Centric Software Tools, as opposed to autonomous agents.

Devin is closer to a long-running process that you can interact with as it is processing tasks, whereas Cursor is closer to a function call: once you've made the call, the only think you can do is wait for the result.

Re: Thoughts on a month with Devin

#52

Post by torginus »

ipnon wrote: post #5

Now is the time for us to hold seemingly contradictory propositions: A child born today will live to see 99% of all computer code written by artificial intelligence, but the current AI boom is massively overcapitalized.

I'd argue that software is being written (either by humans or AI) in an order that it progressively adds less marginal value (if we define value in the capitalistic sense).

Most of the value that software will ever create has already been created.

The only truly valuable missing things are stuff whose value is not easy to translate to capitalists, or need some visionary work.

Re: Thoughts on a month with Devin

#53

Post by timabdulla »

paradite wrote: post #35

I also wrote my first impressions on Devin, more focused on the user experience and analysis of its capabilities (with lots of screenshots): https://thegroundtruth.substack.com/p/devin-first-impression...

Your take seems much more positive than theirs. What do you think the key differences are between your experience and the one here?

Re: Thoughts on a month with Devin

#55

Post by nejsjsjsbsb »

coffeebeqn wrote: post #8
davedx wrote: post #6

One thing that surprised me a little is that there doesn't seem to be an "ask for help" escape hatch in it - it would work away for literally days on a task where any human would admit they were stuck? One of the more important features of agents is supposedly that they can stop and ask for human input when necessary. It seems it does do this for "hard stops" - like when it needed a human to setup API keys in their c…

LLMs can create infinite worlds in the error message it’s receiving. It probably needs some outside signal to stop and re-assess. I don’t think LLMs have any ability to reason if they’re lost in their own world on their own. They’ll just keep creating new less and less coherent context for themselves

They need impatience!

Re: Thoughts on a month with Devin

#56

Post by viraptor »

halfmatthalfcat wrote: post #31
ipnon wrote: post #15

Earlier quoted context omitted.

My first job I spent so much time reading Python docs, and the ancient art of Stack Overflow spelunking. But I could intuitively explain a solution in seconds because of my CS background. I used to encounter a certain kind of programmer often, who did not understand algorithms well but had many years of experience with a language like Ruby, and thus was faster in completing tasks because they didn't need to do the re…

I disagree. If anything, CS degrees have proven time and time again they aren't translatable into software development (which is why there's an entire degree field called Software Engineering emerging). If anything, my gut says that the CS concepts are very easy for LLMs to recall and will be the first things replaced (if ever) by AI. Software engineer{ing,s} (project construction, integrations, scaling, organization…

> the CS concepts are very easy for LLMs to recall

They're easy to recall, but you have to know what to recall in the first place. Or even know enough of the territory to realise there's something to recall. Without enough background, you'll get a whole set of amazing tools that you have no idea what to do with.

For example, you may be able to write a long description of your problem with some ideas how to steer the AI to give you possible solutions. And the AI may figure out what the problem is and that the hyperloglog is something that could be useful to you. And you may have the awesome programming skills to implement that. But that's a lot of maybes. It would be much faster/easier if you knew about hyperloglog ahead of time and just asked for the implementation or library recommendation.

Or even if you don't know about the actual solution, you'd have enough of CS vocabulary to ask: "how do I get a fast, approximate distinct count from a multiset". It would take a long imprecise description to get the same thing for a coder with no theory background.

Re: Thoughts on a month with Devin

#57

Post by epolanski »

huijzer wrote: post #32

I’ve used Cursor a lot and the conclusion doesn’t surprise me. I feel like I’m the one *forcing* the system in a certain direction and sometimes an LLM gives a small snippet of useful code. Sometimes it goes in the wrong direction and I have to abort the suggestion and force it into another direction. For me, the main benefit is having a typing assistant which can save me from typing one line here and there. Especial…

I'll add few things at which Cursor with Claude is better than us (at least in time/effort):

- explaining code. Enter some legacy part of your code nobody understands, LLMs aren't limited to keeping few things in memory like us. Even if the code is very obfuscated and poorly written it can understand what it does and the purpose and suggest refactors to make it understandable

- explaining and fixing bugs. Just the other day Antirez posted a bug of him debugging a Redis segfault on some C code providing context and stack trace. This might be a hit or miss at times, but more often than not it saves you hours

- writing tests. It often comes up with many more examples and edge cases than I thought of. If it doesn't, you can always ask it to.

In any case I want to stress that LLMs are only as good as your data and prompts. They lack the nuance of understanding lots of context, yet I see people talking to them like humans that understand the business, best practices and others.

Re: Thoughts on a month with Devin

#59

Post by thegeomaster »

DaveMcMartin wrote: post #54

This only reinforces my bias against AI agents. At this point, they are mostly just hype. I believe that for AI to replace a junior, we would need to achieve at least near-AGI, and we are far from that.

Exactly. LLMs are gullible. They will believe anything you tell them, including incorrect things they have told themselves. This amplifies errors greatly, because they don't have the capacity to step back and try a different approach, or introspect why they failed. They need actual guidance from somebody with much common sense; if let loose in the world, they mostly just spin around in circles because they don't have this executive intelligence.

Re: Thoughts on a month with Devin

#60

Post by noodletheworld »

Xmd5a wrote: post #51
freddref wrote: post #49

Earlier quoted context omitted.

How is Devin different from cursor? I recently used cursor and it has felt very capable in implementing tasks across files. I get that cursor is an IDE but it's ai functionality feels very agentic.. where do you draw the line?

I had to look up MCST: it means Model-Centric Software Tools, as opposed to autonomous agents. Devin is closer to a long-running process that you can interact with as it is processing tasks, whereas Cursor is closer to a function call: once you've made the call, the only think you can do is wait for the result.

It stands for Monte Carlo search tree.

Ie. Better outputs from models, not external tooling and prompt engineering.

https://github.com/zz1358m/MCTS-AHD-master

Post reply on HN