Live data from Hacker News

Thoughts on a month with Devin

answer.ai

41–50 of 194 posts

Re: Thoughts on a month with Devin

#41
post #5

Now is the time for us to hold seemingly contradictory propositions: A child born today will live to see 99% of all computer code written by artificial intelligence, but the current AI boom is massively overcapitalized.

I don't put much stock in predictions about 100 years into the future.

would you like to buy a flying car?

Re: Thoughts on a month with Devin

#42

What model does Devin use? How would it change if it used o1 or even o3 for times when it gets stuck? IE. Generate the initial code using GPT4o/Claude 3.5, then start testing the code, when it gets stuck, use o1/o3 to help.

Yea this is what I was wondering as well. I have o1 not o1 pro but I am gathering from reddit/youtube o1 pro if used correctly is superior for coding tasks.

Re: Thoughts on a month with Devin

#43
post #9

I think one of the big problems with Devin (and AI agents in general) is that they're only ever as good as they are. Sometimes their intelligence feels magical and they accomplish things within minutes that even mid level or senior software engineers would take a few hours to do. Other times, they make simple mistakes and no matter how much help you give, they run around in circles. A big quality that I value in juni…

And when working with people it's fairly easy to intervene and improve when needed. I think the current working model with LLMs is definitely suboptimal when we cannot confine their solution space AND where they should apply a solution precisely, and timely.

Re: Thoughts on a month with Devin

#44
post #37
post #32

I’ve used Cursor a lot and the conclusion doesn’t surprise me. I feel like I’m the one *forcing* the system in a certain direction and sometimes an LLM gives a small snippet of useful code. Sometimes it goes in the wrong direction and I have to abort the suggestion and force it into another direction. For me, the main benefit is having a typing assistant which can save me from typing one line here and there. Especial…

Weird. I have such a different experience with Cursor. Most changes occur with a quick back and forth about top level choices in chat. Followed with me grabbing appropriate interfaces and files for context so Sonnet doesn't hallucinate API, and then code that I'll glance over and around half the time suggest one or more further changes. It's been successful enough I'm currently thinking of how to adjust best practice…

Interesting. What project are you working on? For me it's writing a library in Rust.

Re: Thoughts on a month with Devin

#45
post #32

I’ve used Cursor a lot and the conclusion doesn’t surprise me. I feel like I’m the one *forcing* the system in a certain direction and sometimes an LLM gives a small snippet of useful code. Sometimes it goes in the wrong direction and I have to abort the suggestion and force it into another direction. For me, the main benefit is having a typing assistant which can save me from typing one line here and there. Especial…

I think the .cursorrules and .cursorignore files might be useful here.

Especially the .cursorrules file, as you can include a brief overview of the project and ground rules for suggestions, which are applied to your chat sessions and Cmd/Ctrl K inline edits.

Re: Thoughts on a month with Devin

#46

Earlier quoted context omitted.

If you correct an LLM based agent coder, you are always right. Often, if you give it advice, it pretends like it understands you, then goes on to do something different from what it said it was going to do. Likewise, it will outright lie to you telling you it did things it didn't do. (In my experience)

So when people say these things are like junior developers, they really mean that they’re like the worst _stereotype_ of junior developers, then?

[deleted]

Re: Thoughts on a month with Devin

#47

Disclosure: Working on a company in the space and have recently been compared to Devin in at least one public talk. Devin has tried to do too much. There is value in producing a solid code artifact that can be handed off for review to other developers in limited capacities like P2s and minor bugs which pile up in business backlogs. Focusing on specific elements of the development loop such as fix bugs, add small feat…

You can get a much higher hit rate with more constrained agents, but unfortunately if it's too constrained it just doesn't excite people as much.

Ex. the Grit agent (my company) is designed to handle larger maintenance tasks. It has a much higher success rate, with It's also way less exciting. People want the flashy tool that can solve "everything."

Re: Thoughts on a month with Devin

#48
post #15
post #9

I think one of the big problems with Devin (and AI agents in general) is that they're only ever as good as they are. Sometimes their intelligence feels magical and they accomplish things within minutes that even mid level or senior software engineers would take a few hours to do. Other times, they make simple mistakes and no matter how much help you give, they run around in circles. A big quality that I value in juni…

My first job I spent so much time reading Python docs, and the ancient art of Stack Overflow spelunking. But I could intuitively explain a solution in seconds because of my CS background. I used to encounter a certain kind of programmer often, who did not understand algorithms well but had many years of experience with a language like Ruby, and thus was faster in completing tasks because they didn't need to do the re…

I've found the opposite true as well.

Re: Thoughts on a month with Devin

#49

Those “how I feel about Devin after using it” comments at the bottom are damning, when you compare them to the user testimonials of people using cursor. Seems to me that agents just aren’t the answer people want them to be, just a hype wave obscuring real progress in other areas (eg. MCST) because they’re easy to implement. …but really, if things are easy to implement, at this point, you have to ask why they haven’t…

How is Devin different from cursor?

I recently used cursor and it has felt very capable in implementing tasks across files. I get that cursor is an IDE but it's ai functionality feels very agentic.. where do you draw the line?

Re: Thoughts on a month with Devin

#50
I'm working on AI assistant in Python notebook. It aims to help with data science tasks. I'm not using it to do a full analysis. It will fail. What I ask is to create a code snippet for my next step in the analysis. Many times I need to manually change the code, but it is fine because LLM speed-up my coding a lot. And it is really fantastic in writing matplotlib code for visualization. I don't remember all matplotlib syntax to change axis labels, add annotations or change style, and LLM really can handle it good, in impressive speed.
Post reply on HN