Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Devin: AI Software Engineer

cognition-labs.com

Post reply on HN
Print view View original

201–210 of 604 posts

Re: Devin: AI Software Engineer

#201

Post by meindnoch »

mbwgh wrote: post #179
xcv123 wrote: post #173

Earlier quoted context omitted.

> They produce generic, amateurish copy that reads like it's written by committee. If you were only using GPT 3.5 (free ChatGPT) then your opinion is irrelevant. With GPT-4 you could directly ask it: "rewrite your previous response so that it sounds less generic, less amateurish, and not written by a committee". I'm not even joking. Just provide enough information and tell it what to do. If you don't like the output…

If you haven't actually used GPT-5 yet, your assessment is irrelevant.

But the real game changer will be GPT-6.

Re: Devin: AI Software Engineer

#202

Post by senko »

ThalesX wrote: post #73

As a developer but also product person, I keep trying to use AI to code for me. I keep failing, because of context length, because of shit output from the model, because of lack of any kind of architecture etc etc etc. I'm probably dumb as hell, because I just can't get it to do anything remotely useful, more than helping me with leetcode. Just yesterday I tried to feed it a simple HTML page to extract a selector, I…

In my experiments at Pythagora[0], we've found that sweet spot is technical person who doesn't want to know, doesn't know, or doesn't care about the details, but is still technical enough to be able to guide the AI. Also, it's not either/or, for best effect use human and AI brainpower combined, because what's trivial vs tedious for human and AI is different so actually we can complement each other.

Also, current crop of LLMs are not there yet for large/largish projects. GPT4 is too slow and expensive, while Groq is superfast but open source models are not quite there yet. Claude is somewhere in the middle. I expect somewhere in the next 12 months there's going to be a tipping point where they will be capable, fast, and reliable enough to be in wide use for coding in this style[1].

[0] I have an AI horse in the game with http://pythagora.ai, so yeah I'm biased [1] It already works well for snippet-level cases (eg GitHub copilot or Cursor.sh) where you still have creative control as a human. It's exponentially harder to have the AI be (mostly) in control.

Re: Devin: AI Software Engineer

#203

Post by aster0id »

I have a few years of experience in backend development, and I have realized that LLMs are incredible productivity boosts for generating code only if you know the underlying libraries/frameworks/languages very well. You can then prompt it with very specific instructions and it can go do that. Helps with the typing, but that's pretty much all. I still have to know everything and it can definitely not do everything on autopilot. I would be surprised if this product can do any real work.

Re: Devin: AI Software Engineer

#204

Post by CipherThrowaway »

xcv123 wrote: post #173
CipherThrowaway wrote: post #112

Earlier quoted context omitted.

Ditto. I started out excited about LLMs and eager to use them everywhere, but have become steadily disillusioned as I have tried to apply them to daily tasks, and seen others try and fail in the same way. Honestly, LLMs can't even get language right. They produce generic, amateurish copy that reads like it's written by committee. GPT can't perform to the level of a middle market copywriter or content marketer. I am c…

> They produce generic, amateurish copy that reads like it's written by committee. If you were only using GPT 3.5 (free ChatGPT) then your opinion is irrelevant. With GPT-4 you could directly ask it: "rewrite your previous response so that it sounds less generic, less amateurish, and not written by a committee". I'm not even joking. Just provide enough information and tell it what to do. If you don't like the output…

Yes, I've used GPT-4. The writing sounds better, but it still sucks at writing. Most importantly, it feels like it sucks just as much as GPT-3.5 in some deeply important ways.

If you use GPT-4 day-to-day, you've probably encountered this sense of a capability wall before. The point where additional prompting, tweaking, re-prompting simply doesn't seem to be yielding better results on the task, or it feels like the issue is just being shifted around. Over time, you develop a bit of a mental map of where the strengths and weaknesses are, and factor that into your workflows. That's what writing with LLMs feels like, compared to working with a professional writer.

Most writers have already realized that LLMs can't write in any meaningful way.

Re: Devin: AI Software Engineer

#205

Post by jajko »

burningChrome wrote: post #180
MSFT_Edging wrote: post #67

Humans seek work that provides satisfaction and meaning in their life. For every technological advancement, artisans are the first to be made obsolete. Sure we have landfills full of unworn textiles, the market says its good, but overall, we keep destroying what allows humans to seek meaning. Our governments and society have made it clear, if you don't produce value, you don't deserve dignity. We have outsourced art…

I often ask a similar question of what happens when we, as humans, have offloaded everything to technology, to AI to Robots? What kind of a society will we have then? When you no longer have to think about how to do something, or how to build or repair something, or create something original from your imagination. I shudder to think the direction this is all leading to.

Think about it in cycles. Handcrafted products were frowned upon since you could see imperfections and flaws. Now they are the thing, cheap perfect same mass machine-produced things often look bland and cheap, you pay massive premium over it. No reason this won't repeat in some form again.

True art will be rare and treasured, few artists will be rockstars. Till next fad we can't even see coming again mixes it all up.

But yeah these transitions do make tons of people miserable, losing jobs. Also overall middle class is disappearing, but thats the trend for quite some time. But it always paid off to anticipate what next generation of showels will be.

Re: Devin: AI Software Engineer

#206

Post by siva7 »

eloisant wrote: post #62
siva7 wrote: post #17

Earlier quoted context omitted.

I would have dismissed this thought a year ago but seeing how fast openai is moving, in 5 years those ai assistants will be what nowadays human junior/intermediate devs are.

That's exactly what people were saying of self driving cars 15 years ago. "We're so close, within 5 years we're have full self driving, and in 10 nobody will need a driver's licence!"

We're pretty close now, aren't we from within 10 years now!?

Re: Devin: AI Software Engineer

#207

Post by sesm »

ThalesX wrote: post #73

As a developer but also product person, I keep trying to use AI to code for me. I keep failing, because of context length, because of shit output from the model, because of lack of any kind of architecture etc etc etc. I'm probably dumb as hell, because I just can't get it to do anything remotely useful, more than helping me with leetcode. Just yesterday I tried to feed it a simple HTML page to extract a selector, I…

I use ChatGPT every day and it’s excellent at:

- replacing StackOverflow and library documentation

- library search

- converting between formats and languages

- explaining existing code/queries

- deobfuscating code

- explaining concepts (kinda hit or miss)

- helping you get unstuck when debugging or looking for solution (‘give me possible reasons for …’)

I feel like many of this things require asking the right questions, which assumes certain level of experience. But once you reach this level, it’s an extremely valuable assistant.

Re: Devin: AI Software Engineer

#208

Post by Eager »

mbwgh wrote: post #179
xcv123 wrote: post #173

Earlier quoted context omitted.

> They produce generic, amateurish copy that reads like it's written by committee. If you were only using GPT 3.5 (free ChatGPT) then your opinion is irrelevant. With GPT-4 you could directly ask it: "rewrite your previous response so that it sounds less generic, less amateurish, and not written by a committee". I'm not even joking. Just provide enough information and tell it what to do. If you don't like the output…

If you haven't actually used GPT-5 yet, your assessment is irrelevant.

We already have Claude 3 Opus and it is clear for anyone who has used it that it is way better than GPT-4, especially for coding.

The model names, version numbers or who makes them are irrelevant.

Re: Devin: AI Software Engineer

#209

Post by CipherThrowaway »

burningChrome wrote: post #154
CipherThrowaway wrote: post #112

Earlier quoted context omitted.

Ditto. I started out excited about LLMs and eager to use them everywhere, but have become steadily disillusioned as I have tried to apply them to daily tasks, and seen others try and fail in the same way. Honestly, LLMs can't even get language right. They produce generic, amateurish copy that reads like it's written by committee. GPT can't perform to the level of a middle market copywriter or content marketer. I am c…

>> Honestly, LLMs can't even get language right. They produce generic, amateurish copy that reads like it's written by committee. I've had the same experience as well. I heard tons of people clamoring about the ability for LLM's to write SEO copy for you and how you can churn out web content so much faster now. I tried using it to churn out some very specific blog posts for an aborist client of mine. The results were…

It says a lot about SEO copy that this is one of the areas where LLMs low quality doesn't seem to have impeded adoption. There are a ton of shitty content marketers using LLMs to churn out spam content.

>After my trial, I was really wondering how obvious it was that he was doing that and how his clients thought about him knowing how poorly the stuff these LLM's were putting out.

I feel the same way about this stuff as when devs say they push out LLM code with no refactoring or review. Ah, good luck!

Re: Devin: AI Software Engineer

#210

Post by huimang »

sesm wrote: post #207
ThalesX wrote: post #73

As a developer but also product person, I keep trying to use AI to code for me. I keep failing, because of context length, because of shit output from the model, because of lack of any kind of architecture etc etc etc. I'm probably dumb as hell, because I just can't get it to do anything remotely useful, more than helping me with leetcode. Just yesterday I tried to feed it a simple HTML page to extract a selector, I…

I use ChatGPT every day and it’s excellent at: - replacing StackOverflow and library documentation - library search - converting between formats and languages - explaining existing code/queries - deobfuscating code - explaining concepts (kinda hit or miss) - helping you get unstuck when debugging or looking for solution (‘give me possible reasons for …’) I feel like many of this things require asking the right questi…

I like it for condensing long stack traces and very very simple requests, but it really falters when you try to do anything domain specific.

Library documentation? Yeah, it doesn't really save time when GPT makes up functions and libraries, making me check the docs anyways...

I was initially hopeful but I find it gets in my way for anything not trivial.

Post reply on HN