Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Devin: AI Software Engineer

cognition-labs.com

Post reply on HN
Print view View original

331–340 of 604 posts

Re: Devin: AI Software Engineer

#331

Post by rewgs »

ThalesX wrote: post #73

As a developer but also product person, I keep trying to use AI to code for me. I keep failing, because of context length, because of shit output from the model, because of lack of any kind of architecture etc etc etc. I'm probably dumb as hell, because I just can't get it to do anything remotely useful, more than helping me with leetcode. Just yesterday I tried to feed it a simple HTML page to extract a selector, I…

This is exactly my experience. Furthermore, I've become acutely aware that spending time prompting either a) prevents me from going down rabbit holes, all but denying me the kind of learning that can only really happen during those kinds of sessions, and b) prevents me from "getting my reps in" on stuff that I already know. It stands to reason that my ability to coax actually useful information out of LLMs will atrophy with time.

I'm quite wary of the long-term implications and downstream effects of that occurring at scale. AI is typically presented as "the human's hands are still on the wheel," but in reality I think we're handing the wheel over to the AI -- after all, what else would the endgame be? By definition, the more it can do without requiring human intervention, the "better" it is. Even if replacing people isn't the intention, I fail to see how any other effect could usurp that.

Assuming AI keeps developing as it has been, where will we be in 20 years? 50? Will anyone actually have the knowledge to evaluate the code it produces? Will it even matter?

Perhaps it's because Dune is in the air, but I'm really feeling the whole "in a time of increased technology, human capabilities matter more than ever" thing it portrays.

Re: Devin: AI Software Engineer

#332

Post by xcv123 »

YeGoblynQueenne wrote: post #326
xcv123 wrote: post #286

Earlier quoted context omitted.

Fix your prompt. Just accepting the default style is a rookie mistake. Ask it to "rewrite that in the tone of an English professor" or "rewrite that in the style of a redneck rapper" or "make that sound less like generic ad copy". Get into an argument back and forth with the LLM and tell it the previous response is crap because of XYZ.

Or, you know, spend the half hour that would take writing your stuff yourself.

These models can do something in a second that would take many hours for a human writer.

Re: Devin: AI Software Engineer

#333

Post by mediaman »

ThalesX wrote: post #310
PheonixPharts wrote: post #288

Earlier quoted context omitted.

It's worth pointing out that on their eval set for "issues resolved" they are getting 13.86%. While visually this looks impressive compared to the others, anything that only really works 13.86% of the time, when the verification of the work takes nearly as much time as the work would have anyway, isn't useful. The problem with this entire space is that we have VC hype for work that should ultimately still be being do…

> The problem with this entire space is that we have VC hype for work that should ultimately still be being done in research labs. I also have two crypto-bro friends that are hyping it up without having anything to show for it. Which is why I'm sort of complaining about they hype surrounding it. I agree with your post to a large extent. This is not production ready technology. Maybe tomorrow.

LLMs are quite good at text based tasks such as summarization and extracting entities.

These generally don't require advanced logic or thought, though they can require some moderate reasoning ability to summarize two slightly conflicting text extracts.

Lots of corporate work would be enhanced by better summarization, better information dissemination, and better text extraction. Most of it is pretty boring work, but there's a lot of it.

VC hypes seem to want to mostly focus on fantastical problems, though, which sound impressive at dinner parties but don't actually work well.

If you're a VC, do you want to talk about your investment in a company that finds discrepancies in invoices, or one that self-writes consumer iPhone apps?

Only one of those is actually doable today.

Re: Devin: AI Software Engineer

#334

Post by blibble »

pzo wrote: post #119
noncoml wrote: post #74

Earlier quoted context omitted.

Call me fool but s this hype reminds me the hype about self-driving cars coming in a year back in 2014. AI engineers are coming, but I think we would have long retired before they reach a point where they can replace us.

The difference with self-driving cars hype is that they need to be 99.999% good so pretty much perfect to be useful on road and be incorporated mainstream. AI doing some tasks 90% as good as human is good enough. Self driving cars got massively improved in the last 15 years. I remember 15 years ago when DARPA were doing their first self driving challenge and the current tech we have is like magic comparing to what we…

need a few more 9's than that otherwise everyone would be dead within 50 years

Re: Devin: AI Software Engineer

#335

Post by z7 »

spaceman_2020 wrote: post #230
ThalesX wrote: post #73

As a developer but also product person, I keep trying to use AI to code for me. I keep failing, because of context length, because of shit output from the model, because of lack of any kind of architecture etc etc etc. I'm probably dumb as hell, because I just can't get it to do anything remotely useful, more than helping me with leetcode. Just yesterday I tried to feed it a simple HTML page to extract a selector, I…

As with everything about AI, HN once again shows a remarkable inability to project into the future. This site has honestly been absolutely useless when discussing new technology now. No excitement, no curiosity. Just constantly crapping on anything new and lamenting that a brand new technology is not 100% perfect within a year of launch. Remove "Hacker" from this site's name, because I see none of that spirit here an…

I just think there's a bias involved when some people are emotionally invested in AI not being good.

Re: Devin: AI Software Engineer

#336

Post by brigadier132 »

HakuG wrote: post #236
brigadier132 wrote: post #189

Earlier quoted context omitted.

Claude Opus is working for me. It's not perfect but it definitely handles busy work well enough that it's a net positive. Like I add some new fields to a table and ask it to update all the files that depend on the field and it works after 1 or 2 tries. There is a time saving benefit but there is also an avoiding mental fatigue benefit for busywork.

What are you using on top of Claude Opus that helps it access your file system?

cmd c cmd v, definitely not ideal

Re: Devin: AI Software Engineer

#337

Post by Buttons840 »

MSFT_Edging wrote: post #67

Humans seek work that provides satisfaction and meaning in their life. For every technological advancement, artisans are the first to be made obsolete. Sure we have landfills full of unworn textiles, the market says its good, but overall, we keep destroying what allows humans to seek meaning. Our governments and society have made it clear, if you don't produce value, you don't deserve dignity. We have outsourced art…

Just pick a path, doctor, artist, therapist, any path will do. You'll soon realize you're better than the AI, but nobody will give a shit, they'd rather have the cheap AI knockoff.

Re: Devin: AI Software Engineer

#338

Post by ThalesX »

kjqgqkejbfefn wrote: post #317
ThalesX wrote: post #271

Earlier quoted context omitted.

> replacing StackOverflow and library documentation I find it horrible at replacing library documentation > I feel like many of this things require asking the right questions, which assumes certain level of experience. But once you reach this level, it’s an extremely valuable assistant. I've been using LLM products since incipience. I use them in my daily work life. It's a bit tiring hearing this 'right questions', '…

I recently had an introspective dream revealed to be based on a literal prompt at the end: "Game to learn to talk about It and its player." When I asked GPT to craft a plot from this prompt's title (and the fact it is revealed at the end), it reproduced the dream's outline, down to the final scene: GPT reconstruction: The dream reaches its peak when you meet the "final boss" of the game: an entity that embodies the u…

Look I don't mean to downplay. Or maybe I do. But we're talking about LLM replacing professional problem solvers, software architects, not generating great sounding probability modeled token distributions.

Re: Devin: AI Software Engineer

#339

Post by psygn89 »

ThalesX wrote: post #73

As a developer but also product person, I keep trying to use AI to code for me. I keep failing, because of context length, because of shit output from the model, because of lack of any kind of architecture etc etc etc. I'm probably dumb as hell, because I just can't get it to do anything remotely useful, more than helping me with leetcode. Just yesterday I tried to feed it a simple HTML page to extract a selector, I…

The other day I thought I had the perfect task for AI and to clean up some repetitive parts in my scss and to leverage mixins. It failed terribly and was hallucinating scss features. It seems to struggle in the code visual realm.

Re: Devin: AI Software Engineer

#340

Post by ThalesX »

Eager wrote: post #218
parhamn wrote: post #148

Earlier quoted context omitted.

You really need to try Opus. Try a provider that works across models (one in my bio).

It's incredible how far behind HN of all places is w.r.t. what the current best tech is. So many people talking about GPT-4 here, or even 3.5 when the SOTA has moved way along. Gemini Advanced is also a great model, but for other reasons. That thing really knows a boat load of low level optimization tricks.

> So many people talking about GPT-4 here, or even 3.5 when the SOTA has moved way along.

I'm talking about 4-turbo, 4-turbo preview and self hosted LLama2. What in God's name is not SOTA about this?

Post reply on HN