Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Devin: AI Software Engineer

cognition-labs.com

Post reply on HN
Print view View original

311–320 of 604 posts

Re: Devin: AI Software Engineer

#311

Post by gitfan86 »

gardenhedge wrote: post #293
pushedx wrote: post #200

Scott Wu! I met Scott at a competitive programming event a few years back. He is one of a very small group of people (going back to 1989) to get a perfect raw score at the IoI, the olympiad for competitive programming. https://stats.ioinformatics.org/people/2686 Glad to see that he's putting his (unbelievable) talents to use. To give you a sense, at the event where I met him, he solved 6 problems equivalent to Leetco…

Sounds like he's talented. Isn't Devin "just" a AI wrapper tool? Devin's play is that it will be the first comprehensive option available but it will soon be eaten by OpenAI, Microsoft, Google and countless others.

Yes, but AGI will first emerge from keeping state between calls to multiple models and assessing how closely they resemble humans intelligence, and using a loop to keep it going and updating the state. Which is what they are basically doing here

Re: Devin: AI Software Engineer

#312

Post by __loam »

gtirloni wrote: post #308
__loam wrote: post #275

Earlier quoted context omitted.

> For me the "plateau of productivity" after the disillusionment has been using LLMs a bit like search engines. Quick standalone summaries, snippets or thoughts. A nice day-to-day productivity boost, but nothing that's going to allow me to work less hard. And it only took one of the most computationally expensive processes ever devised by man.

If you ignore how much energy you're burning while searching for dozens and dozens of articles that may or may not give you the answer you're looking for. I'd say the electricity that LLMs burn is nothing compared to my energy and time in that regard.

Id bet $50 the inference is more expensive

Re: Devin: AI Software Engineer

#313

Post by ThalesX »

cloudking wrote: post #287
ThalesX wrote: post #73

As a developer but also product person, I keep trying to use AI to code for me. I keep failing, because of context length, because of shit output from the model, because of lack of any kind of architecture etc etc etc. I'm probably dumb as hell, because I just can't get it to do anything remotely useful, more than helping me with leetcode. Just yesterday I tried to feed it a simple HTML page to extract a selector, I…

What kind of prompts are you using? You'd be surprised how much better your output is using prompting techniques tailored for your goal. There are research papers that show different techniques (e.g one shot, role playing, think step by step etc) can yield more effective results. From my own anecdotal experience coding with ChatGPT+ for the past year, I find this to be true.

> What kind of prompts are you using?

I hack on them till I get something sort of satisfying.

> You'd be surprised how much better your output is using prompting techniques tailored for your goal.

The biggest problem I encounter is context length, not necessarily the output for small inputs. It starts forgetting very fast, whether it's Claude, GPT+ or other self hosted models I've tried.

Re: Devin: AI Software Engineer

#314

Post by mrguyorama »

Verdex wrote: post #193
hackerlight wrote: post #106

Earlier quoted context omitted.

"If I believed that, I'd be starting a development agency tomorrow." The company making Devin is doing just that. As we can see, it will take some work to perfect.

If it was actually working for anyone, then they would be selling software engineering time at the same but slightly cheaper price as existing software engineering time costs today so they could capture those sweet margins. This is a company spending investor money selling pickaxe hand grips during a gold rush. For real evidence, look for companies selling engineering time much greater than the amount of their total…

It's hilarious. If Devin were any good, they wouldn't be selling access to it to random SWEs, they would be replacing Microsoft, Apple, Google, etc for that sweet sweet trillions of dollars!

Where's the app they built in an afternoon using Devin? Where's the software product that Devin actually built a month ago and was being used by thousands of people?

Their actual business seems to be closer to "Lets milk some of that sweet sweet high income from SWEs with FOMO about AI"

Re: Devin: AI Software Engineer

#315

Post by RyEgswuCsn »

sailingparrot wrote: post #284
RyEgswuCsn wrote: post #279

If you need AI to help you program an algorithm, then you shouldn't be using it because you can't tell if AI's solution is correct. If you can tell if a solution is correct or not --- well, then you don't need to have AI write it for you. I think AI programming can only work when the industry begin to treat "almost working" systems backed by human customer service as acceptable.

> If you can tell if a solution is correct or not --- well, then you don't need to have AI write it for you. Did you just solve P=NP? Many things are trivial to verify, but hard/time consuming to code up. You probably shouldn't rely on this to write critical software, no matter the amount of manual QA you throw at it afterwards, but there is an abundance of non-critical use cases where you can quickly check if a solu…

What I meant to say is that most people can only verify an algorithm is correct if they already know the correct solution.

If they already know the answer then it’s probably more efficient if they write it themselves rather than having AI produce a potentially difficult to verify answer and try to verify it.

Re: Devin: AI Software Engineer

#316

Post by babyshake »

jacob019 wrote: post #295
n4r9 wrote: post #281

Earlier quoted context omitted.

What really annoys me in all these discussions is how no one's tested what happens if they wait until 2050 and try GPT-19.

That's well after the AI meta consciousness understood that it was necessary to destroy all humans to save the planet. GPT-6 was the last of the GPT series.

Perhaps the strangest element of the AI alignment conversation is that what is most aligned with human civilization (at least the most powerful elements of it) and alignment with sustainable life on the planet are at odds, and "destroy humans to save planet" is a concern mostly because it seems to be a somewhat rational conclusion.

Re: Devin: AI Software Engineer

#317

Post by kjqgqkejbfefn »

ThalesX wrote: post #271
sesm wrote: post #207

Earlier quoted context omitted.

I use ChatGPT every day and it’s excellent at: - replacing StackOverflow and library documentation - library search - converting between formats and languages - explaining existing code/queries - deobfuscating code - explaining concepts (kinda hit or miss) - helping you get unstuck when debugging or looking for solution (‘give me possible reasons for …’) I feel like many of this things require asking the right questi…

> replacing StackOverflow and library documentation I find it horrible at replacing library documentation > I feel like many of this things require asking the right questions, which assumes certain level of experience. But once you reach this level, it’s an extremely valuable assistant. I've been using LLM products since incipience. I use them in my daily work life. It's a bit tiring hearing this 'right questions', '…

I recently had an introspective dream revealed to be based on a literal prompt at the end: "Game to learn to talk about It and its player." When I asked GPT to craft a plot from this prompt's title (and the fact it is revealed at the end), it reproduced the dream's outline, down to the final scene:

GPT reconstruction:

The dream reaches its peak when you meet the "final boss" of the game: an entity that embodies the ultimate barrier to communication. To overcome this obstacle, you must synthesize everything you've learned about "it" in the dream and present a coherent vision that is true to yourself. As you articulate your final understanding of "it", the maze dissolves around you, leaving you in front of a giant mirror. In this mirror, you see not just your reflection but also all the characters, passions, and ideas you encountered in the dream. You realize that "it" is actually a reflection of yourself and your ability to understand and share your inner world. The dream ends with the title revealed, "Game to Learn to Communicate about It and Its Player", meaning the whole process was a metaphor for learning to know and communicate your own "it" - your personality, thoughts, and emotions - with others, and that you are both the creator and the discoverer of your own communication game.

My note:

The continuation of the dream corresponds to an abrupt change of scene. I find myself in my bed, in the dim light of my room, facing a mysterious silhouette. As I repeatedly inquire about its identity, I stretch my hands towards its face to feel its features as I cannot clearly see them. Then, a struggle begins, during which I panic, giving the dream a nightmarish turn. Noticing that the dark figure mirrors my movements, I realize it's myself. Suddenly under my duvet and as I struggle to get out, I feel jaws and teeth against the sheets. I call out for my mother, whom I seem to hear downstairs, and that's when my vision fades, and I see the dream's source code displayed behind. It consists of ChatGPT prompts shared on the lime green background of an image-board. At the bottom, I then see the dream's title: "Game to learn how to communicate about It and its player."

Re: Devin: AI Software Engineer

#318

Post by sanderjd »

ThalesX wrote: post #73

As a developer but also product person, I keep trying to use AI to code for me. I keep failing, because of context length, because of shit output from the model, because of lack of any kind of architecture etc etc etc. I'm probably dumb as hell, because I just can't get it to do anything remotely useful, more than helping me with leetcode. Just yesterday I tried to feed it a simple HTML page to extract a selector, I…

Exactly where I'm at! Totally transformative set of tools for me to use to do my day to day work significantly more productively and also a giant distance away from being capable of doing my day to day work.

Re: Devin: AI Software Engineer

#319

Post by YeGoblynQueenne »

>> With our advances in long-term reasoning and planning, Devin can plan and execute complex engineering tasks requiring thousands of decisions.

They'd better have really advanced reasoning and planning capabilities way beyond everything that anyone else knows how to do with LLMs. There's a growing body of literature that leaves no doubt that LLMs can't reason and can't plan.

For a quick summary of some such results see:

https://arxiv.org/pdf/2403.04121.pdf

Post reply on HN