[deleted]
Devin: AI Software Engineer
461–470 of 604 posts
Re: Devin: AI Software Engineer
#462Earlier quoted context omitted.
This is part of the problem with the whole discourse of comparing human writers to LLMs. Superficial things like style and tone aren't the problem, but they are overwhelmingly the focus of these discussions. It's funny to see, because developers are so sensitive about being treated like code monkeys by their non-technical colleagues. But these same devs turn around to treat other professionals as word monkeys, or pix…
>and low information density of the text. Personally, I group all these things under 'style'. Perhaps, i should have used, 'presentation' instead. You've latched on that specific word and gone off. Point is that the post-training of these models, especially GPT from Open ai is doing a lot to how the writing (the default at least) presents long strings of text. Like how GPT-4 is almost compelled to end bouts of fictio…
> You've latched on that specific word and gone off.
No, I haven't. I'm not talking about style, but something deeper. What I'm talking about is something you don't even seem to realize exists in professional writing - which is why you keep thinking I'm misunderstanding you when I am not.
I've worked with professional writers, and nothing in the LLM space even comes close to them. It's not a matter of low quality vs high quality, or benchmarking, or style. It's simply an apples and oranges comparison.
The economics of LLMs for shortform copy will never make sense, because producing the words is the cheapest part of that process. They might become the best way for writers themselves to produce longform copy on the execution side, but they can't replace the writer's ability to work with the client to figure out exactly what they are trying to write, and why, and what a good result even looks like. And no, this isn't a prompting issue, or a UI issue, or a context window length issue, or anything like that.
Elsewhere in this thread someone mentioned how invaluable LLMs are for producing internal business copy. I could easily see these amateur writing tasks being replaced by LLMs. But the implication there isn't that LLMs are any good at writing, but that these tasks don't require good writing to begin with.
Re: Devin: AI Software Engineer
#463[flagged]
Re: Devin: AI Software Engineer
#464[flagged]
Re: Devin: AI Software Engineer
#465[flagged]
Re: Devin: AI Software Engineer
#466As a developer but also product person, I keep trying to use AI to code for me. I keep failing, because of context length, because of shit output from the model, because of lack of any kind of architecture etc etc etc. I'm probably dumb as hell, because I just can't get it to do anything remotely useful, more than helping me with leetcode. Just yesterday I tried to feed it a simple HTML page to extract a selector, I…
> I'm probably dumb as hell, because I just can't get it to do anything remotely useful
Rather, your “problem” is that you’re likely not writing uninteresting cookie cutter boilerplate that everyone can do and has done hundreds of times. The current crop of AI is cool for coding demos, not for solving real relevant problems.
> Just give me a product that works as advertised and I'll throw money your way
The people hyping this crap only care about the second part of that sentence. The first one is an afterthought.
Re: Devin: AI Software Engineer
#467hey , I am a newbie in field of AI , i want to know about the nearby future of AI . after devin I am little ! or rather I should say deeply terrified about the future (5 to 8 years) of software eng. can someone explain ?
Re: Devin: AI Software Engineer
#468There is no way this is going to make it so that "engineers can focus on more interesting problems and engineering teams can strive for more ambitious goals." Instead it will mean that bosses can fire 75-90% of the (very expensive) engineers, with the ones who remain left to prompt the AI and clean up any mistakes/misunderstandings. I guess this is the future. We've coded ourselves out of a job. People are smiling an…
You are vastly overselling current generation AI here. It can do some things -- GitHub Copilot has been useful for people to reduce the boilerplate generation, for example -- but in terms of actual programming which 98% of the time is maintenance (fixing bugs, debugging, adding tests, refactoring, adding features) it's performing mostly bad. It's only good at generating code and maybe "understanding" some of it. Prov…
what do you think about devin? just curious
Re: Devin: AI Software Engineer
#469Earlier quoted context omitted.
I use ChatGPT every day and it’s excellent at: - replacing StackOverflow and library documentation - library search - converting between formats and languages - explaining existing code/queries - deobfuscating code - explaining concepts (kinda hit or miss) - helping you get unstuck when debugging or looking for solution (‘give me possible reasons for …’) I feel like many of this things require asking the right questi…
Things AI is "excellent" at, includes "explaining concepts (kinda hit or miss)". Did you use an AI assistant while making that list?
This looks silly, I admit. I made this correction after reviewing what I’ve written, but should have corrected in 2 places. The list is handwritten, but English is not my native language.
Re: Devin: AI Software Engineer
#470Earlier quoted context omitted.
It's worth pointing out that on their eval set for "issues resolved" they are getting 13.86%. While visually this looks impressive compared to the others, anything that only really works 13.86% of the time, when the verification of the work takes nearly as much time as the work would have anyway, isn't useful. The problem with this entire space is that we have VC hype for work that should ultimately still be being do…
> It's worth pointing out that on their eval set for "issues resolved" they are getting 13.86%. While visually this looks impressive compared to the others, anything that only really works 13.86% of the time, when the verification of the work takes nearly as much time as the work would have anyway, isn't useful. Yeah, I remember speech recognition taking decades to improve, and being more of a novelty - not useful at…
Even now, automatic speech recognition is a big timesaver, but you _need_ a human to look through the transcript to pick out the obviously wrong stuff, let alone the stuff that's wrong buy could be right in context.