Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Debunking Devin: "First AI Software Engineer" Upwork Lie Exposed [video]

youtube.com

Post reply on HN
Print view View original

11–20 of 47 posts

Re: Debunking Devin: "First AI Software Engineer" Upwork Lie Exposed [video]

#11

Post by mkumar10 »

the__prestige wrote: post #5
mike_hearn wrote: post #2

An extremely solid and convincing rebuttal. Sad. I wonder what the Devin team will say in response, if anything. Summarizing the video: • Devin is sold as being able to solve arbitrary Upwork tasks. In the video demo the problem it was asked to solve doesn't match the stated requirements of the customer (who asked for setup instructions, not code). • Devin is shown fixing errors in the source of a GitHub repo, but th…

In other words they managed to successfully recreate most people's experience on upwork.

Underrated

Re: Debunking Devin: "First AI Software Engineer" Upwork Lie Exposed [video]

#12

Post by xmprt »

lordswork wrote: post #7

Great video. As a counterpoint to the video author's claims, it's worth pointing out that Devin doesn't have to be anywhere close to as fast as a human software engineer to be useful. Even if it turns an hour task to a day long task, it's still going to cost a fraction of what it would to pay the engineer, so that bar it must meet is quite low.

I've often felt that reading and reviewing bad code is more cognitively challenging than writing my own code. Add onto it the unreliability of not knowing when the code will be delivered (so now there's a chance I'll be blocked if Devin can't finish when I need it to), and I can't see how this would be useful. I'm sure there will come a point when the code that Devin writes is easy to review just like how ChatGPT is able to write quite meaningful English sentences but I worry that it will make things even worse because now you can have correct looking code that's riddled with bugs.

Re: Debunking Devin: "First AI Software Engineer" Upwork Lie Exposed [video]

#13

Post by nikincn »

Faking in demo video is now an essential evil to get virality . Because social media is about virality . All commenting on the video and this thread are now curious to try Devin, to prove it works or to prove it doesnt .

So now if it works faking helped it get virality , more users , more demand for product .

If it doesnt work good enough still it will be good enough for some of the users who discovered it because of the virality .

Only worst case is it is too hopelessly bad or doesnt work at all or tried to get to the moon and got nowhere . Hope the founders are smart enough to not be this bad

Re: Debunking Devin: "First AI Software Engineer" Upwork Lie Exposed [video]

#18

Post by maziyar »

the__prestige wrote: post #5
mike_hearn wrote: post #2

An extremely solid and convincing rebuttal. Sad. I wonder what the Devin team will say in response, if anything. Summarizing the video: • Devin is sold as being able to solve arbitrary Upwork tasks. In the video demo the problem it was asked to solve doesn't match the stated requirements of the customer (who asked for setup instructions, not code). • Devin is shown fixing errors in the source of a GitHub repo, but th…

In other words they managed to successfully recreate most people's experience on upwork.

In other words they managed to fake it until they make! Like most visionaries in silicon valley, lie now, tweet about it, prompt it through fake influencers with their mouth open on YouTube, get that VC money without any due diligence, hire smart people and force them to do it!

Re: Debunking Devin: "First AI Software Engineer" Upwork Lie Exposed [video]

#19

Post by heyitaki »

Picking apart Devin based solely on the demo video while ignoring all of the primary source testimonials on Twitter as to Devin's effectiveness seems somewhat intellectually dishonest... A demo video will of course cherrypick impressive-looking moments, even if they're not really.

Re: Debunking Devin: "First AI Software Engineer" Upwork Lie Exposed [video]

#20

Post by wokwokwok »

heyitaki wrote: post #19

Picking apart Devin based solely on the demo video while ignoring all of the primary source testimonials on Twitter as to Devin's effectiveness seems somewhat intellectually dishonest... A demo video will of course cherrypick impressive-looking moments, even if they're not really.

You can only report on facts.

They failed to provide any examples of facts with regard to Devin.

This is like arguing that it’s not fair to critique people claiming to have made superconductors because “some people said they are really superconductors” but no one can share samples with anyone for some reason.

A reasonable counter argument would be:

> Here is evidence of Devin actually doing things.

How, other than the available evidence was anyone supposed to evaluate Devin?

There is a broad opportunity for the developers to respond to this, but they haven’t.

Why is that?

It is because he’s right.

Regardless of what Devin can do that video was deceptive and misleading. There no two ways about it.

Post reply on HN