Copilot regurgitating Quake code, including sweary comments • Hacker News Board

Re: Copilot regurgitating Quake code, including sweary comments

#591

Post by manquer » Sat, Jul 03, 2021, 12:28 AM UTC

computerex wrote: post #563

abeppu wrote: post #417

I may be over-reading, but I think this kind of example not only demonstrates the pragmatic legal issues, but also the fundamental weaknesses of a solely text-oriented approach to suggesting code. It doesn't really seem to have a representation of the problem being solved, or the relationship between things it generates and such a goal. This is not surprising in a tool which claims to work at least a little for almos…

I think it’s pretty clear that program synthesis good enough to replace programmers requires AGI. This solely text based approach is simply “easy” to do, and that’s why we see it. I think it’s cool and results are intriguing but the approach is fundamentally weak and IMO breakthroughs are needed to truly solve the problem of program synthesis.

Maybe slightly less than AGI.

Parsing intent in a programing context is easier than others. Also most the code is written to be parsed for a machine anyway. So with ASTs and all other static and maybe even some dynmaic checks it should be possible.

We already some of it with type detection , intellisense etc

It is hard set of problem with no magic solutions like this with years of development time needed. That approach will not happen commercially, only incrementally in the community.

Re: Copilot regurgitating Quake code, including sweary comments

#592

Post by rvz » Sat, Jul 03, 2021, 12:37 AM UTC

influx wrote: post #20

I get why marketing calls machine learning “AI”. I don’t get why engineers would think this is. Dumb.

Exactly.

The clever OpenAI marketing hype squad on HN and Twitter know that they are re-selling a snake oil contraption. This 'thing' completely needs assistance from a human since it is producing insecure code, code that is also copyrighted and most of the times garbage from other sources, which is again totally dangerous.

Just look at this [0] Do a simple 'typo' in the signature and the whole implementation is wrong.

I have to say that OpenAI, GitHub and Microsoft are very clever in selling this scam to engineers who use the code produced by this contraption as 'safe to use' in their projects; especially since GPT-3 still cannot explain why it is generating the code its generating, or if the code is under a license that is non-commercial or under a restrictive licence.

No thanks and most certainly no deal.

[0] https://twitter.com/leifg/status/1411083360756146177

Re: Copilot regurgitating Quake code, including sweary comments

#593

Post by thesz » Sat, Jul 03, 2021, 12:39 AM UTC

rgbrenner wrote: post #284

So this makes it official... this post[0] and the comments on the announcement[1] concerned about licensing issues were absolutely correct... and this product has the possibility of getting you sued if you use it. Unfortunately for GitHub, there's no turning back the clocks. Even if they fix this, everyone that uses it has been put on notice that it copies code verbatim and enables copyright infringement. Worse, ther…

I think that proper coding assistant should help with not writing code (and I stress that it is "not writing code") - how to rearrange your code base for new requirements, for example.

Code not written does not have defects, does not need support and, as you point it out, is not a liability.

Re: Copilot regurgitating Quake code, including sweary comments

#594

Post by eightails » Sat, Jul 03, 2021, 12:48 AM UTC

krick wrote: post #577

throwaway_egbs wrote: post #474

This reply from @AzureDevOps is bizarre: "We understand. However, the way to report this issues related to Windows 11 is through our Windows Insider even from another device. Thanks in advance." I think I'm gonna give "AI" a few more years. https://twitter.com/AzureDevOps/status/1411018079849619458

Wow. To think of it, nothing in this HN thread, including your link, is truly new and unexpected, but in this context it felt somehow more dystopian than ever. Talking about machines pretending to be humans doing stupid stuff, getting automated responses from machines pretending to be humans, that also are the same kind of stupid stuff... Almost feels like drowning.

We understand. However, the way to report this issues related to im̄̽̚m͚͠i͙̬͈̟̹̳ͨ͆̀ͅn̲͚̻ͩ̐͒ͩ̊è̹̱͖̼̰n̘̯ͥ̿̌͛͌t̳̖̣̻̯̱ͥ̅̿̇͜ ̥̻̺͒ͣ͒͠A͔͔͓ͨÌ̖̲̆͒̐̍ͅ ̝̙̼̤͖͍̆̀ͪdͤͨ͑̈҉̭̖y̤͔̮͚̞̺ͬͦͦ̎ͮ͐́s̤͓̲͓̖̪̊t͎̰̤̩̞̞͇͐̎͂̉̆̚o̱̣̰͇̟̻͎̿͒̋̎p̫̰̮̌͐ͧ͗̔̀ͣi̫̱̩̠̫͔͒̉ͤa̶ͧͦͭͩ is through our Citizen Satisfaction Department, even from another Autonomous Azure® Sub-district. Thanks in advance.

Re: Copilot regurgitating Quake code, including sweary comments

#595

Post by Filligree » Sat, Jul 03, 2021, 12:48 AM UTC

grawprog wrote: post #526

zxzax wrote: post #519

Earlier quoted context omitted.

I don't get it, that seems like standard fare for an R-rated movie? And then it seems like some complained because they decided to start editing it down to a PG-13 movie?

Essentially, from my understanding, there was a data leak they never commented on, they instituted a poorly made content filter without saying anything. The filter frequently has false positives and negatives, someone discovered they trained the game using content the filter was designed to block, meaning the ai itself would frequently output filter triggering stuff, more people found out their private unpublished st…

Also, before and while all this was going on, the quality of the AI's output has been steadily dropping to the point where NovelAI.net now generates what's in many ways better writing.

That's GPT-J-6B, to be clear. A 6-billion-parameter model is producing better output than a 300 billion parameter model, because of what I can only assume to be sheer incompetence on AI Dungeon's part. I've also used the raw GPT-3 API, and it does better at writing than either. In other words: Doing nothing would have been better than whatever they've been doing.

Re: Copilot regurgitating Quake code, including sweary comments

#596

Post by chipotle_coyote » Sat, Jul 03, 2021, 12:52 AM UTC

syshum wrote: post #589

jerf wrote: post #559

Earlier quoted context omitted.

Ask your legal department about that. Sure, engineers don't care about licensing at all, but we are not the only players here.

Are legal departments in the habit of reviewing all code line by line? Seems like that would be cost prohibitive...

Obviously they aren't, but just as obviously, "the legal department didn't review this, therefore it's safe to assume it's legal" would not pass muster with said legal department. :) Kiro's comment ("if someone gets access to their code and sees all the violations they're screwed anyway") is probably technically accurate, even if in practice you're unlikely to get caught. As other people have noted elsewhere in the comments here, the Google v. Oracle case over Java definitely suggests that verbatim copying of just a few lines, even for trivial functions, is enough to get you in trouble if those lines aren't licensed in a way that lets you do that.

Re: Copilot regurgitating Quake code, including sweary comments

#597

Post by Filligree » Sat, Jul 03, 2021, 12:52 AM UTC

caconym_ wrote: post #397

gnulinux wrote: post #300

Earlier quoted context omitted.

It's more like a writer using an autocomplete tool to write the first chapter to their novel.

As someone who gets paid to write code (nominally) and has also written a few novels, I don't agree with this characterization. From what I've seen of Copilot, it's more like having a text editor generate your next sentence or paragraph^[1]. The idea (as I see it) is that you might use it to generate some prose "boilerplate", e.g. environmental descriptions, and hack up the results until you're satisfied. It's conten…

> ^[1] At least one tool like this (using GPT-3) has been posted on HN. At this point in time I wouldn't use it, but I have to admit that it was sort of cool.

Have a poke at novelai.net if you get a chance.

It's... not very smart. It's pretty decent at wordcrafting, though, and as an amateur writer I find it invaluable for busting writer's block. Probably if you spend all day writing fiction you'll find ways around that, but for me the solution has become "Ask the AI to try".

It'll either produce a reasonable continuation, or something I can look at and see why it's wrong. Either is better than a blank page.

Re: Copilot regurgitating Quake code, including sweary comments

#598

Post by astrange » Sat, Jul 03, 2021, 1:01 AM UTC

pydry wrote: post #106

slver wrote: post #87

Earlier quoted context omitted.

> And these are the hand picked examples. This product seems like it needs some more thought. Everyone's self-preservation instincts kicking in to attack Copilot is kinda amusing to watch. Copilot is not supposed to produce excellent code. It's not even supposed to produce final code, period. It produces suggestions to speed you up, and it's on you to weed out stupid shit, which is INEVITABLE. As a side note, Excel a…

It's true I probably wouldnt have laughed quite as loudly if there werent a chorus of smug economists telling us that tools like this are gonna put me out of a job.

Economists don't believe this. It's non-economists who do. Economists know that it's not possible to run out of jobs because demand is infinite.

Re: Copilot regurgitating Quake code, including sweary comments

#599

Post by jeroenhd » Sat, Jul 03, 2021, 1:02 AM UTC

qayxc wrote: post #369

bencollier49 wrote: post #231

Earlier quoted context omitted.

That's bonkers. And the beauty of it is that now someone could realistically do a GDPR Erasure request on the Neural Net. I do hope that they're able to reverse data out.

Since the information is encoded in model weights, I doubt that erasure is even possible. Only post-retrieval filtering would be an option. It only goes to show that intransparent black-box models have no place in the industry. The networks leak information left and right, because it's way too easy to just crawl the web and throw terabytes of unfiltered data at the training process.

If this system includes personal information that cannot be removed, corrected or controlled, it's probably a gross violation of all European and some American privacy laws.

Designing a system that you cannot control does not grant you legal immunity for whatever the system does. As Github operates inside the EU, personal information this system contains MUST be deleteable, correctable and retrievable, or it's simply illegal.

Re: Copilot regurgitating Quake code, including sweary comments

#600

Post by dr_kiszonka » Sat, Jul 03, 2021, 1:03 AM UTC

TheDong wrote: post #533

dr_kiszonka wrote: post #345

Earlier quoted context omitted.

There will be a VSCode extension for that.

It's impossible to automate checking for code license violations. If you and I write the exact same 10 lines of code, we both have independent and valid copyrights to it. Unlike patents, independent derivation of the same code _is_ a defense for copyright. If I write 10 lines of code, publish it as GPL (but don't sign a CLA / am not assigning it to an employer), and then re-use it in an MIT codebase, I can do that be…

I should have added /s to highlight that I was being sarcastic. Sorry.