Copilot regurgitating Quake code, including sweary comments • Hacker News Board

Re: Copilot regurgitating Quake code, including sweary comments

#411

Post by heavyset_go » Fri, Jul 02, 2021, 6:12 PM UTC

the_mitsuhiko wrote: post #375

Thomashuet wrote: post #239

Earlier quoted context omitted.

No I haven't. If you think the other demos are more interesting please link to them. I'm just saying that this demo is biased and that we can't draw any conclusion from it. Actually the author has just confessed optimizing it for entertainment in a sister comment. That doesn't mean that the claim is false but it doesn't show that it is true either.

I think you misunderstood my comment. The same code gets generated if you call the function `float fast_rsqrt` or `float fast_isqrt` for instance. I intentionally wanted it to be looking like `Q_rsqrt` so that people pick up on it quicker.

Thanks for making the video in the OP.

Do you have more examples like this that I can share with those who don't use Twitter, like a repo or blog post?

Re: Copilot regurgitating Quake code, including sweary comments

#412

Post by runeb » Fri, Jul 02, 2021, 6:14 PM UTC

sktrdie wrote: post #395

rgbrenner wrote: post #284

So this makes it official... this post[0] and the comments on the announcement[1] concerned about licensing issues were absolutely correct... and this product has the possibility of getting you sued if you use it. Unfortunately for GitHub, there's no turning back the clocks. Even if they fix this, everyone that uses it has been put on notice that it copies code verbatim and enables copyright infringement. Worse, ther…

If they get rid of licensed stuff it should be ok no? I really want to use this and seems inevitable that we'll need it just as google translate needs all of the books + sites + comments it can get a hold of.

How would they do that?

Re: Copilot regurgitating Quake code, including sweary comments

#413

Post by heavyset_go » Fri, Jul 02, 2021, 6:16 PM UTC

NullPrefix wrote: post #19

bencollier49 wrote: post #15

This does make me wonder if this is susceptible to the same form of trolling as that MS AI got. Commit a load of grossly offensive material to multiple repos, and wait for Copilot to start parroting it. I think they're going to need some human moderation.

Coding with Adolf?

Jojo Rabbit except Adolf is in the cloud and not in a kid's imagination.

Re: Copilot regurgitating Quake code, including sweary comments

#414

Post by nadsat2021 » Fri, Jul 02, 2021, 6:17 PM UTC

"sweary comments"

When I hear phrases like that, I worry more about human intelligence. "I'm a little tea pot, short and stout," said social media.

I watched Kubrik's "A Clockwork Orange" again this week, after a certain amount of fearful anticipation.

When Alex said "eggy weggie", it clicked. It's like Burgess time-traveled to 2021 to document our modern infantilization and antisocialization. He forgot to include the Internet, loss of humor, and emerging AI, but I guess he was overwhelmed by the enormity of the "baddiwad".

Later on, droogs.

Re: Copilot regurgitating Quake code, including sweary comments

#415

Post by GenerocUsername » Fri, Jul 02, 2021, 6:19 PM UTC

raffraffraff wrote: post #399

Closi wrote: post #385

Earlier quoted context omitted.

Ahh, so it's the most pointless interpretation of the phrase "filters to block offensive words", where it is stopping the user from causing offense to the AI rather than the other way around.

Easily offended AI is exactly what the world needs

We have too many easily offended NPC's as is.

Re: Copilot regurgitating Quake code, including sweary comments

#416

Post by username90 » Fri, Jul 02, 2021, 6:19 PM UTC

saynay wrote: post #47

bob1029 wrote: post #9

This is pretty clearly just a search engine with more parameters. I thought there was something more going on with copilot, but the fact that it is regurgitating arbitrary code comments tells me that there is zero semantic analysis going on with the actual code being pulled in.

It's more that the model is so large it is capable of memorizing a lot. This can be seen in other language models like GPT-3 as well. Comments, I suspect, will be more likely to be memorized since the model would be trained to make syntactically correct outputs, and a comment will always be syntactically correct. That would mean there is nothing to 'punish' bad comments.

The model in this case is just a lossy compression of github, and you search that.

Re: Copilot regurgitating Quake code, including sweary comments

#417

Post by abeppu » Fri, Jul 02, 2021, 6:20 PM UTC

I may be over-reading, but I think this kind of example not only demonstrates the pragmatic legal issues, but also the fundamental weaknesses of a solely text-oriented approach to suggesting code. It doesn't really seem to have a representation of the problem being solved, or the relationship between things it generates and such a goal. This is not surprising in a tool which claims to work at least a little for almost all languages (i.e. which isn't built around any firm concept of the language's semantics).

I'd be much more excited by (and less unnerved by) a tool which brought program synthesis into our IDEs, with at least a partial description of intended behavior, especially if searching within larger program spaces could be improved with ML. E.g. here's an academic tool from last year which I would love to see productionized. https://www.youtube.com/watch?v=QF9KtSwtiQQ

Re: Copilot regurgitating Quake code, including sweary comments

#418

Post by pjmlp » Fri, Jul 02, 2021, 6:20 PM UTC

adwn wrote: post #362

pjmlp wrote: post #329

Earlier quoted context omitted.

Rust I agree, Python not really as the language doesn't provide any way to keep invariants.

> Python not really as the language doesn't provide any way to keep invariants Again, how is that relevant? If there's no way to enforce an invariant in custom data types , then there's also no way to enforce invariants in code using built-in data types .

It is surely relevant.

Rust provides the mechanisms to enforce them, while in Python, like all dynamic languages, everything is up for grabs.

Re: Copilot regurgitating Quake code, including sweary comments

#419

Post by adrianN » Fri, Jul 02, 2021, 6:21 PM UTC

duckmysick wrote: post #396

croes wrote: post #79

Earlier quoted context omitted.

Good luck checking every code line for license violations

SaaS idea: code linter, but for licenses.

Extend Fossology: https://www.fossology.org/

Re: Copilot regurgitating Quake code, including sweary comments

#420

Post by fragmede » Fri, Jul 02, 2021, 6:21 PM UTC

eganist wrote: post #384

rgbrenner wrote: post #284

So this makes it official... this post[0] and the comments on the announcement[1] concerned about licensing issues were absolutely correct... and this product has the possibility of getting you sued if you use it. Unfortunately for GitHub, there's no turning back the clocks. Even if they fix this, everyone that uses it has been put on notice that it copies code verbatim and enables copyright infringement. Worse, ther…

Adding to this: I run product security for a large enterprise, and I've already gotten the ball rolling on prohibiting copilot for all the reasons above. It's too big a risk. I'd be shocked if GitHub could remedy the negative impressions minted in the last day or so. Even with other compensating controls around open source management, this flies right under the radar with a c130's worth of adverse consequences.

Do you also block stack overflow and give guidance to never copy code from that website or elsewhere on the Internet? I'm legitimately curious - my org internally officially denounces the copying of stack overflow snippets. Thankfully for my role it's moot as I mostly work with an internal non-public language, for better or worse, and I have no idea how well that's followed elsewhere in the wider company.