Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Copilot regurgitating Quake code, including sweary comments

twitter.com

Post reply on HN
Print view View original

461–470 of 672 posts

Re: Copilot regurgitating Quake code, including sweary comments

#461

Post by sombremesa »

SahAssar wrote: post #458
dragonwriter wrote: post #450

Earlier quoted context omitted.

Nazi “concentration camps” were not actual concentration camps (a thing which long predates the Nazi camps), they were extermination camps for which “concentration camp” was a minimizing euphemism. US WWII “internment” and “relocation” centers were actual concentration camps (“relocation center” was itself a euphemism, but “internment” referred to a formal legal distinction impacting treaty obligations.)

Sure, but I don't know if I've ever heard anyone use the term "concentration camp" without qualifiers to refer to anything else than the nazi concentration camps (or something equivalent). If someone says that something is "_literally_ a concentration camp" I think that most people will think of ovens and genocide. Perhaps it's a regional thing, but that is how I interpreted it.

It's not so much a regional as a political thing. Want it to sound worse? Use concentration camp. Want it to sound better? Use internment camp (or in some cases, re-education facility).

Re: Copilot regurgitating Quake code, including sweary comments

#463

Post by sombremesa »

caconym_ wrote: post #455
rckstr wrote: post #442

Earlier quoted context omitted.

They did! In the faq which I can't find anymore they said: >GitHub Copilot is a code synthesizer, not a search engine: the vast majority of the code that it suggests is uniquely generated and has never been seen before. We found that about 0.1% of the time, the suggestion may contain some snippets that are verbatim from the training set.

This actually seems like an explicit acknowledgement that regurgitation is possible, and not remotely a claim that it is "simply not possible". It stands to reason that cases where people are intentionally trying to produce regurgitation will strongly overlap with the minority of cases where it actually happens. So I think we are probably suffering from some selection bias in discussions on HN and similar forums--tha…

I think only Github's lawyers would interpret what GP posted the way you did. Looks like weasel wording to make such an interpretation possible, while making customers believe that code is more or less synthesized in realtime. "Snippets" makes one think one or two lines of code, not entire functions and classes.

Re: Copilot regurgitating Quake code, including sweary comments

#464

Post by canadev »

mediaman wrote: post #428
fragmede wrote: post #420

Earlier quoted context omitted.

Do you also block stack overflow and give guidance to never copy code from that website or elsewhere on the Internet? I'm legitimately curious - my org internally officially denounces the copying of stack overflow snippets. Thankfully for my role it's moot as I mostly work with an internal non-public language, for better or worse, and I have no idea how well that's followed elsewhere in the wider company.

Who really copies stack overflow snippets verbatim? It's usually just easier to refer to it for help figuring out the right structure and then adapt it for your own needs. Usually it needs customization for your own application anyway (variables, class instances, etc).

Yeah! I've uh, ... never copied a bit of code into my repo verbatim, right?

yeah right. I wish.

(Not saying every dev does this)

Re: Copilot regurgitating Quake code, including sweary comments

#465

Post by derefr »

Closi wrote: post #385
minimaxir wrote: post #346

Earlier quoted context omitted.

In my testing of Copilot, the content filters only work on input , not output. Attempting to generate text from code containing "genocide" just has Copilot refuse to run. But you can still coerce Copilot to return offensive output given certain innocuous prompts.

Ahh, so it's the most pointless interpretation of the phrase "filters to block offensive words", where it is stopping the user from causing offense to the AI rather than the other way around.

I believe the concept is to stop users from prompting the AI to generate offensive stuff specifically, and then publishing the so-generated stream of offensive stuff as negative PR for GitHub, in the same way the generated stream of offensive stuff coming from Microsoft’s AI was a big PR disaster.

Re: Copilot regurgitating Quake code, including sweary comments

#466

Post by [deleted] »

SahAssar wrote: post #458
dragonwriter wrote: post #450

Earlier quoted context omitted.

Nazi “concentration camps” were not actual concentration camps (a thing which long predates the Nazi camps), they were extermination camps for which “concentration camp” was a minimizing euphemism. US WWII “internment” and “relocation” centers were actual concentration camps (“relocation center” was itself a euphemism, but “internment” referred to a formal legal distinction impacting treaty obligations.)

Sure, but I don't know if I've ever heard anyone use the term "concentration camp" without qualifiers to refer to anything else than the nazi concentration camps (or something equivalent). If someone says that something is "_literally_ a concentration camp" I think that most people will think of ovens and genocide. Perhaps it's a regional thing, but that is how I interpreted it.

[deleted]

Re: Copilot regurgitating Quake code, including sweary comments

#467

Post by abeppu »

whimsicalism wrote: post #434
abeppu wrote: post #417

I may be over-reading, but I think this kind of example not only demonstrates the pragmatic legal issues, but also the fundamental weaknesses of a solely text-oriented approach to suggesting code. It doesn't really seem to have a representation of the problem being solved, or the relationship between things it generates and such a goal. This is not surprising in a tool which claims to work at least a little for almos…

> fundamental weaknesses of a solely text-oriented approach to suggesting code. I don't think it is clear that such "fundamental weaknesses" exist. A text-based approach can get you incredibly far.

I mean, the cases where it tries to assign copyright to another person in a different year highlights that context other than the other text in the file is semantically extremely important, and not considered by this approach. Merely generating text which looks appropriate to the model given surrounding text is ... misguided?

If you think about it, program synthesis is one of the few problems in which the system can have a perfectly faithful model dynamics of the problem domain. It can run any candidate it generates. It can examine the program graph. It can look at what parts of the environment were changed. To leave all that on the table in favor of blurting out text that seems to go with other text is like the toddler who knows that "five" comes after "four", but who cannot yet point to the pile of four candies. You gotta know the referents, not just the symbols. No one wants a half-broken Chinese Room.

Re: Copilot regurgitating Quake code, including sweary comments

#469

Post by croes »

atatatat wrote: post #386
croes wrote: post #360

Earlier quoted context omitted.

Same trust issue

It's people for your lawyers to blame, all the way down! /s

It's the same problem s with self driving cars, you gets sued. The company that provides the service/car or the the programmer/driver? I think the latter.

Re: Copilot regurgitating Quake code, including sweary comments

#470

Post by nitrogen »

shados wrote: post #368
nitrogen wrote: post #347

Earlier quoted context omitted.

It would be nice if there were a way to automate the "remembering what that one function is called and what order the parameters are in" portion of my job. IME the best thing for this is looking at the method listing in the docs for the classes I'm using. E.g. for Ruby, it's usually looking at the methods in Enumerable, Enumerator, Array, or Hash. Or I'll drop a binding.pry into the function, run it, and then type ls…

Even in the 90s that was a solved problem in Visual Basic with autocomplete. That a lot of dev environments "lost" the ability to do it is mind boggling. With that said, doesn't Rubymine let you do that with autocomplete with the prompt giving you all the info you need? (I haven't done Ruby in a long time). Still, having to look up the doc or run the code to figure out how to type it is orders of magnitude slower tha…

orders of magnitude slower than proper auto complete

Having worked extensively with verbose but autocomplete-able languages like Java, compact dynamic languages like Ruby, and a variety of others including C, Scala, and Kotlin, I've come to the conclusion that, for me, autocomplete is a crutch and I develop deeper understanding and greater capabilities when I go to the docs. IDE+Java encourages sprawl, which just further cements the need for an IDE. Vim+Ruby+FZF+ripgrep+REPL encourages me to design code that can be navigated without an IDE, which ultimately results in cleaner designs.

If there's any lag whatsoever in the autocomplete, it breaks my flow state as well. I can maintain flow better when typing out code than when it just pops into being after some hundreds of milliseconds delay. Plus, there's always the chance for serendipity when reading docs. The docs were written by the language creators for a reason. Every dev should be visiting them often.

Post reply on HN