Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Copilot regurgitating Quake code, including sweary comments

twitter.com

Post reply on HN
Print view View original

641–650 of 672 posts

Re: Copilot regurgitating Quake code, including sweary comments

#641

Post by YeGoblynQueenne »

computerex wrote: post #563
abeppu wrote: post #417

I may be over-reading, but I think this kind of example not only demonstrates the pragmatic legal issues, but also the fundamental weaknesses of a solely text-oriented approach to suggesting code. It doesn't really seem to have a representation of the problem being solved, or the relationship between things it generates and such a goal. This is not surprising in a tool which claims to work at least a little for almos…

I think it’s pretty clear that program synthesis good enough to replace programmers requires AGI. This solely text based approach is simply “easy” to do, and that’s why we see it. I think it’s cool and results are intriguing but the approach is fundamentally weak and IMO breakthroughs are needed to truly solve the problem of program synthesis.

There's a few decades worth of work on program synthesis and it works very well. You don't need AGI.

You need either a) a complete specification of the target program in a formal language (other than the target language) or b) an incomplete specification in the form of positive and negative examples of the inputs and outputs of the target program, and maybe some form of extra inductive bias to direct the search for a correct program [edit: the latter setting is more often known as program induction].

In the last few years the biggest, splashiest result in program synthesis was the work behind FlashFill, from Gulwani et al: one-shot program learning, and that's one shot, from a single example, not with a model pretrained on millions of examples. It works with lots of hand-crafted DSLs that try to capture the most common use-cases, a kind of programming common sense that, e.g. tells the synthesiser that if the input is "Mr. John Smith" and the output is "Mr" then if the input is "Ms Jane Brown" the output should be "Ms". It works really, really well but you didn't hear about it because it's not deep learning and so it's not as overhyped.

Copilot tries to circumvent the need for "programming common sense" by combining the spectacular ability of neural nets to interpolate between their training data with billions of examples of code snippets, in order to overcome their also spectacular inability to extrapolate. Can language models learned with neural nets replace the work of hand-crafting DSLs with the work of collecting and labelling petabytes of data? We'll have to wait and see. There are also many approaches that don't rely on hand-crafted DSLs, and also work really, really well (true one-shot learning of recursive programs without an example of the base case and the synthesis terminates) but those generally only work for uncommon programming languages like Prolog or Haskell, so they're not finding their way to your IDE, or your spreadsheet app, any time soon.

But, no, AGI is not needed for program synthesis. What's really needed I think is more visibility of program synthesis research so programmers like yourself don't think it's such an insurmountable problem that it can only be solved by magickal AGI.

Re: Copilot regurgitating Quake code, including sweary comments

#642

Post by YeGoblynQueenne »

whimsicalism wrote: post #434
abeppu wrote: post #417

I may be over-reading, but I think this kind of example not only demonstrates the pragmatic legal issues, but also the fundamental weaknesses of a solely text-oriented approach to suggesting code. It doesn't really seem to have a representation of the problem being solved, or the relationship between things it generates and such a goal. This is not surprising in a tool which claims to work at least a little for almos…

> fundamental weaknesses of a solely text-oriented approach to suggesting code. I don't think it is clear that such "fundamental weaknesses" exist. A text-based approach can get you incredibly far.

>> I don't think it is clear that such "fundamental weaknesses" exist. A text-based approach can get you incredibly far.

Mnyeah, not really that "incredibly". Remember that neural network models are great at interpolation but crap at extrapolation. So as long as you accept that the code generated by Copilot stays inside a well-defined area of the program space, and that no true novelty can be generated that way, then yes, you can get a lot out of it and I think, once the wrinkles are ironed out, Copilot might be a useful, everyday tool in every IDE (or not; we'll have to wait and see).

But if you need to write a new program that doesn't look like anything anyone else has written before, then Copilot will be your passenger.

How often do you need to do that? I don't know. But there's still many open problems in programming that lots of people would really love to be able to solve. Many of them have to do with efficiency and you can't expect Copilot to know anything about efficiency.

For example, suppose we didn't know of a better sorting algorithm than bubblesort. Copilot would not generate mergesort. Even given examples of divide-and-conquer algorithms, it wouldn't be able to extrapolate to a divide-and-conquer algorithm that gives the same inputs for the same outputs as bubblesort. It wouldn't be able to, because it's trained to reproduce code from examples of code, not to generate programs from examples of their inputs and outputs. Copilot doesn't know anything about programs and their inputs and outputs. It is a language model, not a magickal pink fairy of wish fulfillment and so it doesn't know anything about things it wasn't trained on.

Again, how often do you need to write truly novel code? In the context of professional software development, I think not that often. So if it turns out to be a good boilerplate generator, Copilot can go a long way. As long as you don't ask it to generate something else than boilerplate.

There are approaches that work very well in the task of generating programs that they've never seen before from examples of their inputs and outputs, and that don't need to be trained on billions of examples. True one-shot learning of programs (without a model pre-trained on billions of examples) is possible. With current approaches. But those approaches only work for languages like Prolog and Haskell, so don't expect to see those approaches helping you write code in your IDE anytime soon.

Re: Copilot regurgitating Quake code, including sweary comments

#643

Post by pydry »

SahAssar wrote: post #572
pydry wrote: post #557

Earlier quoted context omitted.

Your primary worry was that somebody might read that sentence and believe that the US is gassing immigrants? Seems unlikely.

You seem to think I have some political motive, I don't. I just saw a comment that from my perspective and historical education seemed to equate two things that I regard as different and said that it might be helpful to not conflate those. It seems like you did not intend to conflate them and it is a difference in what you and I read into the term "actual concentration camp". From my perspective this conversation is…

>You seem to think I have some political motive

I'm not really sure what your motive for trying to police my language was. All I know is that the reasons you have given me all seem rather unlikely.

Re: Copilot regurgitating Quake code, including sweary comments

#644

Post by YeGoblynQueenne »

rckstr wrote: post #442
caconym_ wrote: post #405

Earlier quoted context omitted.

> Oftentimes the authors will claim regurgitation is simply not possible Do the Copilot authors claim this? I get that you're suggesting that Copilot may benefit from absolute claims made by the authors of other, similar systems (or their proponents), but I also don't think it's reasonable to exclude nuance and the specifics of Copilot from ongoing discussions on that basis. The Copilot authors have publicly acknowle…

They did! In the faq which I can't find anymore they said: >GitHub Copilot is a code synthesizer, not a search engine: the vast majority of the code that it suggests is uniquely generated and has never been seen before. We found that about 0.1% of the time, the suggestion may contain some snippets that are verbatim from the training set.

There's a risk of confirmation bias though, because the search was performed by the developers of the system who are strongly motivated to find no problem with their work.

I have done this to myself many, many times. I look, carefully, at length, for problems with my work until I satisfy myself that there's no obvious problem with it. Then someone else points out the obvious problem I was overlooking. Actually, this has happened often enough and it's painful enough that I learned to really look nowadays.

In the context of the search for "snippets that are verbatim from the training set" there's all sorts of things that can go wrong. The search (a regex search I think?) can be unintentionally made too weak to catch obvious cases. Or too strong, probably. The search for "snippets that are verbatim from the training set" may ignore snippets that are 80% verbatim. Or the code generated during the experiment can be generated in such a way as to only generate verbatim snippets very rarely, contrary to more typical use. And so on and so forth.

There's so many ways to fool oneself when looking for errors in one's own work.

Edit: They explain their search methodology and with only a quick look I gave it, it seems legit, but it was a quick look. The devil is in the details, yes? Maybe people who are really interested in this issue should take a closer look.

Re: Copilot regurgitating Quake code, including sweary comments

#645

Post by abeppu »

computerex wrote: post #563
abeppu wrote: post #417

I may be over-reading, but I think this kind of example not only demonstrates the pragmatic legal issues, but also the fundamental weaknesses of a solely text-oriented approach to suggesting code. It doesn't really seem to have a representation of the problem being solved, or the relationship between things it generates and such a goal. This is not surprising in a tool which claims to work at least a little for almos…

I think it’s pretty clear that program synthesis good enough to replace programmers requires AGI. This solely text based approach is simply “easy” to do, and that’s why we see it. I think it’s cool and results are intriguing but the approach is fundamentally weak and IMO breakthroughs are needed to truly solve the problem of program synthesis.

Also the goal doesn't need to be "to replace programmers". As with copilot, the point of a program synthesis tool can be to assist the programmer. The point of the system in the video linked above is partly that interactively using such a system can aid development. My main point is this can be a lot better in combination with approaches from outside the ML community, which may involve much tighter integration to specific languages, as well as some awareness of a goal for the synthesized portion.

To "replace programmers", an organization would need to have a way of specifying to the system a high level program behavior, and to confirm that an output from the system satisfies that high level behavior. I think for specifications of any complexity, producing and checking them would look like programming just of a different sort.

Re: Copilot regurgitating Quake code, including sweary comments

#646

Post by Retric »

Noumenon72 wrote: post #629
vanous wrote: post #618

Earlier quoted context omitted.

> "Too short to bother copying"? I copy single words of text to avoid typing and typos. I would never type out even a single line of code when I could paste and edit. Very honest suggestion: learn how to touch type. You can still copy if needed, but your typed input will be much faster.

I'm somewhere between 45-75 wpm. But Ctrl+C Ctrl+V can type 300wpm! Typing when you could paste is like having that Github Copilot put the right sentence right in front of you and you decide to type over it instead. Not only does it feel like wasted and robotic effort, typing everything leads to RSI. I'm not sure why people disagree. Another symptom is that I insist on aliases for everything while others type out all…

You need to highly the correct test first and move the curser to the correct location to paste text. I bet you can type 123123 several times faster than you can highlight that text in this comment and past it into a reply.

Re: Copilot regurgitating Quake code, including sweary comments

#647

Post by 0xbadcafebee »

louthy wrote: post #357
0xbadcafebee wrote: post #280

Earlier quoted context omitted.

> programming languages should be an attempt to step up to a higher level of abstraction Adding abstraction buries complexity. If all you do is keep adding more abstractions, you end up with an overcomplicated, inefficient mess. Which is part of why application sizes are so bloated today. People just keep adding layers, as long as they have room for more of them. Everything gets less efficient and definitely not bett…

> Adding abstraction buries complexity. If all you do is keep adding more abstractions, you end up with an overcomplicated, inefficient mess. Which is part of why application sizes are so bloated today. People just keep adding layers, as long as they have room for more of them. Everything gets less efficient and definitely not better. Presumably you're writing code in binary then? This is a non-argument, because ther…

That's an extreme form of simplification. Simplification is not performance optimization at all; it's removing non-essential complexity. You can still have abstraction and layers, monads and features. The thing is not to keep adding them when a refactor makes them redundant.

Like I say, our designs are ancient and lack features; we need to add more stuff to the code. But that will enable us to remove abstractions that were added only because our previous designs were crap.

Re: Copilot regurgitating Quake code, including sweary comments

#648

Post by voakbasda »

root_axis wrote: post #622
rgbrenner wrote: post #284

So this makes it official... this post[0] and the comments on the announcement[1] concerned about licensing issues were absolutely correct... and this product has the possibility of getting you sued if you use it. Unfortunately for GitHub, there's no turning back the clocks. Even if they fix this, everyone that uses it has been put on notice that it copies code verbatim and enables copyright infringement. Worse, ther…

The practical utility will outweigh the legal concerns. Engineers using this are going to be more productive and this is a competitive advantage that companies won't eschew.

If the legal concerns are well-known, then what you are describing might be viewed as criminal negligence (at worst) and or insufficient duty of care (at best). Such engineers should be held fully responsible and accountable for their actions.

Re: Copilot regurgitating Quake code, including sweary comments

#649

Post by jsjohnst »

pizza234 wrote: post #150
gentleman11 wrote: post #94

Earlier quoted context omitted.

> The python example is using floats for currency. Dumb question, but what is the proper way to handle currency? Custom number objects? Strings for any number of decimal places?

This is a complex topic, mainly for two reasons: 1. it works on two layers (storage and code) 2. there is a context to take care of. [Modern] programming languages have decimal/rational data types, which (within limits) are exact. Where this is not possible, and/or it's undesirable for any reason, just use an int and scale it manually (e.g. 1.05 dollars = int 105). However, point 2 is very problematic and important t…

Generally what is done is that “int 1 != $0.01” rather it’s “int 100 = $0.01”, as in the base of the integer is 1/100th a cent. That doesn’t perfectly solve your example case perfectly though admittedly.

Re: Copilot regurgitating Quake code, including sweary comments

#650

Post by edanm »

Kiro wrote: post #555
rgbrenner wrote: post #284

So this makes it official... this post[0] and the comments on the announcement[1] concerned about licensing issues were absolutely correct... and this product has the possibility of getting you sued if you use it. Unfortunately for GitHub, there's no turning back the clocks. Even if they fix this, everyone that uses it has been put on notice that it copies code verbatim and enables copyright infringement. Worse, ther…

No-one cares about this. People have no clue about licenses and just copy-paste whatever. If someone gets access to their code and see all the violations they're screwed anyway.

This is absolutely not true. While some individuals might not care and might not always conform to their companies' policies, most companies have policies, and most employees are aware of and mindful of these policies.

It's absolutely the case that before using certain libraries, most engineers in large corporations will make sure they are allowed to use that library. And if they don't, they are doing their job very badly IMO.

Post reply on HN