Copilot regurgitating Quake code, including sweary comments • Hacker News Board

Re: Copilot regurgitating Quake code, including sweary comments

#651

Post by summerlight » Sat, Jul 03, 2021, 7:23 PM UTC

Noumenon72 wrote: post #551

summerlight wrote: post #544

Earlier quoted context omitted.

Google (and most of other big techs I guess?) also explicitly prohibit employees from use of stack overflow code snippets.

I tried Googling this and couldn't find it. I also don't want to believe it because it seems like the world suddenly turned into an apocalyptic hellscape with no place for developers like me. Do you have a source?

First, I work at Google and its onboard training explicitly mentions Stack Overflow as a forbidden example due to CC-BY-SA license (SA is the problematic part). The following link is the official reference.

https://opensource.google/docs/thirdparty/licenses/#restrict...

Re: Copilot regurgitating Quake code, including sweary comments

#652

Post by computerex » Sat, Jul 03, 2021, 7:26 PM UTC

YeGoblynQueenne wrote: post #641

computerex wrote: post #563

Earlier quoted context omitted.

I think it’s pretty clear that program synthesis good enough to replace programmers requires AGI. This solely text based approach is simply “easy” to do, and that’s why we see it. I think it’s cool and results are intriguing but the approach is fundamentally weak and IMO breakthroughs are needed to truly solve the problem of program synthesis.

There's a few decades worth of work on program synthesis and it works very well. You don't need AGI. You need either a) a complete specification of the target program in a formal language (other than the target language) or b) an incomplete specification in the form of positive and negative examples of the inputs and outputs of the target program, and maybe some form of extra inductive bias to direct the search for a…

I said program synthesis good enough to replace programmers requires AGI. Program synthesis based off of informal specifications in natural language. Not talking about highly constrained environments with formal specs.

I am not belittling the work going in this space, and I’m sure for highly constrained and narrow use cases a lot can be done even now. But I believe solving the general problem of program synthesis based on informal spec requires AGI. I am hardly the only one who thinks this.

Re: Copilot regurgitating Quake code, including sweary comments

#653

Post by ec109685 » Sat, Jul 03, 2021, 7:37 PM UTC

Retric wrote: post #646

Noumenon72 wrote: post #629

Earlier quoted context omitted.

I'm somewhere between 45-75 wpm. But Ctrl+C Ctrl+V can type 300wpm! Typing when you could paste is like having that Github Copilot put the right sentence right in front of you and you decide to type over it instead. Not only does it feel like wasted and robotic effort, typing everything leads to RSI. I'm not sure why people disagree. Another symptom is that I insist on aliases for everything while others type out all…

You need to highly the correct test first and move the curser to the correct location to paste text. I bet you can type 123123 several times faster than you can highlight that text in this comment and past it into a reply.

Double click to select a word is fast, and then you are in per word selection mode.

Re: Copilot regurgitating Quake code, including sweary comments

#654

Post by nitrogen » Sat, Jul 03, 2021, 7:44 PM UTC

shados wrote: post #543

nitrogen wrote: post #470

Earlier quoted context omitted.

orders of magnitude slower than proper auto complete Having worked extensively with verbose but autocomplete-able languages like Java, compact dynamic languages like Ruby, and a variety of others including C, Scala, and Kotlin, I've come to the conclusion that, for me, autocomplete is a crutch and I develop deeper understanding and greater capabilities when I go to the docs. IDE+Java encourages sprawl, which just fur…

That's totally cool but the grandparent was talking about remembering shit they already knew. Not everyone has a fantastic memory, and remember the arguments are A then B or B then A doesn't deepen your understanding of a language. Most of the time the autocomplete and the official doc use the exact same source anyway, formatted the same way, with the same info. But if it works for you, more power to you!

remember the arguments are A then B or B then A doesn't deepen your understanding

What I meant is that you will coincidentally learn new things by going to the docs for old/simple things. In addition to remembering that method ordering, you might learn about a new method that simplifies your task.

Re: Copilot regurgitating Quake code, including sweary comments

#655

Post by Retric » Sat, Jul 03, 2021, 9:06 PM UTC

ec109685 wrote: post #653

Retric wrote: post #646

Earlier quoted context omitted.

You need to highly the correct test first and move the curser to the correct location to paste text. I bet you can type 123123 several times faster than you can highlight that text in this comment and past it into a reply.

Double click to select a word is fast, and then you are in per word selection mode.

Sure, move mouse to text, double click, ctrl-c, ctrl-v it’s still slower than touch typing one word.

Re: Copilot regurgitating Quake code, including sweary comments

#656

Post by YeGoblynQueenne » Sat, Jul 03, 2021, 9:30 PM UTC

computerex wrote: post #652

YeGoblynQueenne wrote: post #641

Earlier quoted context omitted.

There's a few decades worth of work on program synthesis and it works very well. You don't need AGI. You need either a) a complete specification of the target program in a formal language (other than the target language) or b) an incomplete specification in the form of positive and negative examples of the inputs and outputs of the target program, and maybe some form of extra inductive bias to direct the search for a…

I said program synthesis good enough to replace programmers requires AGI. Program synthesis based off of informal specifications in natural language. Not talking about highly constrained environments with formal specs. I am not belittling the work going in this space, and I’m sure for highly constrained and narrow use cases a lot can be done even now. But I believe solving the general problem of program synthesis bas…

>> I am not belittling the work going in this space, and I’m sure for highly constrained and narrow use cases a lot can be done even now.

No. Program synthesis approaches work very well for a broad array of problems, not for "highly constrained and narrow use cases"- that is a misconception of the kind that results from lack of familiarity with modern program synthesis.

Here's a good recent review of the field:

https://www.microsoft.com/en-us/research/wp-content/uploads/...

Sumit Gulwani, that I mentioned in my previous comment, is an author. To clarify, I'm not in any way affiliated with him or his collaborators. I'm actually from a rival camp, if you will, but the paper I link to is a very good summary of the state of the art. It should help you if you wish to understand where program synthesis is at.

>> I said program synthesis good enough to replace programmers requires AGI. Program synthesis based off of informal specifications in natural language.

Program synthesis from natural language is hard to make work because it's difficult to translate natural language specifications to specifications that a program synthesiser can use. But that is a limitation of current natural language analysis, specifically natural language understanding, approaches - not a limitation of program syhtesis approaches.

I think you equate formal specifications, or specification by example, with "narrow use cases". There's no connection between the two.

Re: Copilot regurgitating Quake code, including sweary comments

#657

Post by computerex » Sat, Jul 03, 2021, 9:39 PM UTC

YeGoblynQueenne wrote: post #656

computerex wrote: post #652

Earlier quoted context omitted.

I said program synthesis good enough to replace programmers requires AGI. Program synthesis based off of informal specifications in natural language. Not talking about highly constrained environments with formal specs. I am not belittling the work going in this space, and I’m sure for highly constrained and narrow use cases a lot can be done even now. But I believe solving the general problem of program synthesis bas…

>> I am not belittling the work going in this space, and I’m sure for highly constrained and narrow use cases a lot can be done even now. No. Program synthesis approaches work very well for a broad array of problems, not for "highly constrained and narrow use cases"- that is a misconception of the kind that results from lack of familiarity with modern program synthesis. Here's a good recent review of the field: https…

If program synthesis is as far advanced as you say it is, how come I make six figures doing something that you seem to be arguing can be totally automated?

The reality seems to disagree with your statements. Program synthesis is as of right now limited to academic research and highly narrow use cases. If the opposite was true, I’d be out of a job.

I think copilot is probably the first product of its type that might make its way into the hands of users en masse.

Edit:

Btw I was referring to program synthesis based off informal natural language spec. Spec inference is part of the synthesis pipeline, I think it’s not fair to just ignore that problem.

Re: Copilot regurgitating Quake code, including sweary comments

#658

Post by Gaelan » Sat, Jul 03, 2021, 11:43 PM UTC

wrs wrote: post #505

comex wrote: post #487

Earlier quoted context omitted.

Except that CC-BY-SA is not a permissive license; the SA part is a form of copyleft. It's just that nobody enforces it. From the text [1]: - "[I]f You Share Adapted Material You produce [..] The Adapter’s License You apply must be a Creative Commons license with the same License Elements, this version or later, or a BY-SA Compatible License." - "Adapted Material means material [..] that is derived from or based upon…

Yes. In a past life, after researching the situation, we had to find and remove all the code copied from Stack Overflow into our codebase. I can’t fathom why SO won’t fix the license. What makes it even worse is if you try to do the right thing by crediting SO (the BY part) you’re putting a red flag in the code that you should have known you have to share your code (the SA part).

> I can’t fathom why SO won’t fix the license.

They tried to relicense code snippets to MIT a while back, it was a big mess.

Re: Copilot regurgitating Quake code, including sweary comments

#659

Post by abeppu » Sun, Jul 04, 2021, 3:19 AM UTC

computerex wrote: post #652

YeGoblynQueenne wrote: post #641

Earlier quoted context omitted.

There's a few decades worth of work on program synthesis and it works very well. You don't need AGI. You need either a) a complete specification of the target program in a formal language (other than the target language) or b) an incomplete specification in the form of positive and negative examples of the inputs and outputs of the target program, and maybe some form of extra inductive bias to direct the search for a…

I said program synthesis good enough to replace programmers requires AGI. Program synthesis based off of informal specifications in natural language. Not talking about highly constrained environments with formal specs. I am not belittling the work going in this space, and I’m sure for highly constrained and narrow use cases a lot can be done even now. But I believe solving the general problem of program synthesis bas…

I think you're holding text-based approaches and synthesis based approaches to radically different expectations. Copilot isn't approaching replacing programmers; presumably a programmer is invoking it, deciding what to keep or change, etc, i.e. generating parts of programs under the guidance of a human programmer. Synthesis can work at the level of providing an expression or a helper function, as a useful tool under the guidance of a programmer.

Copilot suggests some code snippets, and not necessarily good ones. To be dismissive of another approach to generate parts of programs because they cannot replace programmers is like saying that belt-drive bikes aren't worth considering over chains because a belt-drive bike isn't a replacement for a Learjet.

Re: Copilot regurgitating Quake code, including sweary comments

#660

Post by hermitdev » Sun, Jul 04, 2021, 6:23 AM UTC

willis936 wrote: post #614

rovr138 wrote: post #554

Earlier quoted context omitted.

What guarantees it’s intact?

It doesn't need to be. If the license isn't positively exactly permissive then you can't use it.

Can you even trust that the License in a random repo is accurate and expresses the actual copyright of all the contained code?

I guess my point is, you can't be positive that even if you're following the license in a repo you forked that the repo owner hasn't already violated someone else's license, and now transitively, so have you.