Copilot regurgitating Quake code, including sweary comments • Hacker News Board

Re: Copilot regurgitating Quake code, including sweary comments

#161

Post by FemmeAndroid » Fri, Jul 02, 2021, 2:04 PM UTC

aj3 wrote: post #130

GhostVII wrote: post #92

Earlier quoted context omitted.

Sure but if you have to audit every suggestion to see if it violates copyright laws that's not a particularly useful tool.

Depends. If you find useful code on Github, Stack Overflow or anywhere else in the internet, you still need to check whether it is suitable with your licensing or not.

But Copilot won't accurately tell you if it's directly copying code, and if so what the license is. If it provides MIT licensed code that I then need to include, how do I know that? Do I need to search for each set of lines of code it provides on GitHub?

When a person gets code from another source on the internet, they generally know where the code has come from.

Re: Copilot regurgitating Quake code, including sweary comments

#162

Post by josefx » Fri, Jul 02, 2021, 2:05 PM UTC

bob1029 wrote: post #9

This is pretty clearly just a search engine with more parameters. I thought there was something more going on with copilot, but the fact that it is regurgitating arbitrary code comments tells me that there is zero semantic analysis going on with the actual code being pulled in.

They openly claim it is an AI. What about the state of AI currently in use made you think that there was any intelligence behind it?

Re: Copilot regurgitating Quake code, including sweary comments

#163

Post by bidirectional » Fri, Jul 02, 2021, 2:05 PM UTC

gentleman11 wrote: post #94

tyingq wrote: post #30

They have 4 hand picked examples on their homepage: https://copilot.github.com/ One has the issue with form encoding: https://news.ycombinator.com/item?id=27697884 The python example is using floats for currency, in an expense tracking context. The golang one uses a word ("value") for a field name that's been a reserved word since SQL-1999. It will work in popular open source SQL databases, but I believe it would bom…

> The python example is using floats for currency. Dumb question, but what is the proper way to handle currency? Custom number objects? Strings for any number of decimal places?

Every front office finance project I have ever worked on has used floating point, so take the dogma with a grain of salt. It depends entirely on the context.

Re: Copilot regurgitating Quake code, including sweary comments

#164

Post by buu700 » Fri, Jul 02, 2021, 2:05 PM UTC

serf wrote: post #91

onion2k wrote: post #45

Earlier quoted context omitted.

Stuff like Copilot makes programming worse and more difficult, not better and easier. Copilot makes programming worse and more difficult if you're aiming for a specific set of coding values and style that Copilot doesn't generate (yet?). If Copilot generates the sort of code that you would write, and it does for a lot of people, then it's definitely no worse (or better) than copying something from SO. The author of a…

>If Copilot generates the sort of code that you would write, and it does for a lot of people, then it's definitely no worse (or better) than copying something from SO. Disagree. Most SO copy-paste must be integrated into your project -- maybe it expects different inputs, maybe it expects or works with different variables -- whatever, it must be partially modified to work with the existing code-base that you're workin…

I'm not firmly for or against the concept of Copilot, but it's fascinating to me that it will introduce an entirely new class of bugs. Rather than specific mistakes in certain blocks of code and edge case errors in handling certain inputs, now we're going to have lazy/overworked/junior developers getting complacent and committing code they haven't reviewed that isn't even close to their intent. Like you could have a backend method that was supposed to run a database query, but instead it sends the content of an arbitrary variable in a POST request to a third-party API or invokes a shell to run `rm -rf /`.

Re: Copilot regurgitating Quake code, including sweary comments

#165

Post by slumdev » Fri, Jul 02, 2021, 2:06 PM UTC

reader_mode wrote: post #70

louthy wrote: post #5

This is utterly damning. I have already instructed my team that Copilot can never be used for our projects. Compromising the product because of unknowable license demands isn't acceptable in the professional world of software engineering. But if we put the licensing to one side for a moment... 1/ Everything I've seen it generate so far is 'imperative hell'. It is practically a 'boilerplate generator'. That might be u…

>2/ The boilerplate is what bothers me the most (as someone who believes in the declarative approach to software engineering). The future for programming and programming languages should be an attempt to step up to a higher level of abstraction, that has been historically the way we step up to higher levels of productivity. As applications get larger and code-bases grow significantly we need abstraction, not more boi…

Tools like xsd or T4 (in the .NET ecosystem) are great time-savers, but you would never consider directly modifying the code they generate. You would leave the generated code untouched (in case it ever needed to be generated again) and subclass it to make whatever changes you intend.

I think Copilot is so unfortunate because it's not building abstractions and expecting you to override parts of them. It's acting as an army of monkeys banging out Shakespeare on a typewriter. And the code it generates is going to require an army to maintain.

Re: Copilot regurgitating Quake code, including sweary comments

#166

Post by bidirectional » Fri, Jul 02, 2021, 2:08 PM UTC

jacobsenscott wrote: post #154

School-Cotton wrote: post #110

Earlier quoted context omitted.

Depends what you’re doing. In fact it’s not always wrong to use floats for currency. For accounting you should probably use a fixed-precision decimal type.

If someone asks how to handle money the best answer is integers or fixed precision decimals. There may be a valid case for using floats, but if someone asks they shouldn't be using floats. Also I'm hard pressed to come up with a case where floats would work. Can you give an example?

Most things in front office use floats in my experience, e.g. derivative pricing, discounting, even compound interest. None of these things are going to be any better with integers or fixed-precision, but maybe harder to write and slower.

Re: Copilot regurgitating Quake code, including sweary comments

#167

Post by gameswithgo » Fri, Jul 02, 2021, 2:08 PM UTC

tyingq wrote: post #30

They have 4 hand picked examples on their homepage: https://copilot.github.com/ One has the issue with form encoding: https://news.ycombinator.com/item?id=27697884 The python example is using floats for currency, in an expense tracking context. The golang one uses a word ("value") for a field name that's been a reserved word since SQL-1999. It will work in popular open source SQL databases, but I believe it would bom…

post process with some language aware heuristics maybe

Re: Copilot regurgitating Quake code, including sweary comments

#168

Post by josefx » Fri, Jul 02, 2021, 2:09 PM UTC

Jyaif wrote: post #65

gok wrote: post #34

Quake and GitHub are both owned by Microsoft now, perhaps we can assume this is relicense?

Wow, Quake is owned by Microsoft. This is mind blowing, and a little sad.

It belongs to id software -> ZeniMax -> Xbox Game Studios -> Microsoft.

Re: Copilot regurgitating Quake code, including sweary comments

#169

Post by slumdev » Fri, Jul 02, 2021, 2:10 PM UTC

vmception wrote: post #76

mimixco wrote: post #16

Earlier quoted context omitted.

Awesome summary and thanks for trying it for the rest of us! Copilot sounded terrible in the press release. The idea that a computer is going to pick the right code for you (from comments, no less) is really just completely nuts. The belief that it could be better than human-picked code is really way off. You bring up a really important point. When you use a tool like Copilot (or copypasta of any kind), you are intro…

Hm odd takes here. It's really weird for software engineers to judge something by its current state and not by its potential state. To me, it's clearly solvable by Copilot filtering the input code by that repository's license. It should only be certain open source licenses, maybe even user-choosable, or code-creators can optionally sublicense their code to Copilot in a very permissable way. Secondly, a way for the cr…

> It's really weird for software engineers to judge something by its current state and not by its potential state.

No, we're not afraid of Copilot replacing us. The thought is ridiculous, anyway. If it actually worked, we would be enabled to work in higher abstractions. We'd end up in even higher demand because the output of a single engineer would be so great that even small businesses would be able to afford us.

Yes, we are afraid of Copilot making the entire industry worse, the same way that "low-code" and "no-code" solutions have enabled generations of novices to produce volumes of garbage that we eventually have to clean up.

Re: Copilot regurgitating Quake code, including sweary comments

#170

Post by dirkt » Fri, Jul 02, 2021, 2:10 PM UTC

bidirectional wrote: post #163

gentleman11 wrote: post #94

Earlier quoted context omitted.

> The python example is using floats for currency. Dumb question, but what is the proper way to handle currency? Custom number objects? Strings for any number of decimal places?

Every front office finance project I have ever worked on has used floating point, so take the dogma with a grain of salt. It depends entirely on the context.

And the all suffer from rounding error problems?

I mean, fixed point and a specific type for currency (which also should include the denomination, while we are at it) are not rocket science. Spreadsheets get that right, at least.