Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Coping with Copilot

sigarch.org

Post reply on HN
Print view View original

211–220 of 463 posts

Re: Coping with Copilot

#211

Post by emeryberger »

shakow wrote: post #70

Could you just use another teaching language? Does copilot work as well with e.g. Racket, Raku or Pascal, which all seem like decent CS101 languages to me?

Hi all - article author here.

A reader of my article sent me this note, pointing me to some relevant hot-off-the-presses work (reproduced with permission):

-----

A group recently evaluated the performance of Codex (and another model) on 18 programming languages:

https://nuprl.github.io/MultiPL-E/

The high-order bits are:

1. The latest and greatest Codex is now twice as good on its own benchmark suite than the original version published a year ago.

2. It's just as good on Python as it is on JS, Scala, C++, Swift, TypeScript... and other languages are not too far behind. It's not bad at bash of all things.

Paper here: https://arxiv.org/abs/2208.08227

Re: Coping with Copilot

#213

Post by mannykannot »

Kamq wrote: post #165
mannykannot wrote: post #109

Earlier quoted context omitted.

> They're, for the most part, things we tried to put in libraries. We need some people who can do things like making those libraries. It also seems plausible that the people who have at least some of the knowledge and judgement to do that effectively will, on average, be more effective on more mundane tasks as well. Larry Wall is one who wrote (somewhat drolly) about the virtues of laziness, but there was nothing laz…

> We need some people who can do things like making those libraries. Do we? For things as simple as copilot tends to put out? Why? Do you also believe we need to keep people around that do other automated things? Plowing fields by hand? Hand compiling higher level languages (as the first lisp compiler was bootstrapped)? I mean, keep the information around. Don't go burning textbooks on subjects just because we automa…

I find this a very puzzling reply, and it may be that I misunderstood to what you are referring with the "they're" in "They're, for the most part, things we tried to put in libraries (or more ideally language standard libraries for a lot of things.)" It might refer to "the things Copilot does for you" or alternatively "the things you need to know yourself to be in industry."

The thing is, regardless of which way you meant it, we need some people who can make the sort of libraries we need in part precisely because automation such as Copilot is no substitute (at least not yet.)

This observation does not (and is not intended to) endorse current methods of instruction or hiring; on the contrary, it supports spcebar's view that riding your way to a degree, certificate or entry-level position on the back of Copilot is not doing yourself any favors.

The point about Larry Wall is that we don't get labor-saving tools without someone making an effort.

Re: Coping with Copilot

#214

Post by Brian_K_White »

varjag wrote: post #200
Brian_K_White wrote: post #180

Earlier quoted context omitted.

"you're handing out certificates or diplomas to people who can't code" The diploma is not issued after only the 101 course.

So people who couldn't find a max value in an array will somehow excel at advanced concepts later?

You seem to have missed the point of this thread.

The problem an earlier comment expressed is that the student will sabotage themselves.

What else did you think that means besides exactly what you just said?

They sail through the beginning and splat in the middle, because they didn't have to learn anything in the beginning.

My comment is reponding to one that says the problem is that students get diplomas witbout learning.

I am saying that diplomas and certs are not a problem, not that there is no problem. The problem is what the earlier comment said in the first place.

Actually I'm not even sure that's a problem either in the long term, but it is the immediate problem. It may just be during a transition period while schools have not yet adapted to reality and are still worried about calculators in math classes. But at this moment, it's a problem.

Re: Coping with Copilot

#215

Post by emeryberger »

russfink wrote: post #31
spcebar wrote: post #11

Cheating is getting easier by the day, but the bottom line is that if the student is cheating, they're probably not absorbing the coursework (or if they are cheating and absorbing the coursework, there might be something wrong with the coursework). The most effective policy might not be to try and mechanically prevent cheating, but to explain that, this is information that you're going to need if you actually want to…

Adjunct instructor here. Before I came into the program, cheating was on the rise. When given programming assignments, students would go to Geeks for geeks and download solutions, some of them mostly correct, and submit those for credit. One remedy was to try to make programming assignments harder and harder. Eventually, they got so hard that we were asking people new to computer science to implement a full arbitrary…

I think grading on multiple dimensions makes sense, though scaling is a challenge.

As for analysis write-up: turns out, language models can also explain what the code does really well, in convincing vernacular. They generate reasonable comments.

https://twitter.com/emeryberger/status/1560618825208320001?s...

As an illustrative example, I just typed in a little test into Copilot. I only typed the first line and the text that starts with "a paragraph explaining...", and typed a "#" for each comment. Copilot filled in the rest.

  def compute_sum_recursively(n):
      """A paragraph explaining what this function does and why, and explaining its running time:
      This function computes the sum of the first n natural numbers.
      The running time of this function is O(n).
      """
      # Base case: n = 0
      if n == 0:
          # The sum of the first 0 natural numbers is 0
          return 0
      else:
          # Recursive case: n > 0
          # The sum of the first n natural numbers is n + the sum of the first n-1 natural numbers
          return n + compute_sum_recursively(n-1)

Re: Coping with Copilot

#217

Post by hibbelig »

thewebcount wrote: post #196
wilg wrote: post #175

Earlier quoted context omitted.

I (surprisingly) could not get it to generate a Fibonacci function that skips 5.

As a human, I’m having trouble understanding the problem description. What does it mean to “skip 5”? I’ve always understood the definition of a Fibonacci sequence to be, starting with 2 numbers (usually 1 and 1) the next number in the sequence is the sum of the previous 2. So starting with 1 and 1, we get 2. 2 and 1 is three. 3 and 2 is five. What do we do from here? Do I not add 5 and 3? If not, then what do I add?…

You compute the Fibonacci sequence first. Then you use that as the input for another function that produces the modified sequence by dropping 5.

Re: Coping with Copilot

#218

Post by skybrian »

Guest42 wrote: post #190
sidlls wrote: post #178

Earlier quoted context omitted.

How is it abusive?

It was trained using source code whose licenses don't permit such usage.

This is unclear. I have never seen an open source license that was explicit about this. Seems like a grey area.

It's not even clear how often training machine learning algorithms on code results in copyright violations. CoPilot does have a setting to detect and disallow direct copying, but how well does it work?

This legal uncertainty is enough that I wouldn't advise using it, but maybe people who use it will be fine?

Re: Coping with Copilot

#219

Post by [deleted] »

tazjin wrote: post #29

These examples of producing known algorithms in response to function signatures make it seem to me like Copilot is an intelligent, vendoring "package manager", not something that solves problems by itself. The perceived productivity boost that people get from it could be interpreted as a condemnation of how verbose a lot of coding still is even in known areas. Lots of brain cycles wasted on doing things suboptimally…

[deleted]

Re: Coping with Copilot

#220

Post by gopher_space »

matthewmacleod wrote: post #160
andrepd wrote: post #156

Earlier quoted context omitted.

Is it a keystroke saver if you need to review every line it proposes to you?

Yes. For my part, it's a very obvious boost in exactly that sort of keystroke-saving way. Here's an example I just now tried in the codebase I had open. I literally typed in the following Go in a function: mean := geom.Vec3D{} for Copilot then autocompleted this, me pressing tab to accept each line, to: mean := geom.Vec3D{} for _, detection := range filtered { mean.X += detection.X mean.Y += detection.Y mean.Z += det…

I can picture autogen boilerplate being collected and distributed in versioned "community expansion packs" to popular languages, and I'm not looking for fragmentation like that in my tools. I really don't want my IDE involved like this.

I'd rather see Copilot used to expand existing libraries. Pulling potential additions to your own library off of Copilot would be an interesting twist on the situation. A hacktoberfest-alike based off this would be weird.

Post reply on HN