CMU Computer Systems: Self-Grading Lab Assignments (2018) • Hacker News Board

Re: CMU Computer Systems: Self-Grading Lab Assignments (2018)

#41

Post by rcxdude » Mon, Apr 29, 2019, 10:27 AM UTC

Fiahil wrote: post #38

albntomat0 wrote: post #14

Earlier quoted context omitted.

> Turnaround time for exams was sub-one-day. What's the current status on accurately grading mistakes? From the exams I remember (Fall 2015), the exams autograded perfect answers well, but gave similar marks to nearly correct, but still wrong answers as complete gibberish. I felt that unless I had the answers exactly correct, it was hard to demonstrate my knowledge of the topic.

When I was studying software engineering, "nearly correct" answers would get you the same grade as "complete gibberish" on purpose. Partially correct programs would have no more value than completely incorrect ones. In retrospect, I think it was crucially important for students to learn the price of their mistakes. Most of my previous classmates are reputable engineers now.

I think this is only fair if the exam takes place under similar circumstances to actual engineering (which is rarely the case).

Re: CMU Computer Systems: Self-Grading Lab Assignments (2018)

#42

Post by westurner » Mon, Apr 29, 2019, 11:16 AM UTC

sk0g wrote: post #40

yjftsjthsd-h wrote: post #33

Earlier quoted context omitted.

What on earth would 200% test coverage mean?

Coverage for all the code that was written, and all the code that has yet to be typed, of course. All thanks to quantum computing :)

⟨100%| + |100%⟩ = 200%!

(Even code with 100% branch coverage may have common weaknesses like those that these (great) labs have students exploit)

Re: CMU Computer Systems: Self-Grading Lab Assignments (2018)

#43

Post by doktrin » Mon, Apr 29, 2019, 12:10 PM UTC

Former 15-213 student as well. This was one of the best CS classes I've ever had the pleasure of taking.

Also worth noting the plagiarism detector actually did its job, at least in some of the most egregious cases that I heard of.

Re: CMU Computer Systems: Self-Grading Lab Assignments (2018)

#44

Post by rogela » Mon, Apr 29, 2019, 1:05 PM UTC

Took the CSAPP course at Peking University. Easily the most valuable course through 4 years...

Re: CMU Computer Systems: Self-Grading Lab Assignments (2018)

#45

Post by lasagnaphil » Mon, Apr 29, 2019, 7:57 PM UTC

PEJOE wrote: post #4

Oh malloc lab.

Oh, good times. By the way, CMU's recitation class ppt for this lab is fucking hilarious: http://www.cs.cmu.edu/afs/cs/academic/class/15213-s08/www/re...

Re: CMU Computer Systems: Self-Grading Lab Assignments (2018)

#46

Post by voidmain » Mon, Apr 29, 2019, 10:30 PM UTC

When I took this class many, many moons ago, the close equivalent of the "cache lab" was graded not on the number of cache misses but on the cache hit ratio. When I pointed the problem with this metric out to a TA he blew me off. So I turned in the original code plus a loop that read the same volatile value a few billion times. It took a long time for the cache simulator to grade, but they dutifully gave me a very high grade (something like 200%) on the lab :-)

Re: CMU Computer Systems: Self-Grading Lab Assignments (2018)

#47

Post by josephg » Tue, Apr 30, 2019, 4:14 PM UTC

techntoke wrote: post #32

josephg wrote: post #23

Earlier quoted context omitted.

I ran an in-class test/competition once where students submitted C code to a web server running on my laptop. My laptop compiled and ran their code - then checked it against a test suite and graded it all live. One student asked about security - and we got into a great discussion about permissions and what havoc he could and couldn’t pull off with the restricted user account. I encouraged him to figure out how to wri…

Ah, how simple it would have been to run in a virtual machine.

How simple it would have been to miss this teaching opportunity!

If my laptop were fully protected from that attack, it wouldn't have been as much fun for my student to wreck my machine in front of everyone. And so he might not have done it, and my class would have missed out on a beautiful demonstration and discussion of practical security engineering.

Re: CMU Computer Systems: Self-Grading Lab Assignments (2018)

#48

Post by josephg » Tue, Apr 30, 2019, 4:16 PM UTC

Cogito wrote: post #28

josephg wrote: post #21

Earlier quoted context omitted.

At UNSW we had a similar system. The marking was a test suite, and each passing test would give you a portion of your grade. The system had a couple of simple smoke tests which were ran on submission time (to catch issues that would make the entire test run fail erroneously). As a TA I spent a fair bit of time fixing trivial issues - eg, sometimes a student would get 3/10 on the auto marker with a basically working s…

When did you TA? I was TA for 1911 (I think?) around 2008-2009 and don't recall any automated grading system. It would have been so nice too, as labs would end up with students trying (struggling!) to complete the lab, get the lab graded, and ask questions about lecture material they hadn't yet understood. The grading would inevitably take up almost all the lab...

I was a TA around 2004-2005. (I started in 2003 and graduated in 2007).

We didn't have automated grading for the labs then either, but we did for all the assignments. Still manual marking on style though, which to this day I have complex feelings about.