Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

CMU Computer Systems: Self-Grading Lab Assignments (2018)

csapp.cs.cmu.edu

Post reply on HN
Print view View original

21–30 of 48 posts

Re: CMU Computer Systems: Self-Grading Lab Assignments (2018)

#21

Post by josephg »

Shoop wrote: post #16
albntomat0 wrote: post #14

Earlier quoted context omitted.

> Turnaround time for exams was sub-one-day. What's the current status on accurately grading mistakes? From the exams I remember (Fall 2015), the exams autograded perfect answers well, but gave similar marks to nearly correct, but still wrong answers as complete gibberish. I felt that unless I had the answers exactly correct, it was hard to demonstrate my knowledge of the topic.

Now there is a regrade system that lets you ask for more partial credit if you feel that the autograder was too harsh. That being said, its still relatively easy to get a lot of points off for a small mistake :(. You definitely have to be really careful during 213 exams but thankfully there's plenty of time to redo all the questions 2 or 3 times.

At UNSW we had a similar system. The marking was a test suite, and each passing test would give you a portion of your grade. The system had a couple of simple smoke tests which were ran on submission time (to catch issues that would make the entire test run fail erroneously). As a TA I spent a fair bit of time fixing trivial issues - eg, sometimes a student would get 3/10 on the auto marker with a basically working solution because of a typo or something. I would dock a mark, fix the bug and rerun the tests - leaving the student with 8/10 or something.

Re: CMU Computer Systems: Self-Grading Lab Assignments (2018)

#23

Post by josephg »

dooglius wrote: post #12

I remember when my University copied a couple of these, I decided to go the full mile on the one where you exploit buffer overflow. I got myself a remote connection via execve, and was going to mess around, but soon realized I could also see and edit others' scores which could get me in significant trouble, so I had to make one log-scrubbing attack then stop. I'm surprised they never fixed the hole by containerizing…

I ran an in-class test/competition once where students submitted C code to a web server running on my laptop. My laptop compiled and ran their code - then checked it against a test suite and graded it all live.

One student asked about security - and we got into a great discussion about permissions and what havoc he could and couldn’t pull off with the restricted user account. I encouraged him to figure out how to write a fork bomb - which he got working after the test. He was nervous about it because he didn’t want to get in trouble. With some reassurances he dove into API documentation and got it working. My computer totally died and needed a reboot. It was a great little teaching opportunity that I’m glad I didn’t pass up

Re: CMU Computer Systems: Self-Grading Lab Assignments (2018)

#24

Post by mav3rick »

abhayb wrote: post #8

Former student and TA here. There is so much great stuff in this class and its assignments! Couple of highlights: - Data lab formally verifies all of the submissions and gives you an exact failing test case if the submission isn't correct. - Malloc lab is run competitively. You don't need to compete to do well, but you do if you want to do great. And the competition drives students to great feats. I know people who r…

I loved this class at CMU. Recently I did shell lab again in Rust (trying to learn Rust) and used the test scripts for sanity checks. 213 helps me to this day :)

Re: CMU Computer Systems: Self-Grading Lab Assignments (2018)

#25

Post by neilv »

swiley wrote: post #20
abhayb wrote: post #8

Former student and TA here. There is so much great stuff in this class and its assignments! Couple of highlights: - Data lab formally verifies all of the submissions and gives you an exact failing test case if the submission isn't correct. - Malloc lab is run competitively. You don't need to compete to do well, but you do if you want to do great. And the competition drives students to great feats. I know people who r…

University is fundamentally a social thing, both between peers and between students and knowledgeable people like the professor. >- The automated grading of assignments meant that the class could scale to many hundreds of students When you have things like this why are we paying so much?

That social thing exists sometimes, but I think it's overstated. Consider, for example, the many students whose only direct interaction with the professor is to try to get an extension on an assignment (arguably late because homework was encroaching on their hangover obligations).

You can think of ways with simple technology and social organization to scale some version of the mythical social thing to much larger numbers of students, and to be available to anyone who will use that sincerely, not some lottery of acceptance and whether/how you can manage to swing the huge tuition (possibly taking on crippling debt for the rest of your life).

I look forward to my degrees becoming officially worthless, with everyone having the same or better education easily available to them. We're already getting there, but we need to keep going in this direction, and to dispense with pretense. Things like better scalable feedback help.

Re: CMU Computer Systems: Self-Grading Lab Assignments (2018)

#26

Post by j16sdiz »

neilv wrote: post #25
swiley wrote: post #20

Earlier quoted context omitted.

University is fundamentally a social thing, both between peers and between students and knowledgeable people like the professor. >- The automated grading of assignments meant that the class could scale to many hundreds of students When you have things like this why are we paying so much?

That social thing exists sometimes, but I think it's overstated. Consider, for example, the many students whose only direct interaction with the professor is to try to get an extension on an assignment (arguably late because homework was encroaching on their hangover obligations). You can think of ways with simple technology and social organization to scale some version of the mythical social thing to much larger num…

> I look forward to my degrees becoming officially worthless, with everyone having the same or better education easily available to them

Lots of degree have become worthless, with the tuition fee keep going up........

Re: CMU Computer Systems: Self-Grading Lab Assignments (2018)

#27

Post by ollien »

closeparen wrote: post #22
ec109685 wrote: post #17

These would have been fun! For the bomb one, are there protections for taking the binary and running it in an isolated environment?

You set a breakpoint at the phone-home routine.

My friends and I went the extra mile and overwrote the call to the server with nops.

Re: CMU Computer Systems: Self-Grading Lab Assignments (2018)

#28

Post by Cogito »

josephg wrote: post #21
Shoop wrote: post #16

Earlier quoted context omitted.

Now there is a regrade system that lets you ask for more partial credit if you feel that the autograder was too harsh. That being said, its still relatively easy to get a lot of points off for a small mistake :(. You definitely have to be really careful during 213 exams but thankfully there's plenty of time to redo all the questions 2 or 3 times.

At UNSW we had a similar system. The marking was a test suite, and each passing test would give you a portion of your grade. The system had a couple of simple smoke tests which were ran on submission time (to catch issues that would make the entire test run fail erroneously). As a TA I spent a fair bit of time fixing trivial issues - eg, sometimes a student would get 3/10 on the auto marker with a basically working s…

When did you TA?

I was TA for 1911 (I think?) around 2008-2009 and don't recall any automated grading system. It would have been so nice too, as labs would end up with students trying (struggling!) to complete the lab, get the lab graded, and ask questions about lecture material they hadn't yet understood.

The grading would inevitably take up almost all the lab...

Re: CMU Computer Systems: Self-Grading Lab Assignments (2018)

#29

Post by jibcage »

Former head TA (Jack) here - 15-213 still remains incredibly foundational to the work I do for my job. Working with the professors of that course (who also wrote the CS:APP textbook, which I still regret never getting signed by them!) was an amazing opportunity and something I hope I never forget.

If you want some exercises to help you learn the foundations of computer systems, I honestly cannot think of a better resource than this!

Re: CMU Computer Systems: Self-Grading Lab Assignments (2018)

#30

Post by yuji »

abhayb wrote: post #8

Former student and TA here. There is so much great stuff in this class and its assignments! Couple of highlights: - Data lab formally verifies all of the submissions and gives you an exact failing test case if the submission isn't correct. - Malloc lab is run competitively. You don't need to compete to do well, but you do if you want to do great. And the competition drives students to great feats. I know people who r…

Definitely was one of my favorite classes at CMU. Absolutely loved it.
Post reply on HN