I'm a Berkeley prof. Working at a startup led me to update my 1500-person class • Hacker News Board

Re: I'm a Berkeley prof. Working at a startup led me to update my 1500-person class

#11

Post by anon22981 » Fri, Oct 27, 2023, 12:05 PM UTC

taftster wrote: post #7

craigts wrote: post #3

I would spend my entire time trying to cheat or subvert the rules.

And you would probably come away with a lot of (well deserved) knowledge from this assignment. If a professor can inspire you to actively seek alternatives and/or think outside the box, I consider that a win.

In my studies, we had a couple of courses on project models and the business end of things etc. The exam materials consisted of several pdfs and/or powerpoints. The exams were remote and not monitored.

Technically it was pretty easy to use the materials on the exams, but this was limited by setting pretty short time limits on these tests. So you’d have to know the answers instead of going through the materials in several different files. Or that was the idea.

My solution was to build a desktop application that parsed through pdfs and powerpoints in a given folder and presented all these pages in a zoomed out view. You could click and open the pages for closer view, etc. The juice was that you could feed keywords to a search and it’d show all pages where the keywords hit. So I could open up all the materials at the same time, write a keyword from the exam question and look through the relevant pages.

Admittedly this didn’t teach me that much about the actual subject, but I think it was a net-positive by coming out as a better programmer.

Re: I'm a Berkeley prof. Working at a startup led me to update my 1500-person class

#12

Post by joshhug » Sat, Oct 28, 2023, 6:26 PM UTC

ccooffee wrote: post #5

joshhug wrote: post #2

This is a weird assignment idea. I’d love to hear what you think, how you might improve it, etc. Some specific open questions I have: 1. How can I nudge students towards being creative with their comparators so that we get more variety? 2. Is there a natural extension that would get students working on the same files, yielding a richer demonstration of CI? 3. Could we allow students to somehow edit the contest code i…

I really don't like this assignment idea as anything more than getting used to the assignment submission and grading system. 1. What is it teaching? Comparator is trivial to implement, and you're adding no requirements on it (except maybe having a human readable shortname). Code reviewing a Comparator which merely has to compile and match the two simple rules of Comparators is not a useful code review. CI tests to ch…

1. It's definitely not about the Comparator. The lesson is all about showing off a CI/code review process. This is intended to be a short exercise, expected to take no more than an hour. The CI tests can't check that the comparator actually does what it claims, they'll just test that it's a valid comparator. And I'd build on this in a future pair assignment, where they can use some or all of the steps of the workflow from this more gimmicky assignment.

2. I agree it's a bad use of student time to see if someone's submission is unique, but I suspect there are ways we could structurally push students to do something unique, e.g. gus_massa's post below.

3. Yeah, it's a little weird. The integer submission is just to keep the macguffin of the assignment going. Also for N submissions, we'll use at least N-1 comparators. If we use RNG that cycles through every comparator before repeating, we can make sure everyone gets used at least once.

Re: I'm a Berkeley prof. Working at a startup led me to update my 1500-person class

#13

Post by joshhug » Sat, Oct 28, 2023, 6:32 PM UTC

gus_massa wrote: post #6

joshhug wrote: post #2

This is a weird assignment idea. I’d love to hear what you think, how you might improve it, etc. Some specific open questions I have: 1. How can I nudge students towards being creative with their comparators so that we get more variety? 2. Is there a natural extension that would get students working on the same files, yielding a richer demonstration of CI? 3. Could we allow students to somehow edit the contest code i…

About 1: * Generate a list of 1000000 pairs of random numbers. Run the Judge of each student on that list to get a new list of ternary numbers. Compare the list with the list of all the other students. The distance is the numbers of differences with the closest one. The one that is more far away wins a chocolate. Perhaps add the variants like 123111213 -> 321333231 to detect mirror criteria. Add 2222... (everyone tie…

I like these! And I encourage a little bit of cheating. I'm curious to see how the particularly clever students will subvert the system.

As it happens, I had a fun assignment all about subverting the system back in the day when I taught security at Princeton back in 2012. In the final assignment (assignment 8), students were given a Linux disk image of a hard drive owned by a guy named Nefarious who had committed a murder. This was an assignment originally created by J Alex Halderman and Ed Felton, but I went a little extra in my version.

In the assignment text, I mentioned that Nefarious was originally arrested due to an anonymous tip from a pseudonymous "Cecco Beppe". Buried in /home/root on Nefarious's drive, there is an innocuously labeled file: CB.7z, which is notable only in that it is the only file on the drive dated 2012 (intentionally), years newer than all of the files on the drive. If one went to the trouble of unzipping this file, they'd realize it is a very small disk image inside the disk image.

Booting this image dropped the user into a chat with a depressed artificial intelligence (a custom ALICE chat bot, but with some hard coded responses to advance my story, this was pre-GPT) who, when prodded with the appropriate keywords, told the tale of the AI's depression about the murder and attempted but failed self-deletion -- leaving the AI nothing more than a pathetic and mostly incoherent chat bot.

Further prodding led the AI to reveal the existence of secret assignment #9 as well as a URL for said assignment.

At the URL, there is a zip file containing a flawed pseudorandom generator, a file encryption tool that uses said PRGen, an encrypted file, and a truncated copy of the corresponding plaintext. The truncated plaintext explains that their next task is to complete decryption of the file, which they can do by exploiting the flaw in the PRGenerator.

The deciphered text then explains that their next task is to go back to the HW1 autograder and submit a new solution for HW1 (which, incidentally, was to develop a PRGen) -- but with the catch that they may only use print statements, i.e. they're just trying to trick the parser. This was based on an security flaw I'd discovered in Princeton's grader while I was working on their first Coursera courses.

Once students submitted this cheating print statement code via the usual Princeton web submit for HW1, the autograder activated a secret message that explained the final part of secret assignment 9, which was to steal the source code for the HW1 autograder and send it to me.

This last part could be done by simply writing code that opening the .class files and prints them to the screen when the autograder script runs.

I had around 10 students (out of ~150) figure this all out, it was great.

Re: I'm a Berkeley prof. Working at a startup led me to update my 1500-person class

#14

Post by joshhug » Sat, Oct 28, 2023, 7:46 PM UTC

taftster wrote: post #9

joshhug wrote: post #2

This is a weird assignment idea. I’d love to hear what you think, how you might improve it, etc. Some specific open questions I have: 1. How can I nudge students towards being creative with their comparators so that we get more variety? 2. Is there a natural extension that would get students working on the same files, yielding a richer demonstration of CI? 3. Could we allow students to somehow edit the contest code i…

I agree with one of the sibling comments here. Comparator is just way too simple of an interface. And you've defined a lot of it in terms of a single individual, and maybe just slightly a "team" in the larger sense of the entire class. It's kind of a "boring" assignment. Just an idea. What if you created some sort of "robot" contest that defined various areas of functionality. Each robot could have sensors, motor con…

I'll get pondering. Having some sort of rube goldeberg-esque mega project was my original idea, but finding a specific form for it has been elusive.

Taking inspiration from Factorio in some way is an interesting proposition. Will consider!

Re: I'm a Berkeley prof. Working at a startup led me to update my 1500-person class

#15

Post by joshhug » Wed, Nov 01, 2023, 11:40 AM UTC

swatcoder wrote: post #8

What do GitHub Actions and "team workflows" have to do with Data Structures? Even before he mentioned what he was inspired to ad, the course seemed scattered and superficial rather than foundational and preparatory. I'm sure it's hard/impossible to drive one through the bureaucracy, but it sounds like he should have pitched a new course on Contemporary Team Practices or something.

Like a lot of schools, our Data Structures course doubles as a soft intro to software engineering. I want them to know how to solve real problems efficiently and manage complexity.

Student time is incredibly precious. The hope here is that students will spend less than an hour on this, and in the process get a taste of how code reviews / automation can help teams function better. I'm hoping I can thread the needle and give them tools that will help them more efficiently complete their capstone project at the end of the semester, without spending too much of their time teaching them these tools.

FWIW, we have discussed the idea of a lower division software engineering class, and I was really hoping Pamela Fox would do this before she headed back to industry (interesting story there: https://blog.pamelafox.org/2022/05/my-experience-as-unit-18-...). We do have an upper div software engineering course as well, though it's not taken by most of our students.