Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Students have to jump through absurd hoops to use exam monitoring software

vice.com

Post reply on HN
Print view View original

31–40 of 242 posts

Re: Students have to jump through absurd hoops to use exam monitoring software

#31

Post by _8091149529 »

BalinKing wrote: post #16
_8091149529 wrote: post #6

Earlier quoted context omitted.

I spent a few years at Caltech in an academic role (not as a student). I never understood the rationale behind the unsupervised take-home exams. Could you elaborate what is it that makes it a 'success', in your opinion? The shortcomings that seem obvious to me are: - Penalizes honesty - Implied notion that Caltech students are "honourable" and honest. How is this achieved in practice?

Regarding "penalizing honesty," I think it actually rewards honestly quite greatly—because of the Honor Code's importance in Caltech culture (i.e. one of the undergrad application essays is specifically about the Honor Code), it means that everyone gets to benefit from take-home exams. I guess what I'm trying to say is that the "implied notion that Caltech students are... honest" is, in my experience, largely true (p…

My counterarguments, in good faith:

Clearly, any society collectively benefits from honesty, whereas an honest act is, at least in a strict game-theoretic view, a loss to the individual in the short term.

The question "Are you honest?" does not necessarily filter out dishonest people. (Incidentally, I always felt like I would not have been admitted to Caltech as a student.)

Re: Students have to jump through absurd hoops to use exam monitoring software

#32

Post by cosmie »

jackson1442 wrote: post #19
cosmie wrote: post #9

Back in the good ol' days, you could just take the exam inside of a VM. The VM is locked down, but the host isn't. Looks like from ProctorU's docs[1] that they got around to probing for the (blatantly obvious[2]) signatures of a VM and stopped that trick. I wonder if those signatures exist in a Windows Sandbox[3] instance, or if you can detect/block the Enhanced RDP Session Windows leverages when running a Sandbox...…

Most of these applications simply look at your manufacturer if I'm remembering correctly (at the very most, it's one easily changeable property). You can generally get past these checks by setting any strings that reference VirtualBox or the likes to Dell.

The easiest one to check (from the browser) is the WebGL Renderer. I'm currently on a Mac running Windows 10 in a Parallels VM, and checking here[1] shows

  ANGLE (Parallels Display Adapter (WDDM) Direct3D11 vs_5_0 ps_5_0)
The "Parallels Display Adapter (WDDM)" seems to be coming from my Display Adapter name, which I could change. But I'm not sure if any other part of that string leaks my computer's status as a VM.

My non-standard screen resolution is another dead giveaway. There may be others as well, but those two seem to be the lowest hanging fruit. And many of these proctoring suites run as browser extensions, and I'm not sure what additional information extensions have access to.

But yea, they're not too hard to trick. You could get fancy, around listening for mouse clicks and keyboard taps, without detecting any corresponding actions within the VM. But that sounds like far too much effort for a higher education software vendor.

[1] https://browserleaks.com/webgl

Re: Students have to jump through absurd hoops to use exam monitoring software

#33

Post by ColanR »

lights0123 wrote: post #30
ColanR wrote: post #26

Earlier quoted context omitted.

That's interesting - I would have guessed that after a certain point, a VM and a physical computer are indistinguishable from the POV of software running inside.

It's easy to make the core device indistinguishable. It's the additional hardware that becomes a problem: QEMU/libvirt hardcodes its hard drive model to include the word "QEMU" (although there is an unmerged patch to make that configurable), and although I don't know how graphics work in a VM, it's far easier to create a higher-level graphics device than e.g. emulating Intel graphics.

I imagine a script could be used with qemu - given that patch you mentioned, which I would love a source for if you have it - to match the names of the virtual devices with names of devices on the physical host. Then there's no way for software to check the device names against a list of VM tools, since it always matches real, physical hardware.

At that point, it seems to me that there isn't much left to distinguish the virtual machine from the physical. Behavioral properties of the CPU? Anyway, that's what I meant when I said it seems like after a certain point, it becomes impossible to tell the difference.

EDIT: based on the link above, it looks like current state-of-the-art doesn't go far beyond making sure names and common virtualbox performance shortcuts aren't present. https://github.com/hfiref0x/VBoxHardenedLoader/blob/master/B...

Re: Students have to jump through absurd hoops to use exam monitoring software

#34

Post by criddell »

aaplok wrote: post #8
bitwize wrote: post #7

Makes me glad for things like the Stevens Honor System, which was how exams were administered at Stevens Institute of Technology. Basically, all work submitted, whether tests or homework, require a pledge with a signature that the work is your own and you didn't cheat. Violations of the honor system -- cheating, plagiarism, etc. could be reported by students or faculty and would be investigated by the Honor Board wit…

People are doing that at my institution. The average GPA increased by several points . The thing is, if it's perceived that it's easier to cheat than to learn, people will do that, honor system or not. I admit that there is a difference between an institutionalised honor system, designed with respect toward and in collaboration with students, and a last-minute arrangement by an unprepared organisation. Students have…

Maybe part of the solution is to get rid of GPAs and switch to a pass-fail system.

Re: Students have to jump through absurd hoops to use exam monitoring software

#35

Post by tzs »

BalinKing wrote: post #17
beisner wrote: post #10

Earlier quoted context omitted.

The idea is that you create questions that are ungoogleable. Most of my upper-level CS courses had take-homes that were open-book, open-internet, and the professor basically said “good luck finding the answer to this problem online; you will not”. This is harder for professors to do, and maybe fails in intro courses, but generally is quite effective.

One of my Ph 1b (freshman special relativity and electrostatics) quizzes guided you through "discovering" a new method of solving that type of problem. I don't remember the specific details, but it's one of the rare tests I've taken that I actually enjoyed! Although most homeworks and exams aren't nearly that cool IME, they're virtually all about synthesis and fundamental understanding rather than rote memorization.…

When Carl Sagan was writing "Contact" he attended a conference that was also attended by his friend Kip Thorne. Sagan knew that one of Thorne's pet peeves was science fiction that just hand-waved away the physics of FTL travel.

Sagan told Thorne he was writing a science fiction novel, needed FTL travel, and asked Thorne if he could suggest something that would be reasonable. Thorne agreed to look into it.

After the conference, Thorne spent a while working on it and came up with a wormhole approach and worked out the physics of it.

In addition to giving it to Sagan, Thorne also put it on the Ph 236 (General Relativity) final exam. He didn't tell the students on the exam that it might imply FTL travel. He just set up the conditions and had them work out the physics. Most of his students succeeded in that, but he was a little disappointed that none of them happened to notice that it implied FTL travel.

(I got the above from an unpublished book Thorne was working on in the early '80s. It was a collection of biographies of and interviews with physicists, astronomers, cosmologists, etc., written and conducted by Thorne. He had the draft chapters and the raw interview transcripts in a world readable directory on the physic's departments VAX, where they were widely read by the rest of us with accounts on that machine).

Re: Students have to jump through absurd hoops to use exam monitoring software

#36

Post by turbinerneiter »

_8091149529 wrote: post #23
908B64B197 wrote: post #13

Earlier quoted context omitted.

This. Else you might as well call the degree "rote" because that's what you measure. > maybe fails in intro courses If someone cheats their way through intro courses they are up for a bad surprise since the upper level courses assumes a mastery of the intro classes!

Interesting, thanks for the replies. I'd still point out that the system is not robust against: - Working in groups. (Bad if tests are supposed to assess individual performance.) - Asking outsiders for help. Also, your downplaying of "rote" learning feels misguided, no matter how advanced/abstract/high-level the domain in question is. Cue the Bruce Lee quote about 10,000 kicks..

Just imagine a world were solving problems in groups and asking for outside help is bad.

We are so used to exams being about doing stuff that literally nobody in real life would ever even consider.

Re: Students have to jump through absurd hoops to use exam monitoring software

#37

Post by cosmie »

advisedwang wrote: post #24
cosmie wrote: post #9

Back in the good ol' days, you could just take the exam inside of a VM. The VM is locked down, but the host isn't. Looks like from ProctorU's docs[1] that they got around to probing for the (blatantly obvious[2]) signatures of a VM and stopped that trick. I wonder if those signatures exist in a Windows Sandbox[3] instance, or if you can detect/block the Enhanced RDP Session Windows leverages when running a Sandbox...…

Many of the onerous requirements mentioned in the article are about the test taker and the environment they are in, not hte computer envirionment. A VM doesn't help you when your professor is requiring you to rearrange your room, set up mirrors, bash earplugs etc.

That's exactly it - all of the onerous requirements are explicitly not about the computer, because the professors are presuming the software has that covered.

Except higher education software is known more for milking it's market than for it's robustness and sophistication. The type of proctoring software used here was trivially bypassed only a few years ago, when the last of my friends were finishing up college. I haven't been exposed to the topic since the last of them graduated, and from ProctorU's help center it looks like the vendors have wised up to the VM trick (in some fashion). But it can still likely be subverted relatively easily, rendering all of those onerous environmental requirements moot since they all explicitly ignore the computer itself.

I sort of wish I had access to it, so I could probe around myself and write up a tutorial. Not that I'm an advocate of cheating, but more so as a big "F* You" to the entire draconian process.

Re: Students have to jump through absurd hoops to use exam monitoring software

#38

Post by WrtCdEvrydy »

ColanR wrote: post #33
lights0123 wrote: post #30

Earlier quoted context omitted.

It's easy to make the core device indistinguishable. It's the additional hardware that becomes a problem: QEMU/libvirt hardcodes its hard drive model to include the word "QEMU" (although there is an unmerged patch to make that configurable), and although I don't know how graphics work in a VM, it's far easier to create a higher-level graphics device than e.g. emulating Intel graphics.

I imagine a script could be used with qemu - given that patch you mentioned, which I would love a source for if you have it - to match the names of the virtual devices with names of devices on the physical host. Then there's no way for software to check the device names against a list of VM tools, since it always matches real, physical hardware. At that point, it seems to me that there isn't much left to distinguish…

This is a classic computer science question of whether the OS should be aware that it's being virtualized.

Re: Students have to jump through absurd hoops to use exam monitoring software

#39

Post by cosmie »

ColanR wrote: post #33
lights0123 wrote: post #30

Earlier quoted context omitted.

It's easy to make the core device indistinguishable. It's the additional hardware that becomes a problem: QEMU/libvirt hardcodes its hard drive model to include the word "QEMU" (although there is an unmerged patch to make that configurable), and although I don't know how graphics work in a VM, it's far easier to create a higher-level graphics device than e.g. emulating Intel graphics.

I imagine a script could be used with qemu - given that patch you mentioned, which I would love a source for if you have it - to match the names of the virtual devices with names of devices on the physical host. Then there's no way for software to check the device names against a list of VM tools, since it always matches real, physical hardware. At that point, it seems to me that there isn't much left to distinguish…

One of the other telltale signs of a VM are "things VMs can do that physical devices can't" - such as weird screen resolutions (when running in windowed mode rather than full screen), weird CPU core counts, physical ram values that aren't cleanly divisible into DIMM slots, etc.

Re: Students have to jump through absurd hoops to use exam monitoring software

#40

Post by _8091149529 »

turbinerneiter wrote: post #36
_8091149529 wrote: post #23

Earlier quoted context omitted.

Interesting, thanks for the replies. I'd still point out that the system is not robust against: - Working in groups. (Bad if tests are supposed to assess individual performance.) - Asking outsiders for help. Also, your downplaying of "rote" learning feels misguided, no matter how advanced/abstract/high-level the domain in question is. Cue the Bruce Lee quote about 10,000 kicks..

Just imagine a world were solving problems in groups and asking for outside help is bad. We are so used to exams being about doing stuff that literally nobody in real life would ever even consider.

I was aware of the implications, hence the disclaimer "(Bad if ...)"

However, I'd assume that doing either during a test would be against the honor code.

Post reply on HN