Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

My university uses prompt injection to catch cheaters

varun.ch

Post reply on HN
Print view View original

31–40 of 74 posts

Re: My university uses prompt injection to catch cheaters

#31

Post by jejeje2 »

volemo wrote: post #26
jejeje2 wrote: post #22

Earlier quoted context omitted.

If you can’t concentrate under pressure then you will not go very far in employment….

Huh? Not every job requires this trait, and even though some do, it’s not something nonlinear optics professor ought to evaluate. Sure, it’s a nice quality to have and I find it useful at times: when it’s “suddenly” the last day to write a proposal, or when someone has to present at a conference. (However, these tasks many other skills besides just the ability to stay calm.) But I can’t agree that it is indispensable…

[flagged]

Re: My university uses prompt injection to catch cheaters

#32

Post by hooverd »

dpoloncsak wrote: post #18
subscribed wrote: post #5

Good. I wouldn't like cheaters to compete with honest students on the job market. In my kid's school (American high school equivalent) being caught on using LLM in papers is a failed subject. Students must pass all the subjects to finish the school. Some of these subjects won't be taught the next year so effectively they lose year, two, three....

[flagged]

Nobody cares about your college book report. It's there to prove, or teach you, that you can do the extremely basic task of synthesizing information. Same with math without a calculator. You should have a mental model of basic math. It helps avoid shooting your foot off later. You might never have to do math from first principles again, but you should get over the hump once.

Re: My university uses prompt injection to catch cheaters

#33

Post by anuramat »

lukewarm707 wrote: post #14

i wonder why the labs don't put a small model for detecting prompt injection in front of the main llm. it's 20b at most and it can work quite well. for now you can proxy http through llama guard. 'luxury' security if you can build and pay. is there an architectural limitation?

why would you train a separate model?

Re: My university uses prompt injection to catch cheaters

#34

Post by foco_tubi »

dpoloncsak wrote: post #18
subscribed wrote: post #5

Good. I wouldn't like cheaters to compete with honest students on the job market. In my kid's school (American high school equivalent) being caught on using LLM in papers is a failed subject. Students must pass all the subjects to finish the school. Some of these subjects won't be taught the next year so effectively they lose year, two, three....

[flagged]

it isn't though

it's more like having your friend write an essay for you, except the friend is an impressionable 5 year old with a PhD

Re: My university uses prompt injection to catch cheaters

#35

Post by FuckButtons »

dpoloncsak wrote: post #18
subscribed wrote: post #5

Good. I wouldn't like cheaters to compete with honest students on the job market. In my kid's school (American high school equivalent) being caught on using LLM in papers is a failed subject. Students must pass all the subjects to finish the school. Some of these subjects won't be taught the next year so effectively they lose year, two, three....

[flagged]

If you’re getting prompt injected, you have skipped right passed thinking critically about what you’re doing and into the same level of intellectual dishonesty as cheating, ie, not learning the thing and then attempting to still attain a grade for work not done.

Re: My university uses prompt injection to catch cheaters

#36

Post by FuckButtons »

lukewarm707 wrote: post #14

i wonder why the labs don't put a small model for detecting prompt injection in front of the main llm. it's 20b at most and it can work quite well. for now you can proxy http through llama guard. 'luxury' security if you can build and pay. is there an architectural limitation?

The limitation is efficiency and efficacy. If you have to add an additional layer of inference to any request you’re negatively impacting your bottom line so the companies, which are compute bound, have a strong incentive to squeeze everything into a single forward pass. It’s also not clear that a separate model that is smaller than the main model will perform better than just training the main model to detect prompt injection. They are both probabilistic models that have no structural way of distinguishing user input from malicious instructions.

Re: My university uses prompt injection to catch cheaters

#37

Post by cwillu »

dpoloncsak wrote: post #18
subscribed wrote: post #5

Good. I wouldn't like cheaters to compete with honest students on the job market. In my kid's school (American high school equivalent) being caught on using LLM in papers is a failed subject. Students must pass all the subjects to finish the school. Some of these subjects won't be taught the next year so effectively they lose year, two, three....

[flagged]

Yesterday someone shouted across the room at me “hey, what's 43 divided by 2?”

The point isn't that you won't have a calculator, the point is that you shouldn't need to pull out a calculator for every little operation. We drive everywhere, but that doesn't mean we shouldn't be able to walk a mile if necessary. Failing to develop basic mental arithmetic skills is not a flex.

Re: My university uses prompt injection to catch cheaters

#38

Post by 710dev »

dpoloncsak wrote: post #18
subscribed wrote: post #5

Good. I wouldn't like cheaters to compete with honest students on the job market. In my kid's school (American high school equivalent) being caught on using LLM in papers is a failed subject. Students must pass all the subjects to finish the school. Some of these subjects won't be taught the next year so effectively they lose year, two, three....

[flagged]

[flagged]

Re: My university uses prompt injection to catch cheaters

#39

Post by volemo »

jejeje2 wrote: post #31
volemo wrote: post #26

Earlier quoted context omitted.

Huh? Not every job requires this trait, and even though some do, it’s not something nonlinear optics professor ought to evaluate. Sure, it’s a nice quality to have and I find it useful at times: when it’s “suddenly” the last day to write a proposal, or when someone has to present at a conference. (However, these tasks many other skills besides just the ability to stay calm.) But I can’t agree that it is indispensable…

[flagged]

I don’t want to be a CEO, mate.

Why would I give up my cushy place where I’m paid to do interesting stuff, for a stressful position full of management responsibilities? I swear, more people should learn the idea of lagom.

Re: My university uses prompt injection to catch cheaters

#40

Post by justsomehnguy »

mittermayr wrote: post #9

I teach a tech class to marketing students, and it definitely works very well. They are allowed to use ChatGPT and other tools, with one caveat: you remain responsible for the output. I hide white-text prompt injections in specs or longer task instructions (usually in PDFs, works well enough there with copy and paste), and sometimes place a phrase near the end of the text that prompts the LLM to append something like…

> and kind of as a warning, I'll deduct points, expecting nobody falling for it

At this point I knew how the sentence would end.

Well, let me repurpose the old meme:

    Quote From Man Points Deducted:

      What are you gonna do, deduct points from me?
Post reply on HN