Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Analyzing student votes across AI models for college essay help

studyarena.com

Post reply on HN
Print view View original

51–60 of 88 posts

Re: Analyzing student votes across AI models for college essay help

#51

Post by dr_dshiv »

Aurornis wrote: post #37

The spread isn't as large as I thought from the headline. It's closer to a toss-up than I would have expected. The length of the response is a huge factor: > Students tended to prefer longer responses. The selected answer was 37% longer on average than the alternatives. The longest response won 47.7% of decisive writing comparisons. The shortest still won 25.0%. So the score is partially a proxy for longest responses…

Hmm, seems like which model teachers prefer would be the more important headline…

Re: Analyzing student votes across AI models for college essay help

#52

Post by pasharayan »

demibabs wrote: post #21

Author, that doesn’t mean you need to get Gemini to write your article.

TBH i did a bit of both - we used AI to get us insights in the db, we then wrote and rewrote the article to be readable. It's really hard because usually when i write reports they're so terse its not fun to read

Re: Analyzing student votes across AI models for college essay help

#53

Post by dzhiurgis »

Been using Gemini for about a year for code (much cheaper and actually sticks to your prompt so savings are many multiples of; I work with Salesforce implementations and Google is a user for Salesforce so I think their code might have went into training data, coupled with their code guidelines it's been a bliss for me) and for other uses for maybe 9 months.

I do go back to ChatGPT for things that need to be calculated, but _Gemini simply hallucinates less_. IMO that's whats important for students.

I did get suckered into extending my Grok subscription which is really fun for images/videos. Grok in car is a must too (my jaw dropped when I asked what is pantsula music and it added multiple albums into playlist for me).

Re: Analyzing student votes across AI models for college essay help

#54

Post by danpalmer »

madhatter999 wrote: post #50
danpalmer wrote: post #49

Earlier quoted context omitted.

That's up to you, but occam's razor would suggest the most likely thing is that they are in fact just doing what they say they do in their T&Cs.

If that's what Occam's razor would suggest, then Occam presumes a world where actors do what they say. We see more and more the opposite of that in Silicon Valley. You may well be right about Google not including the said data but your support for this point is weak.

Yes Occam does quite literally state that – the option with the fewest assumptions is most likely to be correct. Assuming that someone is lying is an extra assumption to make over what they are claiming to do.

Malicious acts from large corporations are certainly well known, but they're also grossly over-represented in the news and mindshare. Mostly things just work as they say they work.

I don't really know what proof you're looking for, it's well known that it's hard to prove a negative. I'm not suggesting taking Google (or anyone else) at face-value, but rather saying that when you think about T&Cs, laws, PR, etc, it's unlikely.

Re: Analyzing student votes across AI models for college essay help

#55

Post by pasharayan »

Aurornis wrote: post #37

The spread isn't as large as I thought from the headline. It's closer to a toss-up than I would have expected. The length of the response is a huge factor: > Students tended to prefer longer responses. The selected answer was 37% longer on average than the alternatives. The longest response won 47.7% of decisive writing comparisons. The shortest still won 25.0%. So the score is partially a proxy for longest responses…

Yeah it's hard to get good data on what's best. We did read the a good sample amount of essays. Lots of arguments about what made them better, but its uncontroversial that longer ones were more preferred.

Interestingly we trained a simple LORA layer on top of Inkling and it turns out a model can smell which model wrote a response 70% of the time. I wonder if the smell of gemini is just more preferred by students.

Re: Analyzing student votes across AI models for college essay help

#56

Post by pasharayan »

dr_dshiv wrote: post #51
Aurornis wrote: post #37

The spread isn't as large as I thought from the headline. It's closer to a toss-up than I would have expected. The length of the response is a huge factor: > Students tended to prefer longer responses. The selected answer was 37% longer on average than the alternatives. The longest response won 47.7% of decisive writing comparisons. The shortest still won 25.0%. So the score is partially a proxy for longest responses…

Hmm, seems like which model teachers prefer would be the more important headline…

You know.. might try to see what we can do here

Re: Analyzing student votes across AI models for college essay help

#58

Post by pasharayan »

nickhodge wrote: post #43

Did they assess if any wanted to use their own brain?

Interestingly, the smartest students I've worked with do use AI - but more in the sense that AI is a good comparison tool and a good bottom ceiling of the quality of work they should bring in.

Lots of research shows being able to see different answers improves the quality of a student's output - especially as the good ones are able to pick out what's good from the other work and then add their own insights or takes on top.

Re: Analyzing student votes across AI models for college essay help

#59

Post by pasharayan »

levelZero wrote: post #57

Claude prose praising Gemini ;-)

Codex - at least initially haha! I wrote lots of insights and findings and pushed out a lot of false logic it picked up. Personally I hate claude writing cause it feels bureaucratic. But definitely wrote a ton more with my bare hands and brain.
Post reply on HN