Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Giving a domain a hill to climb: benchmarking as data activation

sparsethought.com

Post reply on HN
Print view View original

1–9 of 9 posts

Re: Giving a domain a hill to climb: benchmarking as data activation

#3

Post by galsapir »

2key wrote: post #2

interesting take not sure I 100% agree and yet, Interesting take

curious where the disagreement lands: the claim i'm least sure of myself is that measurement alone already counts as activation (nothing in the weights changes, so it's a looser sense of the word than usual) the part i'd defend harder is the eval -> reward one: once a benchmark becomes the thing you train against, its flaws stop being measurement error and start being incentives. if you're pushing back somewhere in there, i'd genuinely like to hear it

Re: Giving a domain a hill to climb: benchmarking as data activation

#4

Post by Terretta »

I wanted to like this and appreciated the genAI tokens were teased into a less AI-ish vernacular, but surfaces and honest caveats drifted back too far to overcome the "I'm human" signaling attempted through lowercase sentences and em dashes replaced with colons.

I agree with the problem described within, and in the dead comment here.

Re: Giving a domain a hill to climb: benchmarking as data activation

#8

Post by galsapir »

Terretta wrote: post #4

I wanted to like this and appreciated the genAI tokens were teased into a less AI-ish vernacular, but surfaces and honest caveats drifted back too far to overcome the "I'm human" signaling attempted through lowercase sentences and em dashes replaced with colons. I agree with the problem described within, and in the dead comment here.

thanks for reading it properly and engaging with the argument!

writing is hard, expressing ideas cleanly is harder! working on it.

Post reply on HN