Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Auto-grading decade-old Hacker News discussions with hindsight

karpathy.bearblog.dev

Post reply on HN
Print view View original

201–210 of 285 posts

Re: Auto-grading decade-old Hacker News discussions with hindsight

#201

Post by DonHopkins »

dang wrote: post #26
lapcat wrote: post #18

Does anyone else think that HN engages in far too much navel-gazing? Nothing gets upvotes faster than a HN submission about HN.

It's true that meta is the crack of internet forums, so we, er, crack down on it quite a bit. That's a longstanding view: https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu... Alternate metaphor: evil catnip - https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que... But yesterday's thread and this one are clearly exceptions—far above the median. https://news.ycombinator.com/item?id=46212180 was parti…

Dang, posting links to searches for your own comments is so meta, no matter the topic, but even more meta when about meta crack. I love how the first hit of meta crack is this, your own message about meta crack.

Re: Auto-grading decade-old Hacker News discussions with hindsight

#202

Post by jacquesm »

godelski wrote: post #193
jacquesm wrote: post #181

Earlier quoted context omitted.

Yes, you are right, this is a real problem. But it really is just a variation on 'the internet never forgets', for instance in relation to teen behavior online. But AI allows for weaponization of such information. I wish the wannabe politicians of 2050 much good luck with their careers, they are going to be the most boring people available.

The internet never forgets but you could be anonymous. Or at least somewhat. But that's getting harder and harder If such a thing isn't already possible (it is to a certain extent), we are headed towards a point where your words alone will be enough to fingerprint you.

Stylometry killed that a long time ago. There was a website, stylometry.net that coupled HN accounts based on text comparison and ranked the 10 best candidates. It was incredibly accurate and allowed id'ing a bunch of people that had gotten banned but that came back again. Based on that I would expect that anybody that has written more than a few KB of text to be id'able in the future.

Re: Auto-grading decade-old Hacker News discussions with hindsight

#203

Post by WithinReason »

khafra wrote: post #195
popinman322 wrote: post #179

It doesn't look like the code anonymizes usernames when sending the thread for grading. This likely induces bias in the grades based on past/current prevailing opinions of certain users. It would be interesting to see the whole thing done again but this time randomly re-assigning usernames, to assess bias, and also with procedurally generated pseudonyms, to see whether the bias can be removed that way. I'd expect de-…

You can't anonymize comments from well-known users, to an LLM: https://gwern.net/doc/statistics/stylometry/truesight/index

That's an overly strong claim, an LLM could also be used to normalise style

Re: Auto-grading decade-old Hacker News discussions with hindsight

#204

Post by n4r9 »

tptacek wrote: post #69

'pcwalton, I'm coming for you. You're going down. Kidding aside, the comments it picks out for us are a little random. For instance, this was an A+ predictive thread (it appears to be rating threads and not individual comments): https://news.ycombinator.com/item?id=10703512 But there's just 11 comments, only 1 for me, and it's like a 1-sentence comment. I do love that my unaccredited-access-to-startup-shares take is…

Yeah, I'm having to pinch myself a little here. Another slightly odd example it picked out from your history: https://news.ycombinator.com/item?id=10735398

It's a good comment, but "prescient" isn't a word I'd apply to it. This is more like a list of solid takes. To be fair there probably aren't even that many explicit, correct predictions in one month of comments in 2015.

Re: Auto-grading decade-old Hacker News discussions with hindsight

#205

Post by godelski »

jacquesm wrote: post #202
godelski wrote: post #193

Earlier quoted context omitted.

The internet never forgets but you could be anonymous. Or at least somewhat. But that's getting harder and harder If such a thing isn't already possible (it is to a certain extent), we are headed towards a point where your words alone will be enough to fingerprint you.

Stylometry killed that a long time ago. There was a website, stylometry.net that coupled HN accounts based on text comparison and ranked the 10 best candidates. It was incredibly accurate and allowed id'ing a bunch of people that had gotten banned but that came back again. Based on that I would expect that anybody that has written more than a few KB of text to be id'able in the future.

You need a person's text with their actual identity to pull that off. Normally that's pretty hard, especially since you'll get different formats. Like I don't write the same way on Twitter as HN. But yeah, this stuff has been advancing and I don't think it is okay.

Re: Auto-grading decade-old Hacker News discussions with hindsight

#206

Post by OccamsMirror »

p-e-w wrote: post #187
OccamsMirror wrote: post #184

Earlier quoted context omitted.

Perhaps the fact that you think this field is only 5 years old means you're probably not enough of an authority to comment confidently on it?

Claiming that AI in anything resembling its current form is older than 5 years is like claiming the history of the combustion engine started when an ape picked up a burning stick.

Your analogy fails because picking up a burning stick isn’t a combustion engine, whereas decades of neural-net and sequence-model work directly enabled modern LLMs. LLMs aren’t “five years old”; the scaling-transformer regime is. The components are old, the emergent-capability configuration is new.

Treating the age of the lineage as evidence of future growth is equivocation across paradigms. Technologies plateau when their governing paradigm saturates, not when the calendar says they should continue. Supersonic flight stalled immediately, fusion has stalled for seventy years, and neither cared about “time invested.”

Early exponential curves routinely flatten: solar cells, battery density, CPU clocks, hard-disk areal density. The only question that matters is whether this paradigm shows signs of saturation, not how long it has existed.

Re: Auto-grading decade-old Hacker News discussions with hindsight

#207

Post by jacquesm »

godelski wrote: post #205
jacquesm wrote: post #202

Earlier quoted context omitted.

Stylometry killed that a long time ago. There was a website, stylometry.net that coupled HN accounts based on text comparison and ranked the 10 best candidates. It was incredibly accurate and allowed id'ing a bunch of people that had gotten banned but that came back again. Based on that I would expect that anybody that has written more than a few KB of text to be id'able in the future.

You need a person's text with their actual identity to pull that off. Normally that's pretty hard, especially since you'll get different formats. Like I don't write the same way on Twitter as HN. But yeah, this stuff has been advancing and I don't think it is okay.

The AOL scandal pretty much proved that anonymity is a mirage. You may think you are anonymous but it just takes combining a few unrelated databases to de-anonymize you. HN users think they are anonymous but they're not, they drop factoids all over the place about who they are. 33 bits... it is one of my recurring favorite themes and anybody in the business of managing other people's data should be well aware of the risks.

Re: Auto-grading decade-old Hacker News discussions with hindsight

#208

Post by johncolanduoni »

Tossrock wrote: post #196
johncolanduoni wrote: post #192

Earlier quoted context omitted.

Never call a man happy until he is dead. Also I don’t think your argument generalizes well - there are plenty of private research investment bubbles that have popped and not reached their original peaks (e.g. VR).

It wasn't a generalized argument, though, it was a specific one, about AI.

Okay, but the only part that’s specific to AI (that the companies investing the money are capturing more value than they’re putting into it) is now false. Even the hyperscalers are not capturing nearly the value they’re investing, though they’re not using debt to finance it. OpenAI and Anthropic are of course blowing through cash like it’s going out of style, and if investor interest drops drastically they’ll likely need to look to get acquired.

Re: Auto-grading decade-old Hacker News discussions with hindsight

#209

Post by intended »

nickff wrote: post #70
cootsnuck wrote: post #58

Earlier quoted context omitted.

The RES (Reddit Enhancement Suite) browser extension indirectly does this for me since it tracks the lifetime number of upvotes I give other users. So when I stumble upon a thread with a user with like +40 I know "This is someone whom I've repeatedly found to have good takes" (depending on the context). It's subjective of course but at least it's transparently so. I just think it's neat that it's kinda sorta a loose…

I am not a Redditor, but RES sounds like it would increase the ‘echo-chamber’ effect, rather than improving one’s understanding of contributors’ calibration.

Echo chambers will always result on social media. I don't think you can come up with a format that will not result in consolidated blocs.

Re: Auto-grading decade-old Hacker News discussions with hindsight

#210

Post by pnt12 »

On the site itself:

it's great that this was produced in 1h with 60$. This is amazing to create small utilities, explore your curiosity, etc.

But the site is also quite confusing and messy. OK for a vibe coded experiment, sure, but wouldn't be for a final product. But I fear we're gonna see more and more of this. Big companies downsizing their tech departments and embracing vibe coded. Comparing to inflation, shrinkflation and skimpflation/ enshittification , will we soon adopt some word for this? AIflation? LLMflation?

And how will this comment score in a couple of years? :)

Post reply on HN