Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Auto-grading decade-old Hacker News discussions with hindsight

karpathy.bearblog.dev

Post reply on HN
Print view View original

51–60 of 285 posts

Re: Auto-grading decade-old Hacker News discussions with hindsight

#51

Post by dietr1ch »

moultano wrote: post #10

Notable how this is only possible because the website is a good "web citizen." It has urls that maintain their state over a decade. They contain a whole conversation. You don't have to log in to see anything. The value of old proper websites increases with our ability to process them.

> because the website is a good "web citizen." It has urls that maintain their state over a decade.

It's a shame that maintaining the web is so hard that only a few websites are "good citizens". I wish the web was a -bit- way more like git. It should be easier to crawl the web and serve it.

Say, you browse and get things cached and shared, but only your "local bookmarks" persist. I guess it's like pinning in IPFS.

Re: Auto-grading decade-old Hacker News discussions with hindsight

#52

Post by neilv »

> I spent a few hours browsing around and found it to be very interesting.

This seems to be the result of the exercise? No evaluation?

My concern is that, even if the exercise is only an amusing curiosity, many people will take the results more seriously than they should, and be inspired to apply the same methods to products and initiatives that adversely affect people's lives in real ways.

Re: Auto-grading decade-old Hacker News discussions with hindsight

#53

Post by btbuildem »

I've spent a weekend making something similar for my gmail account (which google keeps nagging me about being 90% full). It's fascinating to be able to classify 65k+ of emails (surprise: more than half are garbage), as well as summarize and trace the nature of communication between specific senders/recipients. It took about 50 hours on a dual RTX 3090 running Qwen 3.

My original goal was to prune the account deleting all the useless things and keeping just the unique, personal, valuable communications -- but the other day, an insight has me convinced that the safer / smarter thing to do in the current landscape is the opposite: remove any personal, valuable, memorable items, and leave google (and whomever else is scraping these repositories) with useless flotsam of newsletters, updates, subscription receipts, etc.

Re: Auto-grading decade-old Hacker News discussions with hindsight

#54

Post by karpathy »

hackthemack wrote: post #49

I noticed the Hall of Fame grading of predictive comments has a quirk? It grades some comments about if they came true or not, but in the grading of comment to the article https://news.ycombinator.com/item?id=10654216 The Cannons on the B-29 Bomber "accurate account of LeMay stripping turrets and shifting to incendiary area bombing; matches mainstream history" It gave a good grade to user cstross but to my reading of…

Yes I noticed a few of these around. The LLM is a little too willing to give out grades for comments that were good/bad in a bit more general sense, even if they weren't making strong predictions specifically. Another thing I noticed is that the LLM has a very impressive recognition of the various usernames and who they belong to, and I think shows a little bit of a bias in its evaluations based on the identity of the person. I tuned the prompt a little bit based on some low-hanging fruit mistakes but I think one can most likely iterate it quite a bit further.

Re: Auto-grading decade-old Hacker News discussions with hindsight

#55

Post by cootsnuck »

collinmcnulty wrote: post #23

> But if intelligence really does become too cheap to meter, it will become possible to do a perfect reconstruction and synthesis of everything. LLMs are watching (or humans using them might be). Best to be good. I cannot believe this is just put out there unexamined of any level of "maybe we shouldn't help this happen". This is complete moral abdication. And to be clear, being "good" is no defense. Being good often…

To be clear...prior to this recent explosive interest in LLMs, this was already true. Snowden was over 10 years ago.

We can't start clutching our pearls now as if programmatic mass surveillance hasn't been running on all cylinders for over 20 years.

Don't get me wrong, we should absolutely care about this, everyone should. I'm just saying any vague gestures at imminent privacy-doom thanks to LLMs is liable to be doing some big favors of inadvertently sanitizing the history of prior (and still) egregious privacy offenders.

I'm just suggesting more "Yes and" and less "pearl clutching" is all.

Re: Auto-grading decade-old Hacker News discussions with hindsight

#56

Post by Teever »

collinmcnulty wrote: post #41
doctoboggan wrote: post #32

Earlier quoted context omitted.

I've had the same though as Karpathy over the past couple of months/years. I don't think it's good, exciting, or something to celebrate, but I also have no idea how to prevent it. I would read his "Best to be good." as a warning or reminder that everything you do or say online will be collected and analyzed by an "intelligence". You can't count on hiding amongst the mass of online noise. Imagine if someone were to co…

This is my plan at least 1. Don't build the Torment Nexus yourself. Don't work for them and don't give them your money. 2. When people you know say they're taking a new job to work at Torment Nexus, act like that's super weird, like they said they're going to work for the Sinaloa cartel. Treat rich people working on the Torment Nexus like it's cringe to quote them. 3. Get hostile to bots. Poison the data. Use AdNause…

Do you have any suggestions on how to interact online with people who work at Torment Nexus?

Re: Auto-grading decade-old Hacker News discussions with hindsight

#57

Post by moultano »

dietr1ch wrote: post #51
moultano wrote: post #10

Notable how this is only possible because the website is a good "web citizen." It has urls that maintain their state over a decade. They contain a whole conversation. You don't have to log in to see anything. The value of old proper websites increases with our ability to process them.

> because the website is a good "web citizen." It has urls that maintain their state over a decade. It's a shame that maintaining the web is so hard that only a few websites are "good citizens". I wish the web was a -bit- way more like git. It should be easier to crawl the web and serve it. Say, you browse and get things cached and shared, but only your "local bookmarks" persist. I guess it's like pinning in IPFS.

Yes, I wish we could serve static content more like bittorent, where your uri has an associate hash, and any intermediate router or cache could be an equivalent source of truth, with the final server only needing to play a role if nothing else has it.

It is not possible right now to make hosting democratized/distributed/robust because there's no way for people to donate their own resources in a seamless way to keeping things published. In an ideal world, the internet archive seamlessly drops in to serve any content that goes down in a fashion transparent to the user.

Re: Auto-grading decade-old Hacker News discussions with hindsight

#58

Post by cootsnuck »

modeless wrote: post #40

This is a cool idea. I would install a Chrome extension that shows a score by every username on this site grading how well their expressed opinions match what subsequently happened in reality, or the accuracy of any specific predictions they've made. Some people's opinions are closer to reality than others and it's not always correlated with upvotes. An extension of this would be to grade people on the accuracy of th…

The RES (Reddit Enhancement Suite) browser extension indirectly does this for me since it tracks the lifetime number of upvotes I give other users. So when I stumble upon a thread with a user with like +40 I know "This is someone whom I've repeatedly found to have good takes" (depending on the context).

It's subjective of course but at least it's transparently so.

I just think it's neat that it's kinda sorta a loose proxy for what you're talking about but done in arguably the simplest way possible.

Post reply on HN