Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Auto-grading decade-old Hacker News discussions with hindsight

karpathy.bearblog.dev

Post reply on HN
Print view View original

41–50 of 285 posts

Re: Auto-grading decade-old Hacker News discussions with hindsight

#41

Post by collinmcnulty »

doctoboggan wrote: post #32
collinmcnulty wrote: post #23

> But if intelligence really does become too cheap to meter, it will become possible to do a perfect reconstruction and synthesis of everything. LLMs are watching (or humans using them might be). Best to be good. I cannot believe this is just put out there unexamined of any level of "maybe we shouldn't help this happen". This is complete moral abdication. And to be clear, being "good" is no defense. Being good often…

I've had the same though as Karpathy over the past couple of months/years. I don't think it's good, exciting, or something to celebrate, but I also have no idea how to prevent it. I would read his "Best to be good." as a warning or reminder that everything you do or say online will be collected and analyzed by an "intelligence". You can't count on hiding amongst the mass of online noise. Imagine if someone were to co…

This is my plan at least

1. Don't build the Torment Nexus yourself. Don't work for them and don't give them your money.

2. When people you know say they're taking a new job to work at Torment Nexus, act like that's super weird, like they said they're going to work for the Sinaloa cartel. Treat rich people working on the Torment Nexus like it's cringe to quote them.

3. Get hostile to bots. Poison the data. Use AdNauseum and Anubis.

4. Give your non-tech friends the vague sense that this stuff is bad. Some might want to listen more, but most just take their sense of what's cool and good from people they trust in the area.

Re: Auto-grading decade-old Hacker News discussions with hindsight

#43

Post by consumer451 »

jeffbee wrote: post #34
latexr wrote: post #31

Earlier quoted context omitted.

> for some reason, whole conversations get reset to a single timestamp. What do you mean?

There is some action that moderators can take that throws one of yesterday's articles back on the front page and when that happens all the comments have the same timestamp.

I believe that this is called "the second chance pool." It is a bit strange when it unexpectedly happens to one's own post.

Re: Auto-grading decade-old Hacker News discussions with hindsight

#44

Post by mtlynch »

GaggiX wrote: post #42

I was reading the Anki article on 2015-12-13, and the best prediction was by markm248 saying: "Remember that you read it here first, there will be a unicorn built on the concept of SRS" They were right, Duolingo.

Duolingo existed for a while at that point and was already valued at $500M by end of 2015.

Re: Auto-grading decade-old Hacker News discussions with hindsight

#47

Post by brian_spiering »

MBCook wrote: post #17
GaggiX wrote: post #12

I think the most fun thing is to go to: https://karpathy.ai/hncapsule/hall-of-fame.html And scroll down to the bottom.

It’s interesting, if you go down near the bottom you see some people with both A’s and D’s. According to the ratings for example, one person both had extremely racist ideas but also made a couple of accurate points about how some tech concepts would evolve.

That is interesting because of the Halo effect. There is a cognitive bias that if a person is right in one area, they will be right in another unrelated area.

I try to temper my tendency to believe the Halo effect with Warren Buffett's notion of the Circle of Competence; there is often a very narrow domain where any person can be significantly knowledgeable.

Re: Auto-grading decade-old Hacker News discussions with hindsight

#48

Post by GaggiX »

mtlynch wrote: post #44
GaggiX wrote: post #42

I was reading the Anki article on 2015-12-13, and the best prediction was by markm248 saying: "Remember that you read it here first, there will be a unicorn built on the concept of SRS" They were right, Duolingo.

Duolingo existed for a while at that point and was already valued at $500M by end of 2015.

It became a unicorn in December 2019 tho, 4 years later.

Re: Auto-grading decade-old Hacker News discussions with hindsight

#49

Post by hackthemack »

I noticed the Hall of Fame grading of predictive comments has a quirk? It grades some comments about if they came true or not, but in the grading of comment to the article

https://news.ycombinator.com/item?id=10654216

The Cannons on the B-29 Bomber "accurate account of LeMay stripping turrets and shifting to incendiary area bombing; matches mainstream history"

It gave a good grade to user cstross but to my reading of the comment, cstross just recounted a bit of old history. The evaluation gave cstross for just giving a history lesson or no?

Re: Auto-grading decade-old Hacker News discussions with hindsight

#50

Post by slg »

This is a perfect example of the power and problems with LLMs.

I took the narcissistic approach of searching for myself. Here's a grade of one of my comments[1]:

>slg: B- (accurate characterization of PH’s “networking & facade” feel, but implicitly underestimates how long that model can persist)

And here's the actual comment I made[2]:

>And maybe it is the cynical contrarian in me, but I think the "real world" aspect of Product Hunt it what turned me off of the site before these issues even came to the forefront. It always seemed like an echo chamber were everyone was putting up a facade. Users seemed more concerned with the people behind products and networking with them than actually offering opinions of what was posted.

>I find the more internet-like communities more natural. Sure, the top comment on a Show HN is often a critique. However I find that more interesting than the usual "Wow, another great product from John Developer. Signing up now." or the "Wow, great product. Here is why you should use the competing product that I work on." that you usually see on Product Hunt.

I did not say nor imply anything about "how long that model can persist", I just said I personally don't like using the site. It's a total hallucination to claim I was implying doom for "that model" and you would only know that if you actually took the time to dig into the details of what was actually said, but the summary seems plausible enough that most people never would.

The LLM processed and analyzed a huge amount of data in a way that no human could, but the single in-depth look I took at that analysis was somewhere between misleading and flat out wrong. As I said, a perfect example of what LLMs do.

And yes, I do recognize the funny coincidence that I'm now doing the exact thing I described as the typical HN comment a decade ago. I guess there is a reason old me said "I find that more interesting".

[1] - https://karpathy.ai/hncapsule/2015-12-18/index.html#article-...

[2] - https://news.ycombinator.com/item?id=10761980

Post reply on HN