Earlier quoted context omitted.
It’s interesting, if you go down near the bottom you see some people with both A’s and D’s. According to the ratings for example, one person both had extremely racist ideas but also made a couple of accurate points about how some tech concepts would evolve.
That is interesting because of the Halo effect. There is a cognitive bias that if a person is right in one area, they will be right in another unrelated area. I try to temper my tendency to believe the Halo effect with Warren Buffett's notion of the Circle of Competence; there is often a very narrow domain where any person can be significantly knowledgeable.
Auto-grading decade-old Hacker News discussions with hindsight
281–285 of 285 posts
Re: Auto-grading decade-old Hacker News discussions with hindsight
#282Earlier quoted context omitted.
It’s interesting, if you go down near the bottom you see some people with both A’s and D’s. According to the ratings for example, one person both had extremely racist ideas but also made a couple of accurate points about how some tech concepts would evolve.
That is interesting because of the Halo effect. There is a cognitive bias that if a person is right in one area, they will be right in another unrelated area. I try to temper my tendency to believe the Halo effect with Warren Buffett's notion of the Circle of Competence; there is often a very narrow domain where any person can be significantly knowledgeable.
> I try to temper my tendency to believe the Halo effect with Warren Buffett's notion of the Circle of Competence; there is often a very narrow domain where any person can be significantly knowledgeable. (commenter above)
Putting aside Buffett in particular, I'm wary of claims like "there is often a very narrow domain where any person can be significantly knowledgeable". How often? How narrow of a domain? Doesn't it depend on arbitrary definitions of what qualifies as a category? Is this a testable theory? Is it a predictive theory? What does empirical research and careful analysis show?
Putting that aside, there are useful mathematical ways to get an idea of some of the backing concepts without making assumptions about people, culture, education, etc. I'll cook one up now...
Start with 70K balls split evenly across seven colors: red, orange, yellow, green, blue, indigo, and violet. 1,000 show up demanding balls. So we mix them up and randomly distribute 10 balls to every person. What does the distribution tend to look like? What particulars would you tune and/or definitions would you choose to make this problem "sort of" map to something sort of like assessing the diversity of human competence across different areas?
Note the colored balls example assumes independence between colors (subjects or skills or something). But in real life, there are often causally significant links between skills. For example, general reasoning ability improves performance in a lot of other subjects.
Then a goat exploded, because I don't know how to end this comment gracefully.
Re: Auto-grading decade-old Hacker News discussions with hindsight
#283This is a cool idea. I would install a Chrome extension that shows a score by every username on this site grading how well their expressed opinions match what subsequently happened in reality, or the accuracy of any specific predictions they've made. Some people's opinions are closer to reality than others and it's not always correlated with upvotes. An extension of this would be to grade people on the accuracy of th…
I like the idea and certainly would try it. Although I feel in a way this would be an anti-thesis to HN. HN tries to foster curiosity, but if you're (only) ranked by the accuracy of your predictions, this could give the incentive to always fall back to a save and boring position.
Even today, "ASI will kill us all" can be a pretty divisive declaration - hardly safe and boring.
From the couple of threads I clicked, it seemed like this LLM-driven analysis was picking up on that, too: the top comments were usually bold, and some of the worst-rated comments was the "safe and boring" declaration that nothing interesting ever really happens.
Re: Auto-grading decade-old Hacker News discussions with hindsight
#284Earlier quoted context omitted.
Didn't Slashdot have something like the second point with their meta-moderation, many many years ago?
Sorta. IIRC, when comment moderation and scoring came to Slashdot, only a random (and changing) selection of users were able to moderate. Meta-moderation came a bit later. It allowed people to review prior moderation actions and evaluate the worth of those actions. Those users who made good moderations were more likely to become a mod again in the future than those who made bad moderations. The meta-mods had no idea…
Re: Auto-grading decade-old Hacker News discussions with hindsight
#285Earlier quoted context omitted.
Sorta. IIRC, when comment moderation and scoring came to Slashdot, only a random (and changing) selection of users were able to moderate. Meta-moderation came a bit later. It allowed people to review prior moderation actions and evaluate the worth of those actions. Those users who made good moderations were more likely to become a mod again in the future than those who made bad moderations. The meta-mods had no idea…
It's still that way today: if you're active, you'll be randomly given 5 moderation points occasionally, and they expire after a few days. So you have to decide which threads and comments are worth spending a "moderation point" on