Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Does history have a replication crisis?

ageofinvention.xyz

Post reply on HN
Print view View original

11–20 of 22 posts

Re: Does history have a replication crisis?

#11

Post by dmvdoug »

defrost wrote: post #9
dmvdoug wrote: post #8

In order to become a history teacher, I had to get a master’s degree in education. I was required to take a minimum of five graduate-level history classes in the history department as part of it. I was transitioning from being a lawyer. Professors would talk about checking citations (although not systematically) but it was more for the purpose of finding useful material you want to use, not to check how they are bein…

As a degree mathematician who has done a little contract work for both historians and lawyers and may have occasionally regarded law as the study of first order logic and theatrics (fun sledge, amusement rather than disrespect intended) I have to ask whether the history crowd had a valid point wrt to strength of reference in supporting an argument. If five decades onwards one were to write a History of Our Times (the…

> If five decades onwards one were to write a History of Our Times (the Central North American Edition) it would be correct to, say, point out that claims were made in the media and courts regarding a stolen election and to then cite court filings, articles in prominent newspapers, and perhaps videos from both network and youtube archives. It's fair to say that such claims formed the zeitgeist of our recent times for a substantial portion of the population affected and that any citation failed to logically support what it purported to support.

If you argued claims were made in the media regarding a stolen election and cited a variety of newspapers and blogs where people talk about how DT thought he won the election because of 10,000 dead people voting in GA or whatever, so he should be President not Biden, how would those citations fail to logically support your argument that claims were made in the media regarding a stolen election?

Re: Does history have a replication crisis?

#12

Post by defrost »

dmvdoug wrote: post #11
defrost wrote: post #9

Earlier quoted context omitted.

As a degree mathematician who has done a little contract work for both historians and lawyers and may have occasionally regarded law as the study of first order logic and theatrics (fun sledge, amusement rather than disrespect intended) I have to ask whether the history crowd had a valid point wrt to strength of reference in supporting an argument. If five decades onwards one were to write a History of Our Times (the…

> If five decades onwards one were to write a History of Our Times (the Central North American Edition) it would be correct to, say, point out that claims were made in the media and courts regarding a stolen election and to then cite court filings, articles in prominent newspapers, and perhaps videos from both network and youtube archives. It's fair to say that such claims formed the zeitgeist of our recent times for…

We're doomed to be in agreement, I'm afraid I have a hard time avoiding passive indirect citing of "factoids" ..

it's absolutely correct to state that "On the New York Post reported that aliens landed in Washington [1]"

it's deceptive (whether with delibrate intent to mislead others or just self deceptive) to state that "Aliens landed in Washington on [1]"

which is one of the root causes of the proliferation of "alternative facts".

It's not just that something is cited, it also matters how it is cited.

[1] link to New York Post article of reporting the landing of aliens.

Re: Does history have a replication crisis?

#13

Post by dmvdoug »

defrost wrote: post #12
dmvdoug wrote: post #11

Earlier quoted context omitted.

> If five decades onwards one were to write a History of Our Times (the Central North American Edition) it would be correct to, say, point out that claims were made in the media and courts regarding a stolen election and to then cite court filings, articles in prominent newspapers, and perhaps videos from both network and youtube archives. It's fair to say that such claims formed the zeitgeist of our recent times for…

We're doomed to be in agreement, I'm afraid I have a hard time avoiding passive indirect citing of "factoids" .. it's absolutely correct to state that "On the New York Post reported that aliens landed in Washington [1]" it's deceptive (whether with delibrate intent to mislead others or just self deceptive) to state that "Aliens landed in Washington on [1]" which is one of the root causes of the proliferation of "alte…

> It's not just that something is cited, it also matters how it is cited.

You called it: I couldn’t agree more!

Re: Does history have a replication crisis?

#14

Post by FreshStart »

dado3212 wrote: post #3

This is kind of an open problem, especially in the Internet age. Any retraction that’s published will have far less reach than the original incendiary claim. And it’s simply not viable to follow that hundred-chain length sourcing tree for everything used in an analysis. Not sure that there’s a solution that’s actually implementable.

Retractions should defacto be capable to create chain reactions?

Re: Does history have a replication crisis?

#15

Post by nextlevelwizard »

Problem is that no one wants to pay you to replicate a study and if you do replicate a study out of your own pocket it is a kin to setting that money on fire career-wise since no one will care.

The whole peer review science scene is all about making money and getting famous. At worse reviewers are trying to nitpick and block their peers long enough to beat them to publication so they can have the claim to fame and thus get more grant money for their next project.

Re: Does history have a replication crisis?

#16

Post by mike_hearn »

The article focuses on the fact that nobody checks citations, but that isn't a problem specific to history. It's more like a general issue with academia. Science papers have the exact same problem. I found during COVID many papers being widely cited in the press as the consensus of experts which had bogus citations in them.

But there are other issues that are actually specific to history. Some years ago I had fun writing a series of essays about these, through the framing of conspiracy theories about history.

Part 1 looks at claims that key historical sources might have been forged during the Renaissance.

https://blog.plan99.net/meta-historical-conspiracies-part-1-...

Part 2 looks at problems with astronomical dating methodologies.

https://blog.plan99.net/meta-historical-conspiracies-part-2-...

Part 3 looks at methods like radiocarbon dating and tree ring counting.

https://blog.plan99.net/meta-historical-conspiracies-part-3-...

The goal here was to describe other people's claims about historical "replication problems" for entertainment purposes rather than try and discern the truth, so I wouldn't argue they're all real. But the trend towards presenting history as scientific doesn't gel well with the culture in the academic humanities. As dmvdoug writes, historians don't really care about even basic things like whether citations are accurate, let alone more debatable things like the accuracy of radiocarbon dating or whether sources have plausible chains of custody. And as the story of Henry Cort demonstrates, historians will happily make up narratives from whole cloth if it advances their ideological agenda, yet nobody within the academy seems to care.

Re: Does history have a replication crisis?

#17

Post by RetroTechie »

dado3212 wrote: post #3

This is kind of an open problem, especially in the Internet age. Any retraction that’s published will have far less reach than the original incendiary claim. And it’s simply not viable to follow that hundred-chain length sourcing tree for everything used in an analysis. Not sure that there’s a solution that’s actually implementable.

Life's too short to check or even verify the many factoids you encounter in daily life. With:

Check = check cited sources yourself. And

Verify = re-do a study yourself. Outside of hard science like physics (and relatively simple experiments), in most cases that's not even possible. Especially history: original artifacts disappear, witnesses die & the places where things happened are changed.

Written records are just that - what was written down. Accurate or not.

This is true for ordinary people as well as scientists. Though I'd expect scientists to make a bit more effort (+ using field-specific knowledge to sort things out).

But news reports tend to be notoriously unreliable imho. Especially popular media or (yuk) social media.

If some science study is reported on, it's often enlightening to read that study yourself. And find it says very different things than the news reports. Or does not even say what's reported elsewhere. Bias, sensationalism & clickbait are a thing.

Re: Does history have a replication crisis?

#18

Post by dado3212 »

FreshStart wrote: post #14
dado3212 wrote: post #3

This is kind of an open problem, especially in the Internet age. Any retraction that’s published will have far less reach than the original incendiary claim. And it’s simply not viable to follow that hundred-chain length sourcing tree for everything used in an analysis. Not sure that there’s a solution that’s actually implementable.

Retractions should defacto be capable to create chain reactions?

But then you have to link everything. Also, not all retractions will invalidate dependencies, so each cascading dependency (that will greatly increase in number) will need to check if the retraction affects their claim. Maybe they modify it. Now a dependency on that has to check if the modified claim still warrants invalidation. This is where the “implementable” problem comes in.

Re: Does history have a replication crisis?

#19

Post by P_I_Staker »

RetroTechie wrote: post #17
dado3212 wrote: post #3

This is kind of an open problem, especially in the Internet age. Any retraction that’s published will have far less reach than the original incendiary claim. And it’s simply not viable to follow that hundred-chain length sourcing tree for everything used in an analysis. Not sure that there’s a solution that’s actually implementable.

Life's too short to check or even verify the many factoids you encounter in daily life. With: Check = check cited sources yourself. And Verify = re-do a study yourself. Outside of hard science like physics (and relatively simple experiments), in most cases that's not even possible. Especially history: original artifacts disappear, witnesses die & the places where things happened are changed. Written records are just…

Yeah, this has been a pet peeve of mine in online conversations where people call for sources. It's pointless, and maybe harmful. You can't just go looking to attach a citation to prove your point, without critical thought.

I've seen countless examples of "citations" that literally contradict their point. I can recall and exchange on reddit were someone smuggly claimed "they use science" when trying to make this crazy claim about how intermittent fasting is good for managing blood sugar and you should skip breakfast.

The citation said that people with blood sugar issues should always eat breakfast, and have small meals throughout the day. How did any of this help us get to truth?

Re: Does history have a replication crisis?

#20

Post by aidenn0 »

dmvdoug wrote: post #8

In order to become a history teacher, I had to get a master’s degree in education. I was required to take a minimum of five graduate-level history classes in the history department as part of it. I was transitioning from being a lawyer. Professors would talk about checking citations (although not systematically) but it was more for the purpose of finding useful material you want to use, not to check how they are bein…

> Suffice to say, I got the message

And this is at least part of why I don't do well in academia (and, tbf, many other settings); I refuse to get the message.

Post reply on HN