Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Check your fucking sources, people

brodzinski.com

Post reply on HN
Print view View original

31–40 of 88 posts

Re: Check your fucking sources, people

#31

Post by wing-_-nuts »

6stringmerc wrote: post #4
wing-_-nuts wrote: post #2

Ironically, 'source checking' is something AI is quite good at.

Citation needed, please

Personal experience? You ask it for the name of the paper referenced. You google that paper (for some reason it's not great at going out and acquiring the paper). You then upload the pdf and ask it if the paper supports the assertion if it's not quickly findable via ^F. You go read, ask it clarifying questions about hazard ratios, what they controlled for, etc.

AI is quite good when grounded in a source.

Re: Check your fucking sources, people

#32

Post by Kim_Bruning »

throw310822 wrote: post #21
flail wrote: post #9

Earlier quoted context omitted.

There's nuance to that. An LLM is quite capable of suggesting relevant reading, given the context. Especially when the context is broad enough that there's enough training data. "Find me research on code reviews, their size, and quality" would give you more than enough reading. Yet, if you start with a claim, like "Longer PRs mean worse defect detection," the relevant data points fall to few enough for AI to start ha…

Checking is different from finding, though. Source checking means just "verify that this information is actually present in that document". Much harder to hallucinate in this case.

A quick smoke check takes just a few minutes.

"Follow each link in this document. Read each link's contents against the contents in this document. Create a report: for each link list a working hyperlink, whether it exists, what claim it supports, whether it supports or fails to support it, and why"

If it returns a report claiming all correct? That's promising, but human verification is important. You've got a list of hyperlinks, and a list of claims; so you can click each with middle-mouse, Ctrl-F 'till you find the point, and close the tab when you do.

If you find any discrepancies ? Your initial prompt was malformed and/or you picked the wrong LLM, the wrong human, or possibly all three. Whatever the way, the results are built on quicksand; you'll need to start over.

If no sources are provided? Well now: "If there ain't no sources it never happened."

Compare double-entry bookkeeping. It needs to all add up. If you're 1 cent off, that means something is broken. Idem if a single reference is off, it polluted the context. (This works for human-generated and hybrid documents too. Polluted reasoning is polluted reasoning. The process is what counts.)

Re: Check your fucking sources, people

#33

Post by arjie »

You always have to check your sources because citation laundering is a thing[0].

In addition, most mainstream[1] journalists cite sources in a more liberal way than a scientist should so the source might not say what the journalist reports. The Atlantic has a bit on Waymo’s poor detection of minorities[2], e.g.

0: https://wiki.roshangeorge.dev/w/Blog/2026-01-17/Citogenesis

1: Some independent reporters like Matt Yglesias are more rigorous, though their direct reporting can still be bogus

2: https://www.theargumentmag.com/p/no-waymos-arent-racist

Re: Check your fucking sources, people

#34

Post by 1970-01-01 »

wing-_-nuts wrote: post #2

Ironically, 'source checking' is something AI is quite good at.

I disagree. It is a bullshit machine all the way to the core. LLMs in my world fail to cite full sources and consistently conclude with guesses as facts. It does this much more than an average journalist or reporter would. Only when you double-check it will it then apologize and correct itself.

Re: Check your fucking sources, people

#35

Post by wat10000 »

tinfoilhatter wrote: post #27
wat10000 wrote: post #22

Earlier quoted context omitted.

They are generally quite good, and they provide ample background info for you to replicate (or repudiate) their findings on your own if you're so inclined. What's amazing is that people think Snopes or other fact-checkers are automatically wrong. I assume this comes from people who make a habit of believing bullshit and can't handle being corrected.

When there is no independent media, it's not difficult to find sources that back up the lies that Snopes and other fact-checkers peddle. https://fair.org/home/the-digital-media-oligarchy-who-owns-o... https://swprs.org/the-american-empire-and-its-media/

What lies exactly?

Re: Check your fucking sources, people

#36

Post by mountainb »

louiereederson wrote: post #13

Late last year I tried asking ChatGPT to summarize a collection of 10 researchers' views/findings on a topic and provide representative quotes. It initially looked plausible but when I checked the links, the quotes were from clearly AI generated summaries of actual interviews. The paraphrasing was also plausible but subtly and profoundly incorrect. I haven't tested this again on the latest models though, so not sure…

That's more or less how it works. To actually have the system carry out your intention it would have to use significant hardware resources (and even then who knows if it would actually work). Alternatively you would need to break up the work into chunks that the hardware allocated to you by the system would not be overwhelmed.

A lot of people don't realize this because the work that they are having the AI do does not need to be either true or false. It just has to output media that seems like it fits. The system probably took many shortcuts to keep the resource use low while outputting something plausible but false.

And frankly this is sort of fine as long as you know what it's doing and what the limitations are. Hypothetically if you broke up the task into multiple steps that the system can actually ingest properly it might reduce the time that the task took overall, maybe even significantly, but not down to one prompt.

Re: Check your fucking sources, people

#37

Post by arjie »

ubertaco wrote: post #24
6stringmerc wrote: post #3

Also relevant: the derision and mockery directed at JD Vance as a “couch fucker” even used by John Oliver. I read “Hillbilly Elegy” and wondered why it wasn’t in there. Snopes cleared it up in a matter of minutes. Why he hasn’t sued people into oblivion is his prerogative, but it’s a fascinating case study that we are, indeed, living in a Post-Truth environment.

You're getting downvotes because the target of this particular lie was a known liar, so people probably feel like it's some sort of poetic justice (or they know it's just in-kind retaliation and are cathartically satisfied by it). I don't think the right answer to widespread disinformation campaigns is retaliatory disinformation campaigns (even if they're couched – pun not intended – in a just-barely-thin-enough veil…

I’m with you. The net effect actually is something akin to honking one’s horn at a guy who honked at you. You think you’re giving him a taste of his own medicine, but walking by I only see two people honking their horn and I’d ideally prefer not to be around the horn honkers since they’re unpleasant.

Re: Check your fucking sources, people

#38

Post by TZubiri »

>A clearly AI-generated image didn’t help the credibility (a three-legged crow is quite telling)

Actually I checked some sources, and I found some for three-legged crows:

https://en.wikipedia.org/wiki/Kojiki#The_Nakatsumaki_(%E4%B8...

https://en.wikipedia.org/wiki/Three-legged_crow#/media/File:...

https://en.wikipedia.org/wiki/File:Douze_emblemes_des_rites_...

https://en.wikipedia.org/wiki/File:Chengdu_2007_341.jpg

And by refuting this article, I thereby prove that which it sought to refute.

Re: Check your fucking sources, people

#39

Post by dugidugout »

vslira wrote: post #28

People like to blame social media for this kind of bullshit, but social media is just the vector. Just this week I read a "study" because someone claimed on social media that it was made by (Public, famous) Unis A, B and C and reported as an effect an increase in 30% of revenue for the companies that participated in the experiment. The "study" was commissioned by an interest group (bad sign). It was conducted by peop…

Why not link the study?

Re: Check your fucking sources, people

#40

Post by TZubiri »

SllX wrote: post #26

> Not Sweden, but one Swedish startup. Just as an aside jumping off this sentence from the article, I am far less tolerant of the practice of naming countries of origin or general locales rather than specific organizations in headlines and stories. Name the organization, and if you want to in the body, name where they’re from/located/operating as it pertains to the organization. For that matter, if you can offer info…

There's even arguments for doing this even for cases where the actual state entity did something.

"The US did X" The president? The senate? A federal, or municipal body? etc..

But there's arguments against, if "The US bans automatic rifles" then to some extent it's clear what part of the US did it, to some other extent, it doesn't matter, and to some other extent, the part of the country that did the thing represents the whole country by corporization or democritazation.

In History it's very common to say Country did thing, "Germany invaded Poland", "Argentina signed the Roca-Runceman pact" and so on... Possibly because (in addition to the reasons stated above) information needs to be compressed more for the past, we have less space and priority for details of the past than we do for the present, a kind of cold-hot storage mechanism

Post reply on HN