The trust collapse: Infinite AI content is awful • Hacker News Board

Re: The trust collapse: Infinite AI content is awful

#151

Post by zahlman » Thu, Nov 06, 2025, 4:21 PM UTC

ssalka wrote: post #118

everdrive wrote: post #7

Doesn't matter. We must keep building more and more technology no matter the cost. Have an idea for a business? Build it. Does your business make the lives of people worse? Doesn't matter, keep pushing. Could some new technology ruin the lives and relationships that people have? Doesn't matter, just build it. We always need more, need to do more. Every experiment is valid, every impulse must be followed. More complex…

Eric Weinstein refers to this as an Embedded Growth Obligation (EGO), whereby organizations and economies at large assume perpetual growth, and that things really start to unravel when that growth inevitably slows. It is pretty mindblowing how we have basically accepted growth as the default state, it is not at all a given that things always grow and get better.

> It is pretty mindblowing how we have basically accepted growth as the default state

It is completely to be expected, exactly because it is not new.

It's been scarcely a generation since the peak in net change of the global human population, and will likely be at least another two generations before that population reaches its maximum value. It rose faster than exponentially for a few centuries before that (https://en.wikipedia.org/wiki/World_population#/media/File:P...). And across that time, for all our modern complaints, quality of life has improved immensely.

Of all the different experiences of various cultures worldwide and across recent history, "growth" has been quite probably the most stable.

Culture matters. People's actions are informed by how they are socialized, not just by what they can observe in the moment.

Re: The trust collapse: Infinite AI content is awful

#152

Post by yndoendo » Thu, Nov 06, 2025, 4:22 PM UTC

Anyone else see AI as nothing more than a logging system for citing real world experience and expertise?

In order for any AI/ML content to have value it must cite where the accumulation of information came from. By not doing so it nothing more than a custom Wikipedia-esque with a motto _Trust Me Bro_.

Citations and lack there of should be a simple key factor in evaluating trust. What are your sources for this idea / answer?

Re: The trust collapse: Infinite AI content is awful

#153

Post by saulpw » Thu, Nov 06, 2025, 4:22 PM UTC

wannadingo wrote: post #95

stocksinsmocks wrote: post #64

Earlier quoted context omitted.

5% wrong is an extremely charitable take on the NYT. I once went to a school that had complementary subscriptions. The first time I sat down to read one there was an article excoriating President Bush about hurricane Katrina. The entire article was a glib expansion of an expert opinion who was just some history teacher who said that it was “worse than the battle of Antietam” for America. No expertise in climate. No e…

So since incorporating in 1851, let's say they put out 60,000 issues. 1 issue would represent about 0.002% of their output. How do you get to over 5% wrong?

It's a spot check. They checked one article from one of those issues and they spotted an error, so odds of >5% wrongness are high in their view. (They need a larger sample size and some statistics to make such a claim, but certainly your numbers are way off, but in the other direction.)

Re: The trust collapse: Infinite AI content is awful

#154

Post by pron » Thu, Nov 06, 2025, 4:23 PM UTC

tsunamifury wrote: post #85

pron wrote: post #12

Earlier quoted context omitted.

The problem is that too many people just don't know how to weigh different probabilities of correctness against each other. The NYT is wrong 5% of the time - I'll believe this random person I just saw on TikTok because I've never heard of them ever being wrong; I've heard many stories about doctors being wrong - I'll listen to RFK; scientific models could be wrong, so I'll bet on climate change being not real etc.

Once a problem demands second order thinking you immediately lose a significant portion of the population. It’s simply reality, or else propaganda wouldn’t work so well.

... and since we now know the world is more complex than what we used to think, say, 1000 years ago, this kind of "second-order thinking" is required more and more.

Re: The trust collapse: Infinite AI content is awful

#155

Post by BeFlatXIII » Thu, Nov 06, 2025, 4:23 PM UTC

Ironic coming from a blog about B2B sales. Who was answering cold calls and reading LinkedIn profiles before AI?

Re: The trust collapse: Infinite AI content is awful

#156

Post by saulpw » Thu, Nov 06, 2025, 4:24 PM UTC

wartywhoa23 wrote: post #13

alexpotato wrote: post #8

Yuval Noah Harari, Sapiens fame [0], has a great quote (paraphrasing): Interviewer: How will humans deal with the avalanche of fake information that AI could bring? YNH: The way humans have always dealt with fake information: by building institutions we trust to provide accurate information. This is not a new phenomenon btw. In democracies, this is often either the government (e.g. the Bureau of Labor Statistics) or…

> by building institutions we trust to provide accurate information Except those institutions have long lost all credibility themselves.

Yes, the existing ones were captured, and so we have to build new ones.

Re: The trust collapse: Infinite AI content is awful

#157

Post by zahlman » Thu, Nov 06, 2025, 4:24 PM UTC

pron wrote: post #149

ajaisjsbbz wrote: post #20

Earlier quoted context omitted.

Trust is much more nuanced than N% wrong. You have to consider circumstantial factors as well. ie who runs The NY Times, who gives them money, what was the reason they were wrong, even if they’re not wrong what information are they leaving out. The list goes on. No single metric can capture this effectively. Moreover, the more political a topic the more likely the author is trying to influence your thoughts (but not…

> You have to consider circumstantial factors as well This, too, goes into the probability of something being right or wrong. But the problem I'm pointing out is an inconsistent epistemology. The same kind of test should be applied to any claim, and then they have to be compared. When people trust a random TikToker over the NYT, they're not applying the same test to both sides. > It’s also why things like calling you…

My experience has been that people who trust some form of alternative news over the NYT are not preferring "some random TikToker".

And a lot of the time, that trust is specific to a topic, one which matters to them personally. If they cannot directly verify claims, they can at least observe ways in which their source resonates with personal experience.

Re: The trust collapse: Infinite AI content is awful

#158

Post by heddycrow » Thu, Nov 06, 2025, 4:26 PM UTC

AndrewKemendo wrote: post #145

heddycrow wrote: post #131

Earlier quoted context omitted.

This resonates with me. I'd love to see the machine version or hear more of your thoughts about what goes into it.

If you’re really interested at the furthest depth then take a look at my paper: https://kemendo.com/GTC.pdf If that resonates further let me know at my un on icloud domain

Thanks, I will read. I haven't considered what it's like to be interested at the furthest depth, but I will do that now.

Re: The trust collapse: Infinite AI content is awful

#159

Post by lunar_mycroft » Thu, Nov 06, 2025, 4:27 PM UTC

gnarlouse wrote: post #65

giraffe_lady wrote: post #60

Earlier quoted context omitted.

The paperclip maximizer is us.

Unironically: we worry about the ASI control problem but can't even reign in our billionaires

I look at it more like "the fact that we can't align humans/human institutions strongly suggests we won't be able to align something as alien as AGI is likely to be"

Re: The trust collapse: Infinite AI content is awful

#160

Post by pron » Thu, Nov 06, 2025, 4:28 PM UTC

nsxwolf wrote: post #28

pron wrote: post #12

Earlier quoted context omitted.

The problem is that too many people just don't know how to weigh different probabilities of correctness against each other. The NYT is wrong 5% of the time - I'll believe this random person I just saw on TikTok because I've never heard of them ever being wrong; I've heard many stories about doctors being wrong - I'll listen to RFK; scientific models could be wrong, so I'll bet on climate change being not real etc.

COVID ended my trust in media. I went from healthy skepticism to assuming everything is wrong/a lie. There was no accountability for this so this will never change for me. I am like the people who lived through the Great Depression not trusting banks 60 years later and keeping their money under the mattress.

So the position of a sceptic is epistemologically valid: you distrust any claim that is under, say, 95% certainty. But this bar should be applied consistently, and sometimes you have to bet. For example, in the question of getting a vaccine or not, you must choose, and you should choose whatever claim is more likely to get a better result than the other.

The key is that distrusting one side or source does not logically entail trusting another source more. If you think that the media or medical establishment is wrong, say, 45% of the time, you still have to find a source of information that is only wrong 40% of the time to prefer it.