Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT and the Enshittening of Knowledge

castlebridge.ie

Post reply on HN
Print view View original

271–280 of 301 posts

Re: ChatGPT and the Enshittening of Knowledge

#271

Post by Method-X »

0xBABAD00C wrote: post #75
lubujackson wrote: post #17

If you think of the knowledge base of the internet as a living thing, ChatGPT is a like a virus that now threatens its life. This is the same process SEO spam caused for search - it hampers the nature by which things function and the river needs to reroute (pagerank then usage metadata) to replace the lost signal. ChatGPT is more of an existential threat because it will propagate to infect other knowledge bases. Luke…

> ChatGPT is more of an existential threat because it will propagate to infect other knowledge bases. Luke Wikipedia relies on "published" facts as an authority, but ChatGPT output is going to wind up as a source one way or another. And worse, then ChatGPT will digest its own excrement, worsening its own results further. This is what people do collectively, long before any GPTs were in sight. Lots of strong convictio…

This is so on point. While everyone was arguing that LaMDA couldn't possibly be conscious a few months back, I was asking: what if we're not conscious?

Re: ChatGPT and the Enshittening of Knowledge

#272

Post by venatiodecorus »

munificent wrote: post #166
0xBABAD00C wrote: post #75

Earlier quoted context omitted.

> ChatGPT is more of an existential threat because it will propagate to infect other knowledge bases. Luke Wikipedia relies on "published" facts as an authority, but ChatGPT output is going to wind up as a source one way or another. And worse, then ChatGPT will digest its own excrement, worsening its own results further. This is what people do collectively, long before any GPTs were in sight. Lots of strong convictio…

It's not the same. This is something I've observed many times but have never quite been able to put a name to it. When you lower the friction of an action sufficiently, it causes a qualitative change in the emergent behavior of the whole system. It's like how a little damping means the difference between a bridge you can safely drive over versus a galloping Gertie that resonates until it collapses. When a human has t…

sure, we need dampening in our information systems and our social trust systems. it's clearly not there now. if the problem gets out of hand to the point we're forced to address it, i think that's a good thing overall.

Re: ChatGPT and the Enshittening of Knowledge

#273

Post by khiner »

mkmk3 wrote: post #269
khiner wrote: post #265

Earlier quoted context omitted.

That is true, 3 would help steel my strawman. I agree that we’ll increasingly have capabilities to generate and publish garbage that’s _just_ good enough to generate clicks, and incentives to do this. In addition, I think we’ll increasingly have tools to produce content that is much more rich, imaginative, insightful, and factually correct in our future. Some more interesting questions to me are then: What will the r…

How can fact checking be better facilitated as tech develops? It seems like a distinct social issue to me, one that I imagine requires verification from people with reputation (which could be aided by improved social networks probably). Id be very interested to hear if you have other ideas, as you seem optimistic on this front and would like to share in that :)

I can think of a few directions for technology aiding in fact checking:

1. Much of finding out what’s true or false is about finding consistency amongst lots of observations. So this is the science direction. If you can analyze lots of data, say from first-hand direct measurements like from a scientific instrument, or analyzing second hand observational data, say from many news sources reporting on a political event. One could also imagine multimodal analysis combining these first and second-hand kinds of data to arrive at a consensus estimate of a “true” perspective. E.g. analyzing video and audio streams recorded at said political event, combined with many text reports of the event. So this point is about data mining, and jointly estimating semantic meaning from natural language and other kinds of data, in a way that’s consistent with everything else considered factual.

2. Provenance-tracking: Think block chain - if we can provably trace a piece of data back to its primary sources, tracking all its modifications along the way, this could help with establishing provenance, and verifying legitimacy of any modifications along the way.

3. Consensus, staking/voting, etc. A lot of deciding what’s true is about seeking consensus. One thing I’m generally optimistic about here is that, for any given fact “out there,” there are many more ways to describe it incorrectly than correctly. So even though it sounds scary for consensus to be an aspect of truth finding, it always will be, and at the very bottom it’s all we can hope for. One way that’s already getting traction to make consensus mean something, is the idea of staking. So you have to put something down on the table when you claim you believe something to be true. Software can (and already is) helping to build confidence behind some claims more than others by backing claims with value (money).

4. Humans are insanely bad at reasoning rationally, because of lots of reasons. Pick your favorite fallacy. We evolved to survive long enough to reproduce and rationality is a happy accident. One could imagine software being less susceptible to simple tricks, could be less incentivized to outright lie for personal gain or power seeking, or claiming to represent their actual beliefs when they are actually pursuing other goals by conveying something they don’t actually believe.

Re: ChatGPT and the Enshittening of Knowledge

#274

Post by mountainb »

gptgpp wrote: post #102
tablespoon wrote: post #62

Earlier quoted context omitted.

> Could you elaborate on this? If it is full of garbage a couple examples should be very easy to find. I could give examples but I won't, because that would link my HN and Wikipedia accounts. > So "garbage puffed up beyond all belief" and "full of terrible and fake citations that lead to nowhere" sounds a bit hyperbolic, tbh. People unironically describe it as the "sum of all human knowledge," so it's definitely puff…

I mean... There are lots of ways to browse Wikipedia offline without having your IP recorded, absent using something simpler like a VPN. https://en.wikipedia.org/wiki/Wikipedia:Database_download I can understand if this is just too much effort to put into an online discussion though, I probably wouldn't bother myself. And yeah there have been lots of scandals with Wikipedia. This one was pretty infamous: https://www.…

If we look at the Vulture Bee article, it cites a couple semi-relevant journal articles (which do exist but are not exactly on point for a general citation), but then it inflates the number of citations pointlessly by citing multiple pop news articles that all cite one of the previously cited research papers, and some of which just blogspam link to the other useless popular science magazine citations. https://en.wikipedia.org/wiki/Vulture_bee

In many history articles, there are random citations to web pages without any provenance that claim to be translated documents. Sometimes this is done despite the existence of reliable public databases of such documents available through universities, foundations, and governments. Then there is the link rot problem which gets worse over time.

Re: ChatGPT and the Enshittening of Knowledge

#275

Post by skissane »

Arainach wrote: post #255
awb wrote: post #253

Earlier quoted context omitted.

Yeah, but that’s why you have a “human in the loop”, to handle the infinite number of edge cases. You’d never want end-to-end AI for anything mission critical like justice.

You have a human in the loop explicitly to, in your words, not "enforce universally and consistently". * Most people agree that stealing from a store is wrong. * Most people agree that opening food/medicine and consuming it in the store before paying is stealing * Most people believe that helping those in a medical emergency is important If I was in a store and saw someone going into hypoglycemia and grabbed a candy…

> Most people agree that opening food/medicine and consuming it in the store before paying is stealing

In my jurisdiction, that is only stealing if you do it with intention of not paying for it.

Sometimes I go to the supermarket, pick a drink off the shelf, start drinking it, take the partially drunk (or sometimes completely empty) bottle to the checkout to pay. Never got in trouble, staff have never complained - I know the law is on my side, and pretty confident the staff training tells them the same thing.

Re: ChatGPT and the Enshittening of Knowledge

#276

Post by tarsinge »

dr_dshiv wrote: post #215
ethanbond wrote: post #121

Earlier quoted context omitted.

Sure but humans can’t do it at nearly the rate that GPT can, and GPT will never be applying critical thought to the memes it digests and forward on while humans sometimes do.

> and GPT will never be applying critical thought Just wait. This party has just begun. 90% of humans will be like school kids in comparison to the critical thought of GPT-5.

We are talking about a model that at its core is about making statistics of what the next word will be in a sentence based on an existing corpus. It gives that model the ability to find and summarize all of the existing content in relation to a prompt beyond what humans could do, but I still see no critical thinking there.

Re: ChatGPT and the Enshittening of Knowledge

#277

Post by Forgeties79 »

skissane wrote: post #275
Arainach wrote: post #255

Earlier quoted context omitted.

You have a human in the loop explicitly to, in your words, not "enforce universally and consistently". * Most people agree that stealing from a store is wrong. * Most people agree that opening food/medicine and consuming it in the store before paying is stealing * Most people believe that helping those in a medical emergency is important If I was in a store and saw someone going into hypoglycemia and grabbed a candy…

> Most people agree that opening food/medicine and consuming it in the store before paying is stealing In my jurisdiction, that is only stealing if you do it with intention of not paying for it. Sometimes I go to the supermarket, pick a drink off the shelf, start drinking it, take the partially drunk (or sometimes completely empty) bottle to the checkout to pay. Never got in trouble, staff have never complained - I k…

If you’re depending on sussing out peoples intent then you’re accepting that we can’t be clear/zero tolerance about it. If you catch me stealing and I just go “oh no dude, I was totally going to pay” but you don’t believe me, what then? You can’t possibly know what my actual intention was.

Re: ChatGPT and the Enshittening of Knowledge

#278

Post by skissane »

Forgeties79 wrote: post #277
skissane wrote: post #275

Earlier quoted context omitted.

> Most people agree that opening food/medicine and consuming it in the store before paying is stealing In my jurisdiction, that is only stealing if you do it with intention of not paying for it. Sometimes I go to the supermarket, pick a drink off the shelf, start drinking it, take the partially drunk (or sometimes completely empty) bottle to the checkout to pay. Never got in trouble, staff have never complained - I k…

If you’re depending on sussing out peoples intent then you’re accepting that we can’t be clear/zero tolerance about it. If you catch me stealing and I just go “oh no dude, I was totally going to pay” but you don’t believe me, what then? You can’t possibly know what my actual intention was.

The physical design of the store makes it clear in most cases. The checkouts form a physical barrier between the “haven’t paid yet” area and the “have paid“ area. It is difficult to assume an attempt to steal in the former, much easier once one passes to the later with unpaid goods.

The legal definition of theft - at least where I live - is all about intention. It involves an intention to deprive another of their property. No intention, no theft. If you absent-mindedly walk out of a store without paying for something, no theft has occurred. When our kids were babies, we used to put the shopping in the pram. One day I left the supermarket and down the street discovered a loaf of bread in a different section of it, that I’d forgotten to pay for. I went back and explained myself to the security guard, did he call the police? No, he commended me for my honesty, and let me pay for it with the self-serve checkouts.

For a supermarket, their biggest concern with theft is the repeat offenders. If it is an unclear situation, it is in their best interest to give the customer the benefit of the doubt. But, if the same unclear situation happens again and again, that’s when the intent (which is legally required to constitute stealing) becomes obvious. Ultimately though, it is up to the store staff, police, prosecutors and magistrates to apply a bit of common sense in deciding what is likely to be intentional and what likely isn’t. But yes, given theft is defined in terms of inferring people’s intentions, “zero tolerance” is a concept of questionable meaningfulness in that context.

Re: ChatGPT and the Enshittening of Knowledge

#279

Post by Forgeties79 »

skissane wrote: post #278
Forgeties79 wrote: post #277

Earlier quoted context omitted.

If you’re depending on sussing out peoples intent then you’re accepting that we can’t be clear/zero tolerance about it. If you catch me stealing and I just go “oh no dude, I was totally going to pay” but you don’t believe me, what then? You can’t possibly know what my actual intention was.

The physical design of the store makes it clear in most cases. The checkouts form a physical barrier between the “haven’t paid yet” area and the “have paid“ area. It is difficult to assume an attempt to steal in the former, much easier once one passes to the later with unpaid goods. The legal definition of theft - at least where I live - is all about intention. It involves an intention to deprive another of their pro…

“I forgot it was in my pocket.”

And yes, I do realize that intention is part of the law. That wasn’t what I was saying really. I am saying that because we have that, we are implicitly accepting that a lot of this stuff cannot be ironclad. There has to be room for interpretation and enforcement.

Re: ChatGPT and the Enshittening of Knowledge

#280

Post by dennis_jeeves1 »

93po wrote: post #246
brap wrote: post #221

Earlier quoted context omitted.

What’s stopping you from using this token with a bot?

I disagree with top comment a bit - I think we need tokens that are authenticated to an identity much like SSLs certs. If you choose to use your token to post AI generated crap on the internet, I can choose to block it. I can also choose to subscribe to community managed block lists. I think content on the internet needs to be reputation based, and if you choose to read content that isn't signed by a real person's id…

> If you publicly support Jordan Peterson

Do I have to reveal my actual name?

Post reply on HN