Live data from Hacker News

ChatGPT and the Enshittening of Knowledge

castlebridge.ie

111–120 of 301 posts

Re: ChatGPT and the Enshittening of Knowledge

#111

ChatGPT is the bell announcing the death of anonymity on the web. Anonymous content will be generated content, real discussion will shift to gated communities with either paywalls or proofs of identity.

But we also can no longer trust that information published by a verified human was actually created by a human.

Re: ChatGPT and the Enshittening of Knowledge

#112

If you think of the knowledge base of the internet as a living thing, ChatGPT is a like a virus that now threatens its life. This is the same process SEO spam caused for search - it hampers the nature by which things function and the river needs to reroute (pagerank then usage metadata) to replace the lost signal. ChatGPT is more of an existential threat because it will propagate to infect other knowledge bases. Luke…

"If you think of the knowledge base of the internet as a living thing, ChatGPT is a like a virus that now threatens its life."

I am not sure, if the situation is that dramatic, but just wait, until advertisers finds a way to get their "data" into ChatGPT results (or alike).

Then things will get really ugly.

So yes, this is what we will have to do:

"All signs point to this strengthening the value of curation and authenticated sources. "

Re: ChatGPT and the Enshittening of Knowledge

#113
post #70

Earlier quoted context omitted.

>And worse, then ChatGPT will digest its own excrement, worsening its own results further I wonder if we'll get a "dead sea effect" with AI, I've seen some stuff saying they've basically run out of high quality training data and now the training pool will get poisoned by AI generated shit. Basically garbage in, garbage out and these large language models might not be able to improve

Maybe, but there are something like 700,000 books published on average each year, and almost 2 million scientific journal articles. Let's not even consider newspaper. Of course, some of those books will definitely be AI generated or garbage quality, and we all know many of those journal articles can be worth less than the paper they're printed on. Yet even if we cut it down to 100,000 books and half a million scienti…

Journal articles (and newspapers) are also plagued by bullshit.

Humans are capable of producing intelligently sounding word salads just like chatgpt can.

Re: ChatGPT and the Enshittening of Knowledge

#115

Earlier quoted context omitted.

Were going to have shady agencies in 5 years advertising: "get ChatGPT to respond to questions with your content for only $199"

“Detect ChatGPT responses, only $249” It’s just a new chapter in the arms race of spam techniques vs detection. Lots of money presumably made selling both sides.

GPT has a hidden steganographic watermark in its output, so the arms race should be one-sided.

Re: ChatGPT and the Enshittening of Knowledge

#116

If you think of the knowledge base of the internet as a living thing, ChatGPT is a like a virus that now threatens its life. This is the same process SEO spam caused for search - it hampers the nature by which things function and the river needs to reroute (pagerank then usage metadata) to replace the lost signal. ChatGPT is more of an existential threat because it will propagate to infect other knowledge bases. Luke…

> ChatGPT is a like a virus that now threatens its life. Perhaps something needs to be disrupted. The Internet is nothing like what it was 20 years ago, It turned into a bunch of social media walled gardens and SEO spam. ChatGPT is like fresh air because it can actually answer questions in a no-nonsense way without users having to scroll through 5-6 spam websites, paywalls, and crappy user interfaces to get an answer…

Agents like this under the control of users would be pretty great. Man, would companies ever hate if we could use the kinds of tools they use against us, against them. No more shopping for the best price: "ChatGPT, what's the lowest price on a new X, brand Y, model Z? And give me the URL to the product page." No more burning our human time talking to companies' robots: "ChatGPT, get through this shitty phone tree and let me know when you have a person". A true digital assistant. Couple with crowd-sourced data (receipt scanning, junk-mail grocery flier scanning, or the AR goggles that are probably not that far off) and you could even do stuff like have it plot optimal IRL grocery shopping for you (lowest total price on this list of goods, value my time at $X/hr, and factor in cost of transportation... and also ChatGPT assembled the list for me in the first place, because I had it create this week's dinner menu)

ChatGPT as a service that can be used to mislead us to trick us out of our money on the behalf of megacorps, like the entire rest of the Web has become? To "promote" things to us against our interests? Meh. Call me when it's mine and will obey me and will never lie to me or serve someone else's priorities over mine, and I'll be interested.

Re: ChatGPT and the Enshittening of Knowledge

#117
Saw a silly “recommended” video on YouTube of Jordan Peterson talking about ChatGPT.

“It knows everything. It’s smart. It’s smarter than you!”

What a crackpot, talking about stuff he doesn’t understand for clicks. Sadly it works for the lowest common denominator

Re: ChatGPT and the Enshittening of Knowledge

#119

If you think of the knowledge base of the internet as a living thing, ChatGPT is a like a virus that now threatens its life. This is the same process SEO spam caused for search - it hampers the nature by which things function and the river needs to reroute (pagerank then usage metadata) to replace the lost signal. ChatGPT is more of an existential threat because it will propagate to infect other knowledge bases. Luke…

I don't really understand this hypothesis as it assumes that information quality of AI generated content on the internet will drop as a result of ChatGPT, not increase.

The way I see it is that ChatGPT isn't the only tool out there that can create spam and junk content. The only difference is that ChatGPT produces something that's of a high enough quality that it's not as easy for a human to easily classify it as spam. And something you can't easily classify as spam arguably isn't spam.

If you assume that those incentivised to create spam today are creating spam anyway and all ChatGPT will do is allow spammers to create better spam then I don't see why the quality of content online would necessarily drop because of ChatGPT - you might actually find that what was once just spam is actually kinda interesting all of a sudden.

But it's not just the quality of AI spam that will increase with ChatGPT... Consider BuzzFeed... Arguably they're just paying people to write trash content today. And this is very common. Most companies have a blog where they pay someone to write mostly junk content just for SEO. I think ChatGPT might actually produce higher quality content that what is currently being written at places like Buzzfeed and on junk blogs. Or at least these workers now have a tool to write something that's higher quality.

I think the only way you're correct is if ChatGPT were to greatly increase the incentive to publish spam, resulting in a much greater amount of spam that counteracts the positive improvement in spam quality. And although I think it probably will increase the number people producing spam content to some extent I doubt it will have a net-negative impact.

Finally, I think what you'll see happen in future iterations of ChatGPT to improve quality and accuracy is that content will be fed in weighted by how authoritative the source is. This spam singularity that some are predicting, where the prior generation of spam bots produce the content that trains future generations of spam bots makes no sense given these companies are trying to create AI that doesn't just spit out spam and inaccurate information.

Re: ChatGPT and the Enshittening of Knowledge

#120
The test use case of constructing a bio for yourself, hoping it accurately summarizes all the extremely low sample size data it happens to have of you in its web crawled training data, seems like one of the worst possible use cases for ChatGPT. It’s right there on the main page that it’s not to be trusted with factual information like this. ChatGPT will hallucinate details. It’s remarkable to me actually how often it will refuse to hallucinate, given that’s basically what its job is. I don’t find it interesting to find all these edge cases where ChatGPT produces empirically false data. It doesn’t even have the ability to look things up! If I were the OP and wanted help writing my bio, I would first write the draft myself, then use ChatGPT to help with the editing, prose, grammar, style, etc. You are the expert on the factual details of your own life, and if you’re surprised that a language model trained on web crawled data ending in 2018 is not, then all I’ve learned is that you don’t know much about what this thing is.

I also don’t buy these arguments of the form, 1. OpenAI’s public ChatGPT app is often factually inaccurate. 2. ChatGPT is an example of a ML system bootstrapped on web crawled text data. 4. Thus, the long term future of our distributed text-encoded knowledge base will be a cesspool of useless gobbledygook.

ChatGPT is a step forward in generative language modeling. It doesn’t preclude the development of other future systems to help us verify factual accuracy of claims, likely much better than humans can. We’ll be ok gang:)

Post reply on HN