Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT and the Enshittening of Knowledge

castlebridge.ie

Post reply on HN
Print view View original

231–240 of 301 posts

Re: ChatGPT and the Enshittening of Knowledge

#231

Post by layer8 »

3princip wrote: post #142
lubujackson wrote: post #17

If you think of the knowledge base of the internet as a living thing, ChatGPT is a like a virus that now threatens its life. This is the same process SEO spam caused for search - it hampers the nature by which things function and the river needs to reroute (pagerank then usage metadata) to replace the lost signal. ChatGPT is more of an existential threat because it will propagate to infect other knowledge bases. Luke…

A-grade bullshitter as the article puts it is pretty accurate. Thought I would test it and just asked ChatGPT if it knew the Voyager episode "11:59", the answer got everything wrong. Season, number and date, all incorrect. >"11:59" is an episode of the science fiction television series Star Trek: Voyager. The episode originally aired on February 9, 2000 as the 11th episode of the sixth season.

What’s more, ChatGPT also “knows” the Voyager episodes “10:59” and “12:59”, when individually asked.

On the other hand:

$ Are there Voyager episodes titled “10:59”, “11:59”, or “12:59”?

There are no episodes of Voyager titled "10:59", "11:59", or "12:59".

Re: ChatGPT and the Enshittening of Knowledge

#232

Post by yamtaddle »

wadd1e wrote: post #211
yamtaddle wrote: post #116

Earlier quoted context omitted.

Agents like this under the control of users would be pretty great. Man, would companies ever hate if we could use the kinds of tools they use against us, against them. No more shopping for the best price: "ChatGPT, what's the lowest price on a new X, brand Y, model Z? And give me the URL to the product page." No more burning our human time talking to companies' robots: "ChatGPT, get through this shitty phone tree and…

I wish this sort of thing was realistically possible; I think history has taught us that we cannot have nice things one too many times to pretend like this time is any different. Whoever owns ChatGPT/equivalent product in the future would probably end up doing something or the other to ruin this idea. "GhatGPT, what's the cheapest restaurant in this area?" has way too much advertisement potential to be left alone.

Right, ChatGPT without complete loyalty to the user runs into the same problem as auto-restocking schemes from Amazon and such: I can't trust that it's not fucking me, so I have to check manually anyway, at least from time to time. At that point, I may as well just go buy the thing I need when I need it, myself. If, when I ask ChatGPT (or its future, improved successor) to explain the benefits and drawbacks of the best products in some category, at three price points, I have to worry that placement on that list can be bought... then the whole thing's pointless as a tool for "consumers". Just another avenue for tricking us out of our money, and we're already very well served in that department, don't need any more of that, thanks.

Re: ChatGPT and the Enshittening of Knowledge

#233

Post by aimor »

gptgpp wrote: post #46
tablespoon wrote: post #31

Earlier quoted context omitted.

> It's true though, Wikipedia really is terrible and full of fake citations that lead nowhere. It's an anti-knowledge base that sometimes has good information. Yeah, Wikipedia is garbage puffed up beyond all belief. I literally just today saw something just like you describe. It should be viewed very skeptically on anything anyone disagrees over (because then it's just snapshots of an agenda-pushing battle).

Could you elaborate on this? If it is full of garbage a couple examples should be very easy to find. I completely agree that Wikipedia can have errors, but in topics that I am educated in it seems pretty decent and I can't remember the last time I came across any (comp sci for example). The most recent example I can think of is about is an article on vulture bees, and a citation about what their honey tastes like, wh…

While claims of Wikipedia's awfulness may be overstated, I do see a lot of problems. And while I am picking on Wikipedia I don't think it's useless, but it does require caution.

The last Wikipedia page I visited ( Elder_Mother ) someone had, years ago, removed all of the citations for the article. These were websites that contained much more and higher quality content than the Wiki page itself, and had been cited with the original page creation. I only found the citations by chance, because I decided to look at the page's history. This poor curation isn't just bad for the usefulness of Wikipedia, it's borderline plagiarism since the entire article was composited from paraphrasing.

Before that I saw a Wikipedia page ( The Voyage of Life ) that admitted its own plagiarism. The page had a big disclaimer at the top: "This page might contain plagiarism" but more delicately worded. So somebody noticed the verbatim plagiarism, added a flag, and then nothing.

Another issue is the lack of expertise, which leads to misleading wishy-washy statements. The page for slugs, talking about control, says crushed eggshells, "are generally ineffective on a large scale, but can be somewhat useful in small gardens." This is false, eggshells are ineffective in all gardens. But to avoid edit wars the language has to pussyfoot around sensitive topics like gardening advice.

Stemming from the lack of expertise, Wikipedia itself becomes out of date without curation. The problem is while it claims to be more up-to-date than printed media, there's no easy way to identify how significant the information on a page is. If I go to an article am I reading things that were written 20 years ago or 2 years ago? Is the material presented relevant in 2023? Was it ever significant to begin with, or did the author happen to have knowledge and interest in something obsolete?

Most pages are also, I think, poorly organized ( Partial differential equation ). I believe a single voice and more effort to write articles for a well defined audience would help immensely, specifically with math and science pages. Wikipedia keeps trying to condense complex material from a textbook into an encyclopedia article format, and it's not working out.

Re: ChatGPT and the Enshittening of Knowledge

#234

Post by burnished »

carapace wrote: post #159

cf. xkcd/810 "Constructive" https://xkcd.com/810/ - - - - Schimidhuber says that his task is "to create an automatic scientist, and then retire." Not long ago it was mildly insulting for someone to suggest that your writing sounded like the output of GPT, already (for most of us) it has become mildly complementary. GPT may be hallucinating, but it writes well. So what if you connect it to empirical feedback? Make it…

Chat gpt is a terrible writer. At least, every example I have seen has had poor information density and was generally worse than the input prompts people used when they wanted to 'fluff up' a statement or opinion. I would be genuinely interested in a counter example.

Side note - your main point seems overly optimistic. How would we recognize, value, or design a 'sane' machine when by your argument we dont have access to sanity? Seems way more likely to generate a distillation of our neuroses.

Re: ChatGPT and the Enshittening of Knowledge

#235

Post by lsy »

ptx wrote: post #89
visarga wrote: post #38

Earlier quoted context omitted.

One approach to fixing factual errors is to use two rounds of LLM interaction. I forgot the name of the paper. Say you ask "What is the height of Everest?" 1. generate an answer in closed-book model, with the LLM: "The height of Everest is 8723m" = candidate_answer 2. search your references with candidate_answer, find: "At 8,849 meters (29,032 feet), Everest is considered the tallest point on Earth" = search_snippet…

How would you apply this approach to the article's example, the response to the question "Who is Daragh O Brien from Castlebridge", where I count at least 15 separate statements of fact? Should we research all of them and try again with a big table of hits and misses from the first attempt? Seems like a lot of work. Also: Is the generated response really "very good at matching the correct answer"? I suppose it would…

Yes, this is the essential issue. Any system competent to fully ground and check factual statements in a stream of arbitrary text will be phenomenally more complex than the original LLM, and usually will be able to answer queries directly, at which point one wonders what the LLM is adding. I think at best, if we can somehow identify all factual statements that need to be cross-referenced and then offload them to a knowledge base (dubious), we are left this kind of mad-libs connective flow that the LLM has created which approximates the essay style of a human writer. I'm not certain that has much practical value besides allowing for a form of undetectable plagiarism to be published as though it were free-form writing.

Re: ChatGPT and the Enshittening of Knowledge

#236

Post by pugworthy »

phphphphp wrote: post #105
userabchn wrote: post #76

A friend is a member of quite a few Facebook groups. She doubts my belief that these groups already contain ChatGPT-like influence operation chatbots, but was recently surprised when in one of the groups several participants discussed a photograph of a typical large chest of drawers while referring to it as a "locker". Maybe some people do refer to such furniture as a locker, or maybe the first person did for some re…

this is a long-standing meme: any behaviour you don’t understand on the internet must be some sort of sophisticated operation by a nefarious third-party network. The question you should always start with is why and if you can’t come up with a why then you should start from a more neutral position on the behaviour. Spam isn’t free to engage in, especially on platforms like Facebook, and spam is rarely an irrational mo…

You see the same kind of thing in some online multiplayer games. Players acting differently are seen as bots. Same for players who low skill or go AFK.

That said an interesting advance on real game bots would be ones which use AI to chat and respond to other players actions or messages.

Re: ChatGPT and the Enshittening of Knowledge

#237

Post by layer8 »

Beta-7 wrote: post #135
ren_engineer wrote: post #44

Earlier quoted context omitted.

>And worse, then ChatGPT will digest its own excrement, worsening its own results further I wonder if we'll get a "dead sea effect" with AI, I've seen some stuff saying they've basically run out of high quality training data and now the training pool will get poisoned by AI generated shit. Basically garbage in, garbage out and these large language models might not be able to improve

Considering there are "GPT plagiarism" checkers, I don't think this will become an issue. I wonder at which point an extension will come out that will check a page's text if it was written by a human.

Those checkers already have a significant failure rate of false positives/negatives, and that will only get worse as LLMs come closer to human output. Note also that a checker can in principle never outwit a state-of-the-art AI, because the AI can just incorporate and therefore preempt the checker logic.

Re: ChatGPT and the Enshittening of Knowledge

#238

Post by throwaway589275 »

throwaway589275 wrote: post #161

Prediction: OpenAI/GPT (or Google's DeepMind, if you prefer) is going to cause mass unemployment in certain sectors of the economy (for example, graphic designers, copywriters, and many IT professionals) long before it ever addresses any of the fundamental problems that currently substantially reduce human quality and quantity of life, like aging or cancer. In the near-term, AI will just accelerate the winner-take-al…

Also, this isn't just an idle prediction on my part; for what it's worth, I've started substantially shifting my net-worth towards the entities I think will be the winners.

Re: ChatGPT and the Enshittening of Knowledge

#239

Post by 93po »

layer8 wrote: post #237
Beta-7 wrote: post #135

Earlier quoted context omitted.

Considering there are "GPT plagiarism" checkers, I don't think this will become an issue. I wonder at which point an extension will come out that will check a page's text if it was written by a human.

Those checkers already have a significant failure rate of false positives/negatives, and that will only get worse as LLMs come closer to human output. Note also that a checker can in principle never outwit a state-of-the-art AI, because the AI can just incorporate and therefore preempt the checker logic.

And even with good checking it's still trivial to have GPT do 95% of the work and then make some stylistic edits to get around detection.

Re: ChatGPT and the Enshittening of Knowledge

#240

Post by timmytokyo »

oceanplexian wrote: post #88
lubujackson wrote: post #17

If you think of the knowledge base of the internet as a living thing, ChatGPT is a like a virus that now threatens its life. This is the same process SEO spam caused for search - it hampers the nature by which things function and the river needs to reroute (pagerank then usage metadata) to replace the lost signal. ChatGPT is more of an existential threat because it will propagate to infect other knowledge bases. Luke…

> ChatGPT is a like a virus that now threatens its life. Perhaps something needs to be disrupted. The Internet is nothing like what it was 20 years ago, It turned into a bunch of social media walled gardens and SEO spam. ChatGPT is like fresh air because it can actually answer questions in a no-nonsense way without users having to scroll through 5-6 spam websites, paywalls, and crappy user interfaces to get an answer…

You seem to think ChatGPT will somehow be immune to the same forces that lead to the "enshittification" of Internet services like search, social media and e-commerce. Like ChatGPT, these services were a real boon to their users. Then, once they got enough users on board and had to start making a buck, their incentives changed and the users became a secondary concern. The same will happen to things like ChatGPT. See the Cory Doctorow article coining the term "enshittification" for a more elaborate explanation [0].

[0] https://pluralistic.net/2023/01/21/potemkin-ai/#hey-guys

Post reply on HN