ChatGPT and the Enshittening of Knowledge • Hacker News Board

Re: ChatGPT and the Enshittening of Knowledge

#81

Post by foobarbecue » Fri, Jan 27, 2023, 4:44 PM UTC

tablespoon wrote: post #62

gptgpp wrote: post #46

Earlier quoted context omitted.

Could you elaborate on this? If it is full of garbage a couple examples should be very easy to find. I completely agree that Wikipedia can have errors, but in topics that I am educated in it seems pretty decent and I can't remember the last time I came across any (comp sci for example). The most recent example I can think of is about is an article on vulture bees, and a citation about what their honey tastes like, wh…

> Could you elaborate on this? If it is full of garbage a couple examples should be very easy to find. I could give examples but I won't, because that would link my HN and Wikipedia accounts. > So "garbage puffed up beyond all belief" and "full of terrible and fake citations that lead to nowhere" sounds a bit hyperbolic, tbh. People unironically describe it as the "sum of all human knowledge," so it's definitely puff…

Just use a throwaway

Re: ChatGPT and the Enshittening of Knowledge

#82

Post by ericd » Fri, Jan 27, 2023, 4:45 PM UTC

0xBABAD00C wrote: post #75

lubujackson wrote: post #17

If you think of the knowledge base of the internet as a living thing, ChatGPT is a like a virus that now threatens its life. This is the same process SEO spam caused for search - it hampers the nature by which things function and the river needs to reroute (pagerank then usage metadata) to replace the lost signal. ChatGPT is more of an existential threat because it will propagate to infect other knowledge bases. Luke…

> ChatGPT is more of an existential threat because it will propagate to infect other knowledge bases. Luke Wikipedia relies on "published" facts as an authority, but ChatGPT output is going to wind up as a source one way or another. And worse, then ChatGPT will digest its own excrement, worsening its own results further. This is what people do collectively, long before any GPTs were in sight. Lots of strong convictio…

Yep, not sure what the panic here is, ChatGPT is probably churning out better quality stuff than the average SEO spammer. The internet has been mostly garbage for a very long time at this point.

Re: ChatGPT and the Enshittening of Knowledge

#83

Post by miguelazo » Fri, Jan 27, 2023, 4:45 PM UTC

gptgpp wrote: post #46

tablespoon wrote: post #31

Earlier quoted context omitted.

> It's true though, Wikipedia really is terrible and full of fake citations that lead nowhere. It's an anti-knowledge base that sometimes has good information. Yeah, Wikipedia is garbage puffed up beyond all belief. I literally just today saw something just like you describe. It should be viewed very skeptically on anything anyone disagrees over (because then it's just snapshots of an agenda-pushing battle).

Could you elaborate on this? If it is full of garbage a couple examples should be very easy to find. I completely agree that Wikipedia can have errors, but in topics that I am educated in it seems pretty decent and I can't remember the last time I came across any (comp sci for example). The most recent example I can think of is about is an article on vulture bees, and a citation about what their honey tastes like, wh…

It's telling that you only cited examples of scientific subjects. As the other commenter noted, articles of consequence for public debate (politics) are generally terrible and there are lots of "editors" who are working for deep state cut-outs doing nothing but trying to damage the reputation of intellectuals who are a danger to the status quo.

Re: ChatGPT and the Enshittening of Knowledge

#84

Post by narrator » Fri, Jan 27, 2023, 4:46 PM UTC

lubujackson wrote: post #17

If you think of the knowledge base of the internet as a living thing, ChatGPT is a like a virus that now threatens its life. This is the same process SEO spam caused for search - it hampers the nature by which things function and the river needs to reroute (pagerank then usage metadata) to replace the lost signal. ChatGPT is more of an existential threat because it will propagate to infect other knowledge bases. Luke…

This is why the ChatGPT lawyer thing is far away. When you make an argument in a court filing as a lawyer, you as a person put your reputation on the line that that filing is not an argument that uses fake case law, or makes an argument that is nonsense. If that argument is nonsense and has nothing to do with the law or case law, you can lose your license to practice law. It happens with some regularity. People think that the legal system is an API they can spam. It's absolutely not. In fact, many rules in civil procedure are intended to make it a highly disadvantageous strategy to waste the court's time.

Re: ChatGPT and the Enshittening of Knowledge

#85

Post by miguelazo » Fri, Jan 27, 2023, 4:46 PM UTC

tablespoon wrote: post #62

gptgpp wrote: post #46

Earlier quoted context omitted.

Could you elaborate on this? If it is full of garbage a couple examples should be very easy to find. I completely agree that Wikipedia can have errors, but in topics that I am educated in it seems pretty decent and I can't remember the last time I came across any (comp sci for example). The most recent example I can think of is about is an article on vulture bees, and a citation about what their honey tastes like, wh…

> Could you elaborate on this? If it is full of garbage a couple examples should be very easy to find. I could give examples but I won't, because that would link my HN and Wikipedia accounts. > So "garbage puffed up beyond all belief" and "full of terrible and fake citations that lead to nowhere" sounds a bit hyperbolic, tbh. People unironically describe it as the "sum of all human knowledge," so it's definitely puff…

THIS: "people with weird personalities"

Seen it way too many times for it to be a coincidence.

Re: ChatGPT and the Enshittening of Knowledge

#86

Post by Arainach » Fri, Jan 27, 2023, 4:47 PM UTC

0xBABAD00C wrote: post #75

lubujackson wrote: post #17

If you think of the knowledge base of the internet as a living thing, ChatGPT is a like a virus that now threatens its life. This is the same process SEO spam caused for search - it hampers the nature by which things function and the river needs to reroute (pagerank then usage metadata) to replace the lost signal. ChatGPT is more of an existential threat because it will propagate to infect other knowledge bases. Luke…

> ChatGPT is more of an existential threat because it will propagate to infect other knowledge bases. Luke Wikipedia relies on "published" facts as an authority, but ChatGPT output is going to wind up as a source one way or another. And worse, then ChatGPT will digest its own excrement, worsening its own results further. This is what people do collectively, long before any GPTs were in sight. Lots of strong convictio…

Automation and scale create their own threats.

Security cameras have existed for a long time, but storage cheap enough to keep years of footage and algorithms capable of processing thousands of streams in real time create massive privacy problems that didn't exist even with the richest companies paying humans to watch.

Re: ChatGPT and the Enshittening of Knowledge

#87

Post by tablespoon » Fri, Jan 27, 2023, 4:47 PM UTC

foobarbecue wrote: post #81

tablespoon wrote: post #62

Earlier quoted context omitted.

> Could you elaborate on this? If it is full of garbage a couple examples should be very easy to find. I could give examples but I won't, because that would link my HN and Wikipedia accounts. > So "garbage puffed up beyond all belief" and "full of terrible and fake citations that lead to nowhere" sounds a bit hyperbolic, tbh. People unironically describe it as the "sum of all human knowledge," so it's definitely puff…

Just use a throwaway

> Just use a throwaway

Too late now.

Re: ChatGPT and the Enshittening of Knowledge

#88

Post by oceanplexian » Fri, Jan 27, 2023, 4:48 PM UTC

lubujackson wrote: post #17

If you think of the knowledge base of the internet as a living thing, ChatGPT is a like a virus that now threatens its life. This is the same process SEO spam caused for search - it hampers the nature by which things function and the river needs to reroute (pagerank then usage metadata) to replace the lost signal. ChatGPT is more of an existential threat because it will propagate to infect other knowledge bases. Luke…

> ChatGPT is a like a virus that now threatens its life.

Perhaps something needs to be disrupted. The Internet is nothing like what it was 20 years ago, It turned into a bunch of social media walled gardens and SEO spam. ChatGPT is like fresh air because it can actually answer questions in a no-nonsense way without users having to scroll through 5-6 spam websites, paywalls, and crappy user interfaces to get an answer to a simple question.

The only thing that's being threatened is companies like Google who are responsible for the current state of the web.

Re: ChatGPT and the Enshittening of Knowledge

#89

Post by ptx » Fri, Jan 27, 2023, 4:48 PM UTC

visarga wrote: post #38

lsy wrote: post #25

The TLDR of the article is essentially that any time a ChatGPT conversation strays to the factual (in this case a list of data ethics books), the quality of the data (whether those books exist and are written by the stated authors) is extremely low. Ultimately this is where ChatGPT and similar projects seem most likely to hit a wall. Text generators are statistical machines, but the relationship between statistics an…

One approach to fixing factual errors is to use two rounds of LLM interaction. I forgot the name of the paper. Say you ask "What is the height of Everest?" 1. generate an answer in closed-book model, with the LLM: "The height of Everest is 8723m" = candidate_answer 2. search your references with candidate_answer, find: "At 8,849 meters (29,032 feet), Everest is considered the tallest point on Earth" = search_snippet…

How would you apply this approach to the article's example, the response to the question "Who is Daragh O Brien from Castlebridge", where I count at least 15 separate statements of fact?

Should we research all of them and try again with a big table of hits and misses from the first attempt? Seems like a lot of work.

Also: Is the generated response really "very good at matching the correct answer"? I suppose it would work because the search engine's language processing cancels out the useless parts that were generated by the AI (sort of a "human ABI", analogous to the C ABI?) but a more direct query (e.g. "height of everest") would likely be just as effective.

Re: ChatGPT and the Enshittening of Knowledge

#90

Post by narrator » Fri, Jan 27, 2023, 4:48 PM UTC

yellow_postit wrote: post #39

tablespoon wrote: post #26

Earlier quoted context omitted.

>> All signs point to this strengthening the value of curation and authenticated sources. > This is what they said about Wikipedia viz. Britannica... And they were right. If "they" were wrong about anything, it was the assumption that the masses would prioritize quality over cost, but it turns out that cheap wins every time. When it comes to information, it's like most people's taste buds don't work, so they'll pick…

Extending this — it seems to me there will be growing skillset need to identify quality/accuracy. Under formed thought: The proverbial haystack just got a lot larger, the needle stayed the same size, what tools will needle hunters need to develop both to find the needles and to prove to others they are in fact needles.

The funny thing about ChatGPT is it will write code that uses non-existent confabulated APIs. You have to then call it out and it will say, oh sorry, of course you're right, here's another confabulated API, etc. The amount of convincing B.S it can spew is enormous!

Worse, when you stray into topics that are controversial it will often use informal fallacies. When you call it out, it will say, yes, you're right, I used an informal fallacy, here is what I should have said about controversial topic that's not the party line, and because you're so smart I won't b.s you.