Live data from Hacker News

ChatGPT is a blurry JPEG of the web

newyorker.com

251–260 of 317 posts

Re: ChatGPT is a blurry JPEG of the web

#251
post #14

Ugh I’m beginning to think I’m going to spend the next 6-12 months commenting “no, large language models aren’t supposed to somehow know everything in the world. No, that’s not what they’re designed for. Yes, hooking one up to our long-standing record-of-everything-in-the-world (google’s knowledge graph) is going to be powerful.” It’s getting to point where I need to consider stop going on HN. This is like when my fa…

I understand why the mainstream thinks that but it's incredibly annoying that even in tech circles there is very little meaningful discussion, it's mainly just people posting amusing screenshots purportedly showing how smart GPT3 is or in other cases how it's politically biased. Anyone who's played around with it knows that it's fun but it's not a search engine replacement and it doesn't know nor understand things. I…

> We've had GPT3 for ages, it's not like most of us only tried it since Chat GPT3 came out, right?

For myself, I tried GPT models and read the Attention Is All You Need paper before ChatGPT. I also read analysis, for example, from Gwern, about capability overhangs and underestimation of present capabilities in these models. In many cases, I found myself agreeing with the logic. I found that it was very possible to coax greater capabilities out of the model than many presumed them to have. I still find this to be the case and have, in recent memory demonstrated this is true in present models: for example, I posted a method of coaxing the solving of some puzzles by prompting to include the representation of intermediate states in order to successfully solve problems related to reasoning puzzles of a 'can contain' nature, which was a capability that someone claimed these language models lack, despite them gaining that capability when appropriately prompted, which suggests that they always had that capability in their weights, but that it wasn't exercised successfully - the capability was there, but not used, rather than absent, as claimed by the people who claimed it was absent.

That said, I don't think it matters much what most people did or didn't do with regard to this experimentation and, as you imply, ages really did past - I would feel trepidation, not hope, about the quality of my ideas compared to the people who came later. Historically, the passing of ages tends to improve, not diminish. So if I was experimenting with, for example, flying machines in the 1700s, but then ages past and someone who did not do that experimenting was talking to me about flying machines in the early 2000s, I would suspect them to be more informed, not less informed, than I was. They, as a matter of course in casual classroom settings, have probably done better than my best experiments including high effort costly experiments. Their toys fly. A generation ago, we would talk about planes, but now we can also talk about their toys. It is that normal to them. They have so much better priors.

Re: ChatGPT is a blurry JPEG of the web

#252

Earlier quoted context omitted.

I hope I’m not misunderstanding you, but I could be. Are you saying that because the LLM was able to impress you that it must be thinking ? (Whatever that means)

Whatever you want to call the problem solving and persona simulation it can do (in this first commercial generation), you'd never accuse a JPEG engine or an MP3 decoder of anything remotely like it. It's just a really backward-looking conceptualization, underemphasizing everything interesting. You can think of science itself as lossy compression.

>you'd never accuse a JPEG engine or an MP3 decoder of anything remotely like it.

for psychological reasons. Natural language processing makes people prone to anthropomorphize. It's why people treat Alexa in human like ways, or even ELIZA back in the day. You're making the same mistake in your description. You're not teaching ChatGPT anything, you're ever only querying a trained static model. It remains in the same state. It's not "scatterbrained", that's a human quality, it's incorrect. Ted Chiang points to this mistake in the article, mistaking lossiness in an AI model for the kind of error that a human would make.

A photocopier making bad copies is just a flawed machine, but because you don't treat chatgpt like a machine, you think it performing worse is actually a sign of it being smarter. Ironically if it 100% reproduced your language, you'd likely be more sceptical, even if that was due to real underlying intelligence.

Re: ChatGPT is a blurry JPEG of the web

#254

Damn, I hate to plug products on HN, but I'd say that the New Yorker is the one subscription I've loved maintaining throughout my life. First got it right out of college and appreciate it 20 years later. Everyone is publishing think pieces about ChatGPT - yawn. But only the New Yorker said, hmm, how about if we get frickin' Ted Chiang to write a think piece? (It is predictably very well written.)

Interesting that it's such a conservative, opinion-less, air-tight piece. Guess its his technical writing background coming through.

Re: ChatGPT is a blurry JPEG of the web

#255

> ChatGPT is so good at this form of interpolation that people find it entertaining: they’ve discovered a “blur” tool for paragraphs instead of photos, and are having a blast playing with it. “‘blur’ tool for paragraphs” is such a good way of describing the most prominent and remarkable skill of ChatGPT. It is fun, but so obviously trades off against what makes paragraphs great. It is apt that this essay against Chat…

It's a great metaphor and one we should use more. But there's a place for blurred photos: thumbnails. On Hacker News we often complain about headlines because that's all we see at first. But I've been using Kagi's summarizer [1] and I think it's a great tool for getting the gist of certain things, like if you want to know what a YouTube video is about without watching it. (Google Translate is useful for similar reaso…

Thank you for the Kagi mention. I’m using Neeva right now but I didn’t know there were (I didn’t bother looking for) other alternatives.

Re: ChatGPT is a blurry JPEG of the web

#256
post #165

Earlier quoted context omitted.

With subscription models for their apis depending on the use scenario. That could be one reason they opened the service, to see what people are using it for in order to later build services around those use cases. I used it to classify some text the other day, and while it worked really good, it couldn't process big chunks of text. If they offered a pricing model per million characters I'd gladly pay it.

Honestly that doesn't seem too promising as a business prospect. It's essentially an admission that they have a solution but haven't found a problem warranting their initial investment. Even in your case, how will the API generate profit for you?

I already have a service where I curate news articles for specific industries. If an AI can classify the articles it can cut down 80% of the work I’m doing by perusing hundreds of news articles to find the ones that interest my clients.

I'm not arguing that this by itself could justify the tens (or hundreds?) of millions it cost to build the AI, but my guess is that there are dozen of business cases a tool like that could be useful. Just the other day I came upon a company called Persado that provides different marketing copy depending on the age group you're addressing. They could easily eat their lunch with ChatGPT.

Re: ChatGPT is a blurry JPEG of the web

#257
post #244

> ChatGPT is so good at this form of interpolation that people find it entertaining: they’ve discovered a “blur” tool for paragraphs instead of photos, and are having a blast playing with it. “‘blur’ tool for paragraphs” is such a good way of describing the most prominent and remarkable skill of ChatGPT. It is fun, but so obviously trades off against what makes paragraphs great. It is apt that this essay against Chat…

> “‘blur’ tool for paragraphs” is such a good way of describing the most prominent and remarkable skill of ChatGPT. In what way? How, technically, is it anything like that? These comments sound like full-court publicity press for this article. I wonder why.

> How, technically, is it anything like that?

Huh? It isn't. It's a good description because it's figuratively accurate to what reading LLM text feels like, not because it's technically accurate to what it's doing.

Re: ChatGPT is a blurry JPEG of the web

#258

I don't like this analogy; I think why I don't like it is in the intent. With JPEG in the intent is produce an image indistinguishable from the original. Xerox didn't intend to create photocopier that produces incorrect copies. The artifacts are failures of the JPEG algorithm to do what it's supposed to within its constraints. GPT is not trying to create a reproduction of it's source material and simply failing at th…

> With JPEG in the intent is produce an image indistinguishable from the original.

Not necessarily, and even if so, if you continuously opened and saved a JPEG image it would turn to a potato quality image eventually, Xerox machines do the same thing. Happens all the time with memes, and old homework assignments. What I fear is this happening to GPT, especially when people just start outright using its content and putting it on sites. Then it becomes part of what GPT is trained on later on, but what it had previously learned was wrong, so it just progressively gets more and more blurred, with people using the new models to produce content, with a feedback loop that just starts to blur truth and facts entirely.

Even if you tie it to search results like Microsoft is doing, eventually the GPT generated content is going to rise to the top of organic results because of SEO mills using GPT for content to goose traffic...then all the top results agree with the already wrong AI generated answer; or state actors begin gaming the system and feeding the model outright lies.

This happens in people too, sure, but in small subsets not in monolithic fashion with hundreds of millions of people relying on the information being right. I have no idea how they can solve this eventual problem, unless they are just supervising what it's learning all the time; but then at the point it can become incredibly biased and limited.

Re: ChatGPT is a blurry JPEG of the web

#259
post #135

This quote from the article is something I genuinely fear: > "The rise of this type of repackaging is what makes it harder for us to find what we’re looking for online right now; the more that text generated by large-language models gets published on the Web, the more the Web becomes a blurrier version of itself." I am fearful that eventually AI led misinformation is going to be so widespread that it will be impossib…

Agreed about the problem, not the solution. Detection won’t work, it’s way too noisy. We’re heading for bumpy times, soon you no longer need to be a govt to run a credible disinfo campaign. You can run one from your basement, (replacing beer brewing our sourdough making perhaps).

I can see your point on there being too much noise. I don't know a good solution, but feel we may be opening a big can of worms that we'll have to figure out especially in the next decade.

Re: ChatGPT is a blurry JPEG of the web

#260

This is a decent summary. I've been thinking about how ChatGPT by it's very nature destroys context and source reputation. When I search for something on the Internet, I get a link to the original content, which I can then evaluate based on my knowledge and the reputation of the original source. Wikipedia is the same, with a big emphasis on citation. ChatGPT and other LLMs destroy that context and knowledge, giving m…

I would love to know their plan for having new facts propagate into these models. My idle speculation makes me think this is a hard problem. If ChatGPT kills Search it also kills the websites that get surfaced by search that were relying on money from search-directed users. So stores are fine, but "informational" websites are probably in for another cull. Paywall premium publications are probably still fine - the peo…

The other interesting thing is that if people stop using websites, then it reduce revenue for those websites > then development of new pages and sources stops/slows, how does ChatGPT improve? If the information for it to learn isn't there.

We need the source information to be continually generated in order for ChatGPT to improve.

Post reply on HN