Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT is a blurry JPEG of the web

newyorker.com

Post reply on HN
Print view View original

151–160 of 317 posts

Re: ChatGPT is a blurry JPEG of the web

#151

Post by wvenable »

ryangs wrote: post #130
wvenable wrote: post #79

I don't like this analogy; I think why I don't like it is in the intent. With JPEG in the intent is produce an image indistinguishable from the original. Xerox didn't intend to create photocopier that produces incorrect copies. The artifacts are failures of the JPEG algorithm to do what it's supposed to within its constraints. GPT is not trying to create a reproduction of it's source material and simply failing at th…

I don't think JPEG wants to produce an image indistinguishable from the original. It wants to reduce space usage without distorting "too" much. Failing to reduce space usage would be considered a "failure" of JPEG, just as much as distorting too much.

JPEG relies of the limitations of human vision to make an image largely indistinguishable from the original. It specifically throws information away that we are less unlikely to notice. So yes, a good JPEG should be indistinguishable (to humans) from the original. Obviously the more you turn up the compression the harder that is.

Re: ChatGPT is a blurry JPEG of the web

#152

Post by thwayunion »

danans wrote: post #122

> Obviously, no one can speak for all writers, but let me make the argument that starting with a blurry copy of unoriginal work isn’t a good way to create original work. If you’re a writer, you will write a lot of unoriginal work before you write something original. And the time and effort expended on that unoriginal work isn’t wasted; on the contrary, I would suggest that it is precisely what enables you to eventual…

> But it would still require extensive input of my experiences in order to create something new from its compressed models of feasible designs, and those experiences are something it can't hallucinate.

This is exactly why I record almost everything about my life (stored locally, of course).

Others may find it creepy/weird, but I have found enormous value: fine-tuning Stable Diffusion and GPT-2, lots of applications of very simple classifiers and reinforcement learning, etc.

Re: ChatGPT is a blurry JPEG of the web

#153

Post by majormajor »

thundergolfer wrote: post #45

> ChatGPT is so good at this form of interpolation that people find it entertaining: they’ve discovered a “blur” tool for paragraphs instead of photos, and are having a blast playing with it. “‘blur’ tool for paragraphs” is such a good way of describing the most prominent and remarkable skill of ChatGPT. It is fun, but so obviously trades off against what makes paragraphs great. It is apt that this essay against Chat…

The amount of human-generated lowest-common-denominator English-language free content was already so high that I'm not sure the New Yorker has anything (more) to worry about. If you've been paying for the New Yorker already in the days of Medium, Buzzfeed, blogs, and what-have-you, does there being even more uncurated stuff change your equation? (It doesn't for me.)

More cynically: it'll be hard to kill the few legacy zombies that have survived so much destruction at the hand of free internet content already.

Re: ChatGPT is a blurry JPEG of the web

#154

Post by wvenable »

koof wrote: post #128
wvenable wrote: post #79

I don't like this analogy; I think why I don't like it is in the intent. With JPEG in the intent is produce an image indistinguishable from the original. Xerox didn't intend to create photocopier that produces incorrect copies. The artifacts are failures of the JPEG algorithm to do what it's supposed to within its constraints. GPT is not trying to create a reproduction of it's source material and simply failing at th…

Blurriness gets weird when you're talking about truth. Depending on the application we can accept a few pixels here or there being slightly different colors. I queried GPT to try and find a book I could only remember a few details of. The blurriness of GPT's interpretation of facts was to invent a book that didn't exist, complete with a fake ISBN number. I asked GPT all kinds of ways if the book really existed, and i…

I think that strengthens my point. We consider a blurry image of something to still be a true representation of that thing. We should never consider a GPT representation of a thing to be true.

Re: ChatGPT is a blurry JPEG of the web

#155

Post by squeaky-clean »

wbobeirne wrote: post #95
freediver wrote: post #88

Earlier quoted context omitted.

Too bad that this kind of writing is married to 31(!) ads/trackers on that page. Can the journalism like this really not survive without all that crap?

They sell the perfect solution to this: a magazine.

The magazine still has ads in it, doesn't it? I'd rather they know my ip address rather than my mailing address.

Re: ChatGPT is a blurry JPEG of the web

#156

Post by majormajor »

boh wrote: post #142

Does anyone have any idea how ChatGPT will actually make money? As novel as it is to use with all the "potential" applications, the possible revenue streams don't seem to prop up the recent investments into OpenAI. We've already been through enough hype cycles in the past ten years to realize "potential" use-cases or user counts don't necessarily produce a sustainable business model. Nor does a new innovative thing n…

"Search ads" through clever server-side auto-prompt-modification.

Re: ChatGPT is a blurry JPEG of the web

#157

Post by williamcotton »

danans wrote: post #143
williamcotton wrote: post #114

> I think there’s a simpler explanation. Imagine what it would look like if ChatGPT were a lossless algorithm. If that were the case, it would always answer questions by providing a verbatim quote from a relevant Web page. We would probably regard the software as only a slight improvement over a conventional search engine, and be less impressed by it. Tautologically, yes, ChatGPT works because it is, as defined by th…

> This has nothing to do with "looking smarter". It has to do with being reliably proficient at both translating and synthesizing. I think the author's point is about how people perceive lossy text output differently than they perceive lossy image output. Language is a pretty precise symbolic information medium, and our perception of it is based in large part on both our education and what we believe makes humans uni…

I don't think "lossy text" is a useful term because it conflates with th*s k*nd *f l*ss* t*xt as well. Lossy compression is designed to be as reversible as it can be to a given threshold. That's not how ChatGPT was either designed or works in practice. There are definitely a lot of mathematical similarities between the two, I won't deny that.

Would "partial knowledge compression" be a better term? Partial knowledge of both English and French is a requirement to reliably translate from English to French. Partial knowledge of both baseball box scores and entertaining paragraph outlines in English is a requirement to reliably translate from a box score into an entertaining outline, right?

Re: ChatGPT is a blurry JPEG of the web

#158

Post by EVa5I7bHFq9mnYK »

nicbou wrote: post #100
chapium wrote: post #55

Earlier quoted context omitted.

I’m just searching the comments for novel use cases where its effective. Most articles I’ve read seem like either moral panics or snake oil. I like how it can generate songs and poems based on a prompt. Its not particularly useful, but it is entertaining. It really does seem curated at times, leading me to think this will eventually become a fad or replaced by a more advanced algorithm.

The most potent use case of the current implementation of ChatGPT is plagiarism. Businesses who need blog posts for content marketing would hire copywriters who would paraphrase my content and publish it as their own. Now they can do it faster and for free. Blogspam got cheaper and more effective. This applies to other kinds of spam too: email spam, forum spam, comment spam... Spam spam spam. I'm afraid that we'll so…

If it will make homework go away, good riddance)

Re: ChatGPT is a blurry JPEG of the web

#159

Post by [deleted] »

Spivak wrote: post #118
nicbou wrote: post #100

Earlier quoted context omitted.

The most potent use case of the current implementation of ChatGPT is plagiarism. Businesses who need blog posts for content marketing would hire copywriters who would paraphrase my content and publish it as their own. Now they can do it faster and for free. Blogspam got cheaper and more effective. This applies to other kinds of spam too: email spam, forum spam, comment spam... Spam spam spam. I'm afraid that we'll so…

Or less cynically, it's a really good writing assistant. There is a self-service tool that understands English on a level that Grammerly could only dream of. Not sure how to phrase an email in a professional tone, tell GPT to write it and edit from there. This is a tool that is going to be a smash it with non-native speakers or really anyone who just isn't that good with words. > Write an email to a coworker telling…

[deleted]

Re: ChatGPT is a blurry JPEG of the web

#160

Post by majormajor »

nightski wrote: post #116
PaulHoule wrote: post #106

Earlier quoted context omitted.

"Blurry JPEG" for how ChatGPT "compresses" character-based knowledge into vectors. That "compression" process gives ChatGPT an ability to generalize because it learns statistics (unlike JPEG) but like JPEG it is a lossy process.

It's a terrible analogy because the entire point of ML systems is to generalize well to new data, not to reproduce the original data as accurate as possible with a space/time tradeoff.

I don't think you can describe the math in this context as "generalize well to new data."

ChatGPT certainly can't generate new data. It's not gonna correctly tell you today who won the World Series in 2030. It's not going to write a poem in the style of someone who hasn't been born yet.

But it can interpolate between and through a bunch of existing data that's on the web to produce novel mixes of it. I find the "blurring those things together" analogy pretty compelling there, in the same way that blurring or JPEG-compressing something isn't going to give you a picture of a new event but it might change what you appear to see in the data you already had.

(Obviously it's not exactly the same, that's why it's an analogy and not a definition. As an analogy, it works much better if you ignore much of what you know about the implementation details of both of them. It's not trying to teach someone how to build it, but to teach a lay person how to think about the output.)

Post reply on HN