Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT is a blurry JPEG of the web

newyorker.com

Post reply on HN
Print view View original

181–190 of 317 posts

Re: ChatGPT is a blurry JPEG of the web

#181

Post by steveBK123 »

boh wrote: post #142

Does anyone have any idea how ChatGPT will actually make money? As novel as it is to use with all the "potential" applications, the possible revenue streams don't seem to prop up the recent investments into OpenAI. We've already been through enough hype cycles in the past ten years to realize "potential" use-cases or user counts don't necessarily produce a sustainable business model. Nor does a new innovative thing n…

I think it will find a niche, but first it will get all blown out of proportion like voice assistants did.

Everything was going to be voice, voice OS, no more phones, blah blah blah.

Half the time I use mine, I am swearing at it for screwing up. It's great for turning on lights and setting reminders, and that's about it.. 7 years later.

Re: ChatGPT is a blurry JPEG of the web

#182

Post by williamcotton »

wvenable wrote: post #79

I don't like this analogy; I think why I don't like it is in the intent. With JPEG in the intent is produce an image indistinguishable from the original. Xerox didn't intend to create photocopier that produces incorrect copies. The artifacts are failures of the JPEG algorithm to do what it's supposed to within its constraints. GPT is not trying to create a reproduction of it's source material and simply failing at th…

> Compression and GPT are both mathematical processes they aren't the same process;

They're not, but they are very related! GPT has a 1536 dimensional vector space that is conceptually related to a principal component analysis and dimensional reduction in certain compression algorithms.

This does mean that neural networks can overfit and be fully reversible but that is hardly their only useful feature!

They are also very good at translating and synthesizing, depending on the nature of the prompt.

If given an analytic prompt like, "convert this baseball box score into an entertaining paragraph", ChatGPT does a reliable job of acting as a translator because all of the facts about the game are contained in the box score!

But when given a synthetic prompt like, "give me some quotes from the broadcasters", ChatGPT does a reliable job of acting as a synthesizer, because none of those facts about the spoken transcript of the broadcasters is in the prompt. And it is a good synthesizer as well because those quotes sound real!

Terms, etc:

https://williamcotton.com/articles/chatgpt-and-the-analytic-...

Re: ChatGPT is a blurry JPEG of the web

#183

Post by eurekin »

thundergolfer wrote: post #45

> ChatGPT is so good at this form of interpolation that people find it entertaining: they’ve discovered a “blur” tool for paragraphs instead of photos, and are having a blast playing with it. “‘blur’ tool for paragraphs” is such a good way of describing the most prominent and remarkable skill of ChatGPT. It is fun, but so obviously trades off against what makes paragraphs great. It is apt that this essay against Chat…

That reminds me... There is a interestingly relevant japaneese phrase for, to put it nicely, not a bright or sharp person: baka.

Supposedly, if I'm remembering last discussion with a japaneese speaker correctly, the same stem is used for "blur", or "blurry" (bokeh, bokeshi).

Which is kind of interesting parallel here

Re: ChatGPT is a blurry JPEG of the web

#184

Post by boh »

EamonnMR wrote: post #177
boh wrote: post #142

Does anyone have any idea how ChatGPT will actually make money? As novel as it is to use with all the "potential" applications, the possible revenue streams don't seem to prop up the recent investments into OpenAI. We've already been through enough hype cycles in the past ten years to realize "potential" use-cases or user counts don't necessarily produce a sustainable business model. Nor does a new innovative thing n…

Take the starry eyed excitement and use it to fuel a huge exit for investors in the form of an IPO.

It's not the 2010's anymore. Market interest in unprofitable tech companies has waned.Plus OpenAI's organizational structure makes it a poor candidate for an IPO. Maybe MS will buy it, but honestly there's little reason for them to do that. In any case, pinning all expected value on an IPO doesn't really speak well of the firm's potential value.

Re: ChatGPT is a blurry JPEG of the web

#185

Post by boh »

babelfish wrote: post #174
boh wrote: post #142

Does anyone have any idea how ChatGPT will actually make money? As novel as it is to use with all the "potential" applications, the possible revenue streams don't seem to prop up the recent investments into OpenAI. We've already been through enough hype cycles in the past ten years to realize "potential" use-cases or user counts don't necessarily produce a sustainable business model. Nor does a new innovative thing n…

Strong B2B use case

In B2B one B has to have an actual source of revenue to justify paying the other B. What is that source of revenue? Hoping someone else will figure out a reason to use it isn't a business model.

Re: ChatGPT is a blurry JPEG of the web

#186

Post by danuker »

criddell wrote: post #37
danuker wrote: post #8

Earlier quoted context omitted.

"Your Honor, we have evidence Ryan Gosling may have breached our systems."

Would make a good (bad) CSI episode. Enhance the security footage then put out an arrest warrant for Ryan Gosling.

Another episode: the copier switches digits randomly.

https://www.bbc.com/news/technology-23588202

Re: ChatGPT is a blurry JPEG of the web

#187

Post by majormajor »

nightski wrote: post #172
majormajor wrote: post #160

Earlier quoted context omitted.

I don't think you can describe the math in this context as "generalize well to new data." ChatGPT certainly can't generate new data. It's not gonna correctly tell you today who won the World Series in 2030. It's not going to write a poem in the style of someone who hasn't been born yet. But it can interpolate between and through a bunch of existing data that's on the web to produce novel mixes of it. I find the "blur…

It absolutely can generate new data, it does so all the time. If you are claiming otherwise I think we need a more formal definition of what you mean by new data. Are you suggesting because it can't predict the future it can't generate novel data?

It's not just the future, though the examples I gave were future oriented.

But it's all very interpolation/summarization-focused.

A "song lyrics in the style of Taylor Swift" isn't an actual song by Taylor Swift.

A summary of the history of Texas isn't actually vetted by any historian to ensure accuracy.

The answer to a math problem may not be correct.

To me, those things don't qualify as "new data." They aren't suitable for future training as-is. Sometimes for a simple reason: they aren't facts, using the dictionary "facts and statistics collected together for reference or analysis" definition of data. So very simply "not new data."

Sometimes in a blurrier way - the song lyrics, for instance, could be touching, or poignant, or "true" in a Keats sense[0] - but if the internet gets full of GPT-dreams and future models are trained on that, you could slide down further and further into an uncanny valley, especially since most of the time you don't get one of those amazing poignant ones. Most of the time I've gotten something bland.

[0] "What the imagination seizes as beauty must be truth"

Re: ChatGPT is a blurry JPEG of the web

#188

Post by Agraillo »

> Imagine what it would look like if ChatGPT were a lossless algorithm. If that were the case, it would always answer questions by providing a verbatim quote from a relevant Web page. We would probably regard the software as only a slight improvement over a conventional search engine, and be less impressed by it

The story is an impressive piece, but I think as with many of us, it's a personal projection of expectations on results. One example from my experience. In the book "Jim Carter - Sky Spy, Memoirs of a U-2 Pilot" there was an interesting story about the moment when U-2 was used for capturing the photo of a big area at the Pacific to save the life of a lost seaman. The story was very interesting and I always wanted to know more, technical details, people involvement etc. Searching with Google ten years ago didn't help, I rephrased the names, changed the date (used even the range operator) to no avail. And recently I asked several LLM-based bots about it. You can guess it. They ignored my constrains at best and hallucinate at worst. One even invented a mixed reality story when Francis Gary Powers actually flew not one but with a co-pilot and the latter ended up in the Pacific and was saved. Very funny, but I wasn't impressed. But if one of them scraped the far corners of web discussion boards and saved a first-person account of someone who took part in it and gave it to me, I would be really impressed.

Re: ChatGPT is a blurry JPEG of the web

#189

Post by ravenstine »

steveBK123 wrote: post #181
boh wrote: post #142

Does anyone have any idea how ChatGPT will actually make money? As novel as it is to use with all the "potential" applications, the possible revenue streams don't seem to prop up the recent investments into OpenAI. We've already been through enough hype cycles in the past ten years to realize "potential" use-cases or user counts don't necessarily produce a sustainable business model. Nor does a new innovative thing n…

I think it will find a niche, but first it will get all blown out of proportion like voice assistants did. Everything was going to be voice, voice OS, no more phones, blah blah blah. Half the time I use mine, I am swearing at it for screwing up. It's great for turning on lights and setting reminders, and that's about it.. 7 years later.

I find it funny people are still using these voice "assistants" despite the frequent frustration. Alexa gets it wrong more than half the time when people I know try to use it, but they seem to still want to believe.

Re: ChatGPT is a blurry JPEG of the web

#190

Post by AtNightWeCode »

nneonneo wrote: post #84

This is very well written, and probably one of my favorite takes on the whole ChatGPT thing. This sentence in particular: > Indeed, a useful criterion for gauging a large-language model’s quality might be the willingness of a company to use the text that it generates as training material for a new model. It seems obvious that future GPTs should not be trained on the current GPT's output, just as future DALL-Es should…

This is old. This is the reason why Google translate sucks. It can't tell the difference between what it translated from what a competent person translated.
Post reply on HN