An entire article about compression being similar to what a DNN does without a mention of Naftali Tishby's Information Bottleneck principle for neural networks. https://en.wikipedia.org/wiki/Information_bottleneck_method
ChatGPT is a blurry JPEG of the web
81–90 of 317 posts
Re: ChatGPT is a blurry JPEG of the web
#82Damn, I hate to plug products on HN, but I'd say that the New Yorker is the one subscription I've loved maintaining throughout my life. First got it right out of college and appreciate it 20 years later. Everyone is publishing think pieces about ChatGPT - yawn. But only the New Yorker said, hmm, how about if we get frickin' Ted Chiang to write a think piece? (It is predictably very well written.)
Certainly beats a Medium subscription, where you pay (more?) to read 95% garbage when compared with what's put in The New Yorker.
There are people out there that pay for Medium?
Re: ChatGPT is a blurry JPEG of the web
#83Earlier quoted context omitted.
I'll never understand why people find this amusing.
It's just the potential. Something feels in the right direction. Perhaps it will only remain a tool and it will never write the next Infinite Jest , but the next David Foster Wallace will certainly use its potential to become a better writer, just in the same manner auto-spellcheckers make me a better writer. Incidentally, David Foster Wallace would have loved ChatGPT: finally, he no longer needed to write, he could…
> but the next David Foster Wallace will certainly use its potential to become a better writer, just in the same manner auto-spellcheckers make me a better writer.
Really contestable, actually, not certain.
This is also addressed in the text:
> Can large-language models help humans with the creation of original writing? To answer that, we need to be specific about what we mean by that question. There is a genre of art known as Xerox art, or photocopy art, in which artists use the distinctive properties of photocopiers as creative tools. Something along those lines is surely possible with the photocopier that is ChatGPT, so, in that sense, the answer is yes. But I don’t think that anyone would claim that photocopiers have become an essential tool in the creation of art; the vast majority of artists don’t use them in their creative process, and no one argues that they’re putting themselves at a disadvantage with that choice.
Re: ChatGPT is a blurry JPEG of the web
#84This is very well written, and probably one of my favorite takes on the whole ChatGPT thing. This sentence in particular:
> Indeed, a useful criterion for gauging a large-language model’s quality might be the willingness of a company to use the text that it generates as training material for a new model.
It seems obvious that future GPTs should not be trained on the current GPT's output, just as future DALL-Es should not be trained on current DALL-E outputs, because the recursive feedback loop would just yield nonsense. But, a recursive feedback loop is exactly what superhuman models like AlphaZero use. Further, AlphaZero is even trained on its own output even during the phase where it performs worse than humans.
There are, obviously, a whole bunch of reasons for this. The "rules" for whether text is "right" or not are way fuzzier than the "rules" for whether a move in Go is right or not. But, it's not implausible that some future model will simply have a superhuman learning rate and a superhuman ability to distinguish "right" from "wrong" - this paragraph will look downright prophetic then.
Re: ChatGPT is a blurry JPEG of the web
#85Damn, I hate to plug products on HN, but I'd say that the New Yorker is the one subscription I've loved maintaining throughout my life. First got it right out of college and appreciate it 20 years later. Everyone is publishing think pieces about ChatGPT - yawn. But only the New Yorker said, hmm, how about if we get frickin' Ted Chiang to write a think piece? (It is predictably very well written.)
Damn too, actually. Reading the piece, I've been thinkking this publication really dserves to be subscribed to!
Amazing write up. ChatGPT is a blurry JPEG of the internet.
Re: ChatGPT is a blurry JPEG of the web
#86Ugh I’m beginning to think I’m going to spend the next 6-12 months commenting “no, large language models aren’t supposed to somehow know everything in the world. No, that’s not what they’re designed for. Yes, hooking one up to our long-standing record-of-everything-in-the-world (google’s knowledge graph) is going to be powerful.” It’s getting to point where I need to consider stop going on HN. This is like when my fa…
> Yes, hooking one up to our long-standing record-of-everything-in-the-world (google’s knowledge graph) is going to be powerful.
This hasn't happened yet and, while I may just lack imagination, despite having a fairly solid understanding of how the latest round of AI works I can't see how it can be done successfully. Until it is in fact done and my lack of imagination is demonstrated, your "going to be powerful" is a speculation about the future, not an observation about the present, and deserves the level of respect usually accorded to such speculations.
Re: ChatGPT is a blurry JPEG of the web
#87Does this article offer any understanding of what ChatGPT is?
Re: ChatGPT is a blurry JPEG of the web
#88Damn, I hate to plug products on HN, but I'd say that the New Yorker is the one subscription I've loved maintaining throughout my life. First got it right out of college and appreciate it 20 years later. Everyone is publishing think pieces about ChatGPT - yawn. But only the New Yorker said, hmm, how about if we get frickin' Ted Chiang to write a think piece? (It is predictably very well written.)
Too bad that this kind of writing is married to 31(!) ads/trackers on that page. Can the journalism like this really not survive without all that crap?
Re: ChatGPT is a blurry JPEG of the web
#89> The fact that Xerox photocopiers use a lossy compression format instead of a lossless one isn’t, in itself, a problem.
Regardless of the article, I just want to disagree here. RAM is cheap. Xerox machines are expensive as hell. Come on Xerox.
Re: ChatGPT is a blurry JPEG of the web
#90Damn, I hate to plug products on HN, but I'd say that the New Yorker is the one subscription I've loved maintaining throughout my life. First got it right out of college and appreciate it 20 years later. Everyone is publishing think pieces about ChatGPT - yawn. But only the New Yorker said, hmm, how about if we get frickin' Ted Chiang to write a think piece? (It is predictably very well written.)
They also have weekly cryptic crosswords that are cryptic enough to be interesting but as easy to completely solve as a regular crossword. (With cryptics, very good ones are also very hard.)