Earlier quoted context omitted.
You don't need to correct every wrong thing you read. In fact you will probably feel much better if you don't ever do it at all, or at least take a break for while.
Very true :). It doesn’t help that this isn’t exactly a little blog post, it’s a popular New Yorker feature…
ChatGPT is a blurry JPEG of the web
121–130 of 317 posts
Re: ChatGPT is a blurry JPEG of the web
#122I'd add the following to this: The font (as in fountain) of all creativity is the physical and emotional experience of the real world. This is true for writing a great world-changing classic novel as it is for the realm of scientific discovery, new engineering applications, visual or audible art.
It's the stimulus from the natural world, conveyed to us via our senses coupled to our linguistic or symbolic generation capability, that ultimately drives the most novel and relatable rearrangements and transformations of existing information that we eventually call "art". And when a work lacks that foundational experience, or it becomes regurgitated too many times without novel inputs, it begins to feel inauthentic.
For example, when I remodeled my house, I made the plan based on my family's lived experiences, both physical and emotional. Every wall that I bumped up against, every chilly corner, and the ache of my knees carrying laundry up and down stairs informed the remodel. Also, the way I liked to sit when talking to visiting friends.
Sure, some of these things followed well trodden patterns from architecture, remodels and associated trends, but others were quite idiosyncratic, even whimsical, based on the way I like to live. And it's the idiosyncratic and whimsical that creates both novelty and joy in the aesthetic appreciation of things.
Could an AI tool based trained on remodels accelerate aspects of the design? Absolutely (there's a product idea right there). But it would still require extensive input of my experiences in order to create something new from its compressed models of feasible designs, and those experiences are something it can't hallucinate.
Re: ChatGPT is a blurry JPEG of the web
#123Earlier quoted context omitted.
You are arguing with your own straw man interpretation of the article. It isn't talking about all possible uses of LLMs, but focusing on specific uses now being proposed, to use ChatGPT and its possible successors instead of search. You ignore his points about how achieving really good compression requires learning structure in the data that starts to amount to understanding: if you can understand the rules of arithm…
Who is proposing the use of ChatGPT, in its current form, for search? Bing search is not just "ChatGPT" added next to bing search results. Please look up how it works, it is quite sophisticated and (imo) designed well. I have access to it; would you like a demo?
The dozens of posts I've seen here saying "this is going to replace google!" for starters.
Re: ChatGPT is a blurry JPEG of the web
#124"mastery" can be considered a hyper efficient destructive compression (experts are often unable to articulate or teach to beginners) that reduces latency of response to such extreme levels that they seem to be predicting the future or reacting at godlike speeds.
Re: ChatGPT is a blurry JPEG of the web
#125This is very well written, and probably one of my favorite takes on the whole ChatGPT thing. This sentence in particular: > Indeed, a useful criterion for gauging a large-language model’s quality might be the willingness of a company to use the text that it generates as training material for a new model. It seems obvious that future GPTs should not be trained on the current GPT's output, just as future DALL-Es should…
I generally respect the heck out of Chiang but I think it's silly to expect anyone to be happy feeding a language model's output back into it, unless that output has somehow been modified by the real world.
Re: ChatGPT is a blurry JPEG of the web
#126The compression & blur analogy also applies to human minds as well. If you focus on fidelity, you have to increase storage and specialize in a narrow domain. If you want a bit of everything, then blurring and destructive compression is the only way. E.g. a "book smart" vs "street smart" difference. "mastery" can be considered a hyper efficient destructive compression (experts are often unable to articulate or teach t…
Re: ChatGPT is a blurry JPEG of the web
#127Ugh I’m beginning to think I’m going to spend the next 6-12 months commenting “no, large language models aren’t supposed to somehow know everything in the world. No, that’s not what they’re designed for. Yes, hooking one up to our long-standing record-of-everything-in-the-world (google’s knowledge graph) is going to be powerful.” It’s getting to point where I need to consider stop going on HN. This is like when my fa…
It's also worth noting that these models are not designed to replace human intelligence, but rather to augment it and provide valuable insights and assistance in various tasks. And while connecting them to large knowledge graphs like Google's can be powerful, it's still only one piece of the puzzle.
It can be discouraging to face resistance, but it's important to keep in mind that advancements in technology often encounter initial skepticism before they become widely adopted. Just like the computer revolution in the 90s, it will take time for people to fully understand and appreciate the benefits of large language models.
Re: ChatGPT is a blurry JPEG of the web
#128I don't like this analogy; I think why I don't like it is in the intent. With JPEG in the intent is produce an image indistinguishable from the original. Xerox didn't intend to create photocopier that produces incorrect copies. The artifacts are failures of the JPEG algorithm to do what it's supposed to within its constraints. GPT is not trying to create a reproduction of it's source material and simply failing at th…
Depending on the application we can accept a few pixels here or there being slightly different colors.
I queried GPT to try and find a book I could only remember a few details of. The blurriness of GPT's interpretation of facts was to invent a book that didn't exist, complete with a fake ISBN number. I asked GPT all kinds of ways if the book really existed, and it repeatedly insisted that it did.
I think your argument here would be to say that being reversible to a real book isn't the intent, but that's not how it is being marketed nor how GPT would describe itself.
Re: ChatGPT is a blurry JPEG of the web
#129Ugh I’m beginning to think I’m going to spend the next 6-12 months commenting “no, large language models aren’t supposed to somehow know everything in the world. No, that’s not what they’re designed for. Yes, hooking one up to our long-standing record-of-everything-in-the-world (google’s knowledge graph) is going to be powerful.” It’s getting to point where I need to consider stop going on HN. This is like when my fa…
Re: ChatGPT is a blurry JPEG of the web
#130I don't like this analogy; I think why I don't like it is in the intent. With JPEG in the intent is produce an image indistinguishable from the original. Xerox didn't intend to create photocopier that produces incorrect copies. The artifacts are failures of the JPEG algorithm to do what it's supposed to within its constraints. GPT is not trying to create a reproduction of it's source material and simply failing at th…