161–170 of 317 posts
Earlier quoted context omitted.
I thought the author was uncharacteristically perceptive for a reporter. Yann LeCun or Geoff Hinton couldn't have come up with a better analogy.
So they can't afford an actual subject-matter expert for their articles?
Earlier quoted context omitted.
It's a terrible analogy because the entire point of ML systems is to generalize well to new data, not to reproduce the original data as accurate as possible with a space/time tradeoff.
The thing is that generalization is good enough to make people squee and not notice that the output is wrong but not good enough to get the right answer. If it were going to produce ‘explainable’ correct answers for most of what it does that would be a matter of looking up the original sources to make sure they really say what it thinks they do. I mean, I can say, “there’s this paper that backs up my point” but I hav…
Earlier quoted context omitted.
I’m just searching the comments for novel use cases where its effective. Most articles I’ve read seem like either moral panics or snake oil. I like how it can generate songs and poems based on a prompt. Its not particularly useful, but it is entertaining. It really does seem curated at times, leading me to think this will eventually become a fad or replaced by a more advanced algorithm.
its no fad. I use to spend an hour going over an email to management. make it less technical, make it sound nicer / polite, etc. now I can take a sentence and say 'make this more succinct. ok, but take that and make it more polite, great thanks'. I even used it for project proposals. 'write me a 1 page document for this idea'. and then I just use the scaffolding from there. its a huge time saver. also you are notn se…
This is very well written, and probably one of my favorite takes on the whole ChatGPT thing. This sentence in particular: > Indeed, a useful criterion for gauging a large-language model’s quality might be the willingness of a company to use the text that it generates as training material for a new model. It seems obvious that future GPTs should not be trained on the current GPT's output, just as future DALL-Es should…
I think what makes AlphaZero's recursion work is the objective evaluation provided by the game rules. Language models have no access to any such thing. I wouldn't even count user-based metrics of "was this result satisfactory": that still doesn't measure truth. I generally respect the heck out of Chiang but I think it's silly to expect anyone to be happy feeding a language model's output back into it, unless that out…
Does anyone have any idea how ChatGPT will actually make money? As novel as it is to use with all the "potential" applications, the possible revenue streams don't seem to prop up the recent investments into OpenAI. We've already been through enough hype cycles in the past ten years to realize "potential" use-cases or user counts don't necessarily produce a sustainable business model. Nor does a new innovative thing n…
With subscription models for their apis depending on the use scenario. That could be one reason they opened the service, to see what people are using it for in order to later build services around those use cases. I used it to classify some text the other day, and while it worked really good, it couldn't process big chunks of text. If they offered a pricing model per million characters I'd gladly pay it.
Does anyone have any idea how ChatGPT will actually make money? As novel as it is to use with all the "potential" applications, the possible revenue streams don't seem to prop up the recent investments into OpenAI. We've already been through enough hype cycles in the past ten years to realize "potential" use-cases or user counts don't necessarily produce a sustainable business model. Nor does a new innovative thing n…
"Search ads" through clever server-side auto-prompt-modification.
Earlier quoted context omitted.
Wholly agree. I worked there many years ago, leading the re-design and re-platform (fun dealing with 90 years of archival content with mixed usage-rights) and paywall implementation (don't hate me, it funds journalism). When you see how the stories get made and how people work there, well, its just amazing.
I notice that the extremists never use paywalls, meaning extremism is allowed to spread unchecked. If respectable newspapers and magazines cared about society, they'd follow suit, and give the extremists some competition.
Earlier quoted context omitted.
It's a terrible analogy because the entire point of ML systems is to generalize well to new data, not to reproduce the original data as accurate as possible with a space/time tradeoff.
I don't think you can describe the math in this context as "generalize well to new data." ChatGPT certainly can't generate new data. It's not gonna correctly tell you today who won the World Series in 2030. It's not going to write a poem in the style of someone who hasn't been born yet. But it can interpolate between and through a bunch of existing data that's on the web to produce novel mixes of it. I find the "blur…
> ChatGPT is so good at this form of interpolation that people find it entertaining: they’ve discovered a “blur” tool for paragraphs instead of photos, and are having a blast playing with it. “‘blur’ tool for paragraphs” is such a good way of describing the most prominent and remarkable skill of ChatGPT. It is fun, but so obviously trades off against what makes paragraphs great. It is apt that this essay against Chat…
This is very well written, and probably one of my favorite takes on the whole ChatGPT thing. This sentence in particular: > Indeed, a useful criterion for gauging a large-language model’s quality might be the willingness of a company to use the text that it generates as training material for a new model. It seems obvious that future GPTs should not be trained on the current GPT's output, just as future DALL-Es should…
I think what makes AlphaZero's recursion work is the objective evaluation provided by the game rules. Language models have no access to any such thing. I wouldn't even count user-based metrics of "was this result satisfactory": that still doesn't measure truth. I generally respect the heck out of Chiang but I think it's silly to expect anyone to be happy feeding a language model's output back into it, unless that out…
But, for a subset of topics, say, math and logic, a minimal set of core principles (axioms) is theoretically sufficient to derive the rest. For such topics, it might actually make sense to feed the output of a (very, very advanced) LLM back into itself. No reference to the real world is needed - only the axioms, and what the model knows (and can prove?) about the mathematical world as derived from those axioms.
Next, what's to say that a model can't "build theory", as hypothesized in this article (via the example of arithmetic)? If the model is fed a large amount of (noisy) experimental data, can it satisfactorily derive a theory that explains all of it, thereby compressing the data down to the theoretical predictions + lossy noise? Could a hypothetical super-model be capable of iteratively deriving more and more accurate models of the world via recursive training, assuming it is given access to the raw experimental data?