441–450 of 485 posts
Earlier quoted context omitted.
None of the GPT models rely on annotated or classified data. It's unsupervised.
The final training steps of gpt 3.5 did, and we assume the same for 4. The RLHF step.
None of this data is from Reddit, Twitter, etc.
Earlier quoted context omitted.
What’s wrong with: “it is often wrong”? I - or GPT - can think of various variations that do not have links with “lying”. I think it is a bad idea. Let people use it and see for themselves. It has unique failure modes, not unlike every other tool we have. You will only learn by interacting with it. Some people are in the habit of “protecting the people” but I am not quite sure that is necessary.
>>What’s wrong with: “it is often wrong”? Good q; the problem is the passive voice The passive voice connotes far less urgency and importance to any warning. Even "It is often dangerously wrong.", is still a more passive phrasing than "It will lie to you, with a highly confidant tone." I also cringe a bit at using active verbs that imply agency and intent in a statistical model; it makes an inaccurate implication abo…
How is this not “paternalistic”?
Why protect the people, from what? Is this worse than what they are already exposed to on a daily basis? Does facebook say it fucks you up and makes you an addict? Why this urge to enforce “truth in advertising”?
I am not saying you shouldn’t do it, it’s just that I do not get why and the article doesn’t mention it. It is somehow a given that The People need to be protected from this lying machine because they will revert to cannibalism if not properly informed.
Earlier quoted context omitted.
The Matrix turned out worse:)
If you just look at the jihad, maybe it was comparable (As Frank eluded to it). But then there was Muad'dib's subsequent campaign against the known universe which killed billions, Leto II's golden path spanning 3,500 years of tyranny and the famine times before scattering. If I had to choose, it would be the blue pill, though chairdogs do contribute to a strong counter-argument.
Earlier quoted context omitted.
Better than getting extincted, I suppose.
That's one of the possibilities, but if we're talking about those kinds of hypotheticals, allow me another one. If you recall, one of the themes of the later Dune books was that humanity was fleeing from the frontiers (The Scattering) back into the heartland from some unknown but extremely powerful enemy. Perhaps those were simply the neighboring species who did not have a Butlerian Jihad of their own?
Earlier quoted context omitted.
"simply having more data doesn’t turn a machine that can’t reason into one that can" But that's exactly what's been happening. LLM's are a triumph of "big data" used by dumb algorithms to create a reasoning machine.
> dumb algorithms I’m not an expert and haven’t seen architecture of ChatGPT. But it’s probably far from dumb. You can’t simply put more data into any algorithm and expect improvement. You need a model to be able to take advantage of the data and train in reasonable time. That’s exactly what’s been happening in ML field.
Not indefinitely, obviously. But we haven’t exhausted the available data sources yet, and the curves for prediction error vs. amount of training data strongly imply using this additional data would result in increased performance.
Do you have specific reasons for thinking that gpt-4 is near the end of this trend? I haven’t seen any compelling arguments for this.
Earlier quoted context omitted.
> Based on my understanding of the approach behind ChatGPT, it is probably very close to a local maximum in terms of intelligence Is this performance art? I mean it could end up right but I think you basically just made it up and then stated it confidently.
They didn’t state it particularly confidently though
"Based on my understanding" does convey some lack of confidence in a sense. But it's also implying they might have more understanding than other posters here, when the content of their post indicates they have less.
Earlier quoted context omitted.
I believe the relevant assertion is > Based on my understanding of the approach behind ChatGPT, it is probably very close to a local maximum in terms of intelligence To me that is an open question. OpenAI has not revealed a lot of the relevant stats behind GPT-4. I also haven't seen anything about their future LLM research (though I haven't looked very hard), but I think the above assertion remains to be seen.
It seems like people are focusing on training with more data and working the data. A human does not need to learn all the contents of the internet to learn skepticism and logic. A smart Ai should probably not either, that would point me to a different approach and those are not predictable. I have no doubt we can make gpt better and more useful. I do have some doubts we can make it trustworthy enough for most of its…
No, but children spend years learning basic mechanics, how to navigate 3-dimensional space, what causality is, etc. Underneath these laws of nature you find… logic and mathematical relationships. So I don't think it's surprising that a model whose perception is restricted to text (and now single images) as the sole sensory input/output might need much more training and data to learn logical reasoning.
Earlier quoted context omitted.
What on Earth are you about? Babi Yar is one of the single largest massacres of Jews during WW2. Some 30,000 Jews died in the first two days of the massacre. The overall SS commander in charge, Paul Blobel, was sentenced to death at Nuremberg. The event is widely remembered as a massacre of Jews. It's acknowledged by Ukraine as well. This is not "a Russian fake" and there is plenty of evidence for it. Please, don't r…
As I can see in Nuremberg Trials, Babi Yar was mentioned twice: 18 October 1945, Berlin In the Crimea peaceful citizens were gathered on barges, taken out to sea and drowned, over 144,000 persons being exterminated in this manner. In the Soviet Ukraine there were monstrous criminal acts of the Nazi conspirators. In Babi Yar, near Kiev, they shot over 100,000 men, women, children, and old people. In this city in Janua…
But I know what you're doing, actually: what every Holocaust denier always does.
Earlier quoted context omitted.
"The training algorithm is designed to create the most plausible text possible" That may be how they're trained, but these things seem to have emergent behavior.
I'm not sure why you call it "emergent" behavior. Instead, my take away is that much of what we think of as cognition is just really complicated pattern matching and probabilistic transformations (i.e. mechanical processes).
Earlier quoted context omitted.
I don't recall anything indicating that in Frank Herbert's original series. And Brian Herbert's "Omnius" nonsense seems to have very little to do with what Frank actually had in mind wrt Butlerian Jihad etc, so I don't consider it canonical.
Book 4, Sonia's vision with Leto strongly implies that the problem is the machines. Agreed that the endings done by others were incredibly bad.