291–300 of 309 posts
I see ChatGPT as a sleazy sales guy. Very good at well spoken elaborate stories. Will have a confident answer to all of your questions. Will prefer to tell you bullshit instead of just saying "I don't know". And there lies also the problem, you will never know if ChatGPT really knows the answer, or is just bullshitting you. Just like a sleazy sales guy. So as an engineer, I'm not scared yet that my job is in jeopardy…
Earlier quoted context omitted.
One of the problems with ChatGPT is that for the most part, the things that people are most likely to feed it are also the things it is most likely to have training data for. So, you say, "Write me a heap for Java", and it spits out amazing, wonderful code that is almost right. Well, yeah, that's cool and all, though I would point out that its training data probably included a heap that was entirely correct, but, sti…
"Guys the first version of a machine to generate code hasn't put me out of a job, it's obviously bollocks." !remindme 10 years
This is a specific point about people being deceived by the very questions they ask, not a generalized comment about the AI.
It is a shame that Mr. Wolfram cannot write about things without making it 75% about himself. I once bought a book he wrote about great scientists, each chapter about a different scientist. I thought "This guy's stuff is usually so self promotional it is kind of gross, but this will be fun to see his take on these other people". The book was still about him. Amazing.
Earlier quoted context omitted.
> but the fact is that it didn't offer these options, and instead it offered the correct one ...because you gave it a "correct" prompt; or in other words, you gave it a prompt that ChatGPT can respond "successfully" to. But if you keep trying, you will surely find yourself with "failed" responses. And ChatGPT has no way of knowing the difference. That's my point. > Is the other "completely valid and possible output"…
Ok, I think this has moved on from the original point. I quoted you saying: "As I see it, there is clearly no way to advance ChatGPT into anything more than it is today. Impressive as it is, the curtain is wide open for all to see, and the art can be viewed plainly as what it truly is: magic, and nothing more." I was disagreeing with this, and showing ways it can advance. Your reply focuses on the fact that it will s…
That's my whole point. It can't. There is no room for the other system. Any other system will always be either too late in the process to work, or a complete replacement.
> Sometimes, we'd be able to feed its own input back to itself and ask it to evaluate if the previous output it gave is what we're after
And that doesn't accomplish what "evaluate" actually means. It only does the same thing it always does (find something semantically close to "evaluate the thing"), but with a new prompt. If you are lucky, the new prompt will be responded to with output you like, but only if you are lucky. If we keep doing this process, we are effectively guessing what password will match a hash. That's called "brute forcing", and the fact it doesn't work is why your passwords are secure.
> Doesn't matter if it fails sometimes
It fails every time. Sometimes it "fails up". Stumbling into success is still stumbling.
If I write a program that outputs random noise into a filter that hides all but the output I want, then eventually I will see that output. But crucially, the input is still random noise. What is the value of ChatGPT if it's no more than curated randomness?
My point is that there is a very clear difference between impressive and functional. ChatGPT is impressive, not functional. You can't make it functional. You can only make it better at impressing you.
Earlier quoted context omitted.
"Guys the first version of a machine to generate code hasn't put me out of a job, it's obviously bollocks." !remindme 10 years
I cannot speak for anyone else, but the disappointment for me came from abundance of clickbaitey articles/videos saying stuff like “ChatGPT rewrote my website 10x better than me!!!” I don’t dispute that it’s super cool technology made by some equally cool people, and honestly I would be perfectly fine with it putting me out of the job if it gets to that point.
The one thing I want everyone to understand about ChatGPT: ChatGPT interfaces with semantics , and not logic . -- That means that any emergent behavior that appears logically sound is only an artifact of the logical soundness of its training data. It can only echo reason. The trouble is, it can't choose which reason to echo! The entire purpose of ChatGPT is to disambiguate, but it will always do so by choosing the mo…
Seems trivial to me to bolt on some critical thinking filters. Right now ChatGPT acts like pure Amygdala. You poke it, and it spits out something that looks right based on what it was trained on. This is fine, it's just a demo, and this gives a really deep understanding of the direct limitations of these sorts of LLMs. Ideally a more mature chat bot would just do what I do in situations where some critical thinking o…
Earlier quoted context omitted.
I would wager that it will eventually be able to do math and logic, and may already be able to with the right prompting. It can follow instructions quite well, and multi step problems can be solved by breaking the problem into small digestible steps, and having it write out each step, just the same as humans do. If it can't do this yet, it's likely just that it doesn't have enough parameters yet. But I don't see why…
An infinite amount of "fine tuning" would solve any problem, yes... ...but what we just described isn't a language processor anymore: it's just a database. There is no need to use a neural network if you already have a known response to every possible input! The entire point of ChatGPT in the first place is to generate a response that doesn't exist in the training data. If all we needed was to look up the answer we w…
My point here is even if it only "gets" the semantics, it has the ability to perform logic. It's just not very efficient. And, I'd say this isn't that far off from what is happening in our brains.
Do we really "get" logic, or do we rely on heuristics? Why do we know that 12 * 2 is equal to 24? Either because we remember our multiplication tables (look up from a table) or because we break it into smaller steps until we're left with pieces of the problem that we inherently know (including 12 * 2 means 12 + 12 which means take 12 and increment the number +1 for 12x times, or 10 + 10 = 20 and 2 + 2 = 4 so 12 + 12 = 24).
I don't see why that couldn't reasonably scaled up to advanced calculus.
I think the point that you're hitting on is that LLMs aren't "full brains" that have all of the components that human brains have, and that's true. But LLMs appear to essentially have the ability to simulate (or replicate depending on how far you want to go) executive function. As others have hit on, from that, you can either command other specialized components to be able to do specific tasks (like recall facts, perform logic, etc) or you can have it break down the problem into steps. Then take the output from that processing and form a coherent response.
The structure here is flexible enough that I'm struggling to find the upper limit, and if all research in the development of LLMs stopped tomorrow, and everyone focused on building with LLMs, I still don't think that limit will be found.
The one thing I want everyone to understand about ChatGPT: ChatGPT interfaces with semantics , and not logic . -- That means that any emergent behavior that appears logically sound is only an artifact of the logical soundness of its training data. It can only echo reason. The trouble is, it can't choose which reason to echo! The entire purpose of ChatGPT is to disambiguate, but it will always do so by choosing the mo…
I would wager that it will eventually be able to do math and logic, and may already be able to with the right prompting. It can follow instructions quite well, and multi step problems can be solved by breaking the problem into small digestible steps, and having it write out each step, just the same as humans do. If it can't do this yet, it's likely just that it doesn't have enough parameters yet. But I don't see why…
ChatGPT doesn't "know" anything, its just a prediction engine. if you ask it what 2+2 is, it will tell you 4, but it didnt actually do math. it only said 4 because everyone else said it.
there is no generalizable reasoning going on there.
Earlier quoted context omitted.
> There was that microsoft chatbot which “turned nazi”.[1] They then had to turn it off. At the time, I thought it was incredible that Microsoft was held responsible for this! so much so that they had to turn it off because of all the blame. It was really weird and it made no sense. Here we are again.
The problem was that it was made useless. If you ask random people to teach your chatbot, you are at the mercy of those random people. Sure, MS could have left it on, and no one would use it except for two groups: actual neoNazis, and people who think a Nazi chatbot is funny. The mistake MS made was they forgot that people suck, or, at least, enough people suck that what they did was a bad idea.
