Live data from Hacker News

We come to bury ChatGPT, not to praise it

danmcquillan.org

381–390 of 425 posts

Re: We come to bury ChatGPT, not to praise it

#381

Earlier quoted context omitted.

>> How do we know we’re not doing the same or similar in our brains? Because if I ask you to calculate 85 + 73 in your head, you will not start searching for the string that completes "85 + 73 = " with maximum probability, but instead try to add those two numbers using an algorithm you learned at school. Which means you can compute stuff, just like a computer, and not only guess stuff, just like an LLM.

You’re just repeating that computing is not predicting without explaining why. Also, LLMs are not grep. If the string doesn’t match, it will come up with something it deems plausible, like us. It’s even bad at math in eerily similar ways to us, like off-by-a-decimal-place errors.

>> You’re just repeating that computing is not predicting without explaining why.

Yes, because I assume you're a programmer, or a computer scientist, and you already understand the difference.

To clarify, the difference is that a program that computes the most likely next token in a sequence, like a language model (LM) is one kind of program, whereas the set of all program contains infinitely many programs that are not LLMs; and an LLM cannot calculate every such program, nor can it perform the same calculations as those programs, it can only calculate the most likely next token in a sequence.

Is the distinction clearer now? Also, are you a programmer? If so, I would point to the wikipedia article on Turing Machines or the one on the theory of computation. It's perhaps not a very easy subject to explain in a HN comment, but if you have some background in programming computers you should be able to grok it. You will see for example that a Turing machine is a device that manipulates symbols to produce other symbols, which is what we generally mean by "computation" and not a device that predicts the next token in a sequence, as an LLM does.

Sorry if I assume too much about your background - this is HN after all.

Re: We come to bury ChatGPT, not to praise it

#382

Earlier quoted context omitted.

ChatGPT helps a lot but it is necessary to take everything with a grain of salt - so what? It still can save a lot of time. Just Alt-tab ChatGPT/Google. Adjust your mental model from what you think it should be to what it actually is.

gem install hairball

No one gets the reference? Rich Hickey gave a talk where he speaks about how easy it is to import dependencies you didn't know you had. That means you were getting complications to your systems in an uncontrolled fashion because it's so easy to do.

Now imagine that same ease with ideas. It's so easy to just let ChatGPT write that up, I'm going to do that. I'm not even going to edit, even where what it wrote doesn't exactly reflect my thinking on the matter.

At that point, you're cluttering up your own thinking with output from ChatGPT because it's easy.

  gem install hairball

Re: We come to bury ChatGPT, not to praise it

#383

Earlier quoted context omitted.

Just had a dinner conversation where ChatGPT was characterized as automated plagiarism, and then I thought wouldn’t it be cool to get like a set of BibTex entries for all the sources whose content were combined to synthesize an output. Not sure that’s possible, and even if so, that it would be any kind of reasonable or manageable size whatsoever.

You'd see hallucination in the citations too. Ultimately, you can't get away from having to manually verify everything that an LLM outputs.

I run the cheaper self hostable OpenAI alternative https://text-generator.io I've been working on automating this manual verification of everything, with a few components we already have like a search engine and an edit API we can both detect and correct most of these errors to at least be reflective of what a reliable source says like Wikipedia, still a lot of reasoning, logic and math issues will remain, but there's a big step up coming soon in factual generation

Re: We come to bury ChatGPT, not to praise it

#384
post #341

Earlier quoted context omitted.

How well GPTZero can detect ChatGPT generated text by measuring perplexity (how random the words choices are) and burstiness (how diverse the sentence structure is) shows that whatever algorithm our brain uses has stronger creative capabilities than this LLM. GPT-3.5 isn’t a great writer like AlphaGo is a great go player. Maybe one day AI will generate better scripts and novels than humans, but not this model. Medium…

> it’s problematic when the model doesn’t know fact from fiction. This is in no way unique to an AI. Have you ever interacted with humans? Half the population thinks the other half can't tell fact from fiction. The other half thinks the same about the first half. We're all wrong about all the time.

There's a fundamental difference between "some people are wrong some of the time"—or even "half the population has trouble telling fact from fiction some of the time", if we grant that as true for the sake of argument—and "ChatGPT (and similar ML algorithms) don't even have a metric to determine truth from fiction; they just predict what's the most likely set of words to stick together in response to your prompt."

ChatGPT fundamentally cannot ever know when it's wrong. I should hope it goes without saying that that's not true of humans.

Re: We come to bury ChatGPT, not to praise it

#385

Earlier quoted context omitted.

>How so? Sorry, but to categorically affirm such bold claim you need more substance. It's not a bold claim, it's a trivial claim. No existing learning system can reason. If it could we could hand it axioms and it could do the same deductive reasoning that you can. By definition and how these systems work, all they do is generalize from particular examples, and you can't do maths by empirically looking for numbers in…

I don’t consider ChatGPT to be self aware. I think most people don’t. No one, however, can pinpoint exactly why. The vast majority of the population can’t deduce anything from axioms or even multiply large numbers. Yet most won’t doubt their self awareness. If you’re using “self-evident” and “trivial” when discussing AI, conscious, intelligence, you’re fooling yourself.

Picture a line—say, the line defined by y=2x + 3.

Now, tell me what's contained inside that line. Not "what does it mean" or "what's it made up of"; "what is contained inside it?"

The question doesn't make sense. There's no "inside" of a line. It's a one-dimensional mathematical construct. It fundamentally cannot "contain" anything.

"What's inside the line?" is a similar question to "Is ChatGPT self-aware?"—or, more aptly, to "What is ChatGPT thinking?" It's a static mathematical construct, and thinking is an active process. ChatGPT fundamentally cannot be said to be thinking, experiencing, or doing any of the other things that would be prerequisites for self-awareness, however you want to define that admittedly somewhat nebulous concept. Thus, to even ask the question "Why don't you think ChatGPT is self-aware?" doesn't make sense. It's not that far different from asking asking "Why don't you think your keyboard/pencil/coffee mug is self-aware?"

The intelligence of all humans is roughly analogous in ability—even if a given human has not learned to do formal logical deduction and inference, the fundamental structure and processing of the human brain is unquestionably capable of it, and most humans do so informally with no training at all.

Attempting to cast doubt on the human ability to reason, to comprehend, and to synthesize information beyond mere stochastic prediction reflects a very naïve, surface-level view of humans and cognition, and one that has no grounding in modern psychology or neuroscience. Your continued insistence, through several sub-threads, that we cannot be sure we are any better than ChatGPT is very much an extraordinary claim, and you have provided no evidence to support it beyond "I can't imagine a proof that we are not."

Maybe go do some research on how our brains actually work, and then come back and tell us if you still think we're all just predictive chatbots.

Re: We come to bury ChatGPT, not to praise it

#386
post #304

Earlier quoted context omitted.

You're not too far off from the arguments people made about the internet not too long ago. It's too easy to access information that could easily be incorrect...even maliciously so! You're better off sticking to published books and journals from respectable organizations that vet their authors and review their publications! Then again, who's in control of those printing presses? How can you trust the publishers to not…

The problem is people weren't wrong about the internet. In fact, they couldn't grasp the magnitude of the problem it would create, and the complete transformation it would have on media, politics, and culture.

As someone who vaguely remembers the 90s I can tell you that there is no transformation in media, politics or culture. Well, there is.

But the difference people describe, that politics used to be based on sound science and now, with the internet/facebook/fake news/tiktok changed to be based on total bullshit. Not true.

Not because media and politics aren't currently almost exclusively bullshit. But because that wasn't any different in the 90s. Back then media was full of bullshit, and politics reacted 100 times to media bullshit for every time it reacted to actual science. Hell, there's positive evolution too, I think the BBC has actually improved their fact checking since back then, for example. And I actually know what is trustworthy. I didn't know in the 90s.

Re: We come to bury ChatGPT, not to praise it

#388
post #13

It's nice to see a little realism starting to poke through the hype. The rush for automatic bullshit generation for me is very much in the bucket of, "Your scientists were so preoccupied with whether or not they could, they didn't stop to think if they should." As with every new technology, we should be looking at LLMs through a lens of maximizing the benefits and minimizing the harms.

But but but, there's money to be made!

Re: We come to bury ChatGPT, not to praise it

#390
post #356

Earlier quoted context omitted.

You never paraphrased something despite knowing the exact word? [edit] Or explained high level CS concepts to an undergrad at your school?

Sure, but how's that indicative that prediction isn't taking place?

I have the idea before i have thought of the word i would use to express it. Sometimes, i don't even have word to express it, "you know what i mean?".

And like i said. Children will often act, then talk. Or seek their words for long period of time, and when you propose a word to help them vocalize, they can accept it or reject it.

I have another example. Do you ever have slip of the tongue? You're talking about "yoga", tell yourself while speaking "i really shouldn't say 'yoda' here", and say the word "yoda" anyway? This happened to me a lot when i was younger (my speed of thought is lower those days, i don't think i slipped this way in two or three years). I still think a bit faster than i speak tbh.

Post reply on HN