Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

thebullshitmachines.com

Post reply on HN
Print view View original

621–630 of 652 posts

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#621

Post by cristiancavalli »

consumer451 wrote: post #618
cristiancavalli wrote: post #611

Earlier quoted context omitted.

How does this prove reasoning? The thread you point to has several question in it that remain unanswered that ask the same question? How is this not entirely derivative too — there’s a huge number of these kind of 3-box “games” (although I don’t see this as a game really) so something very similar to this is probably in the training data a lot. Writing code to factor a number is definitely very common. Variation of t…

I was unable to find my exact "game" in google's index. Therefore, how does my example not qualify as this, at least: > Analogical reasoning involves the comparison of two systems in relation to their similarity. It starts from information about one system and infers information about another system based on the resemblance between the two systems. https://en.wikipedia.org/wiki/Logical_reasoning#Analogical

Is it actually reasoning though or just pattern matching? Seems like to compare one should also “know” which your above response indicates they do not.

I guess the real question is “does moving down a stochastic gradient of probabilities suffice as reasoning to you” and my awnser is no because you don’t need reason to find the nearest neighbor in this architecture. In this case the model is not actively comparing and inferring its simply associating without “knowing”

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#622

Post by consumer451 »

cristiancavalli wrote: post #621
consumer451 wrote: post #618

Earlier quoted context omitted.

I was unable to find my exact "game" in google's index. Therefore, how does my example not qualify as this, at least: > Analogical reasoning involves the comparison of two systems in relation to their similarity. It starts from information about one system and infers information about another system based on the resemblance between the two systems. https://en.wikipedia.org/wiki/Logical_reasoning#Analogical

Is it actually reasoning though or just pattern matching? Seems like to compare one should also “know” which your above response indicates they do not. I guess the real question is “does moving down a stochastic gradient of probabilities suffice as reasoning to you” and my awnser is no because you don’t need reason to find the nearest neighbor in this architecture. In this case the model is not actively comparing and…

There are many types of reasoning, and LLMs appear to do some of them.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#623

Post by cristiancavalli »

consumer451 wrote: post #622
cristiancavalli wrote: post #621

Earlier quoted context omitted.

Is it actually reasoning though or just pattern matching? Seems like to compare one should also “know” which your above response indicates they do not. I guess the real question is “does moving down a stochastic gradient of probabilities suffice as reasoning to you” and my awnser is no because you don’t need reason to find the nearest neighbor in this architecture. In this case the model is not actively comparing and…

There are many types of reasoning, and LLMs appear to do some of them.

Repeating a point without proffering evidence only makes it seems as if you don’t have anything to argue of substance.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#624

Post by cristiancavalli »

ninetyninenine wrote: post #619
cristiancavalli wrote: post #610

Earlier quoted context omitted.

I did and I fail to see how you can make those guarantees given you given it as a n interview question? You’re able to the vet the training data of O3? I still don’t see how your answer could only be arrived at via reasoning and that it would take “leaps of creativity” to arrive at the correct answer? These all seem like value judgments not hard data or some proof that your question cannot be derived from the trainin…

I don’t think you solved it otherwise you’d know that what I mean by variation is similar to how calculus is a variation of addition. Yea it involves addition but the solution is far more complicated. Think of it like this counting islands exists in the training data in the same way addition exists. The solution to this problem builds off of counting islands in the same way calculus builds off of addition. No trainin…

Yep and yep. Did it on two models and by myself — you know if you ask them to cite similar problems (and their sources) I’ll think you’ll quickly realize how derivative your question is in both question and solution. Given that you’re now accusing me of arguing in bad faith despite the fact I’ve listened to you repeat the same point with the only proof being “this question is a head scratcher for me; must be for everyone else therefore it proves that one must reason” makes me think you don’t actually want to discuss something; you think you can “prove” something and seem to be more interested in that. Given that I say go publish your paper about your impossible question and let the rest of the community review it if you feel like you need to prove something. So far the only thing you’ve proven to me is that you’re not interested in a good-faith discussion; just repeating your dogma and hoping someone concedes.

Also generalization is not always reasoning: I can make a generalization that is not reasoned; I can also make one that is poorly reasoned. Generalization is considered well-defined in regards to reasoning: https://www.comm.pitt.edu/reasoning

Your example still fails to actually demonstrate reasoning given its highly derivative nature, though.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#625

Post by consumer451 »

cristiancavalli wrote: post #623
consumer451 wrote: post #622

Earlier quoted context omitted.

There are many types of reasoning, and LLMs appear to do some of them.

Repeating a point without proffering evidence only makes it seems as if you don’t have anything to argue of substance.

[flagged]

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#626

Post by cristiancavalli »

consumer451 wrote: post #625
cristiancavalli wrote: post #623

Earlier quoted context omitted.

Repeating a point without proffering evidence only makes it seems as if you don’t have anything to argue of substance.

[flagged]

> I keep repeating myself because you seem unable to accept information.

I responded to your post with my thoughts and my own reframing of the question to try and open the conversation; you entirely ignored this in your response. Maybe you think this info is new to me but it’s not and it doesn’t prove anything for many more reasons than just the ones I cited in my prior response.

> Not sure if you are trolling now.

Nope just saying that adding “seem” to your postulation and repeating your point doesn’t make it right.

> I've had more productive conversations will LLMs.

The ad-hominem attacks really just serve to underscore how you have nothing of substance to argue.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#627

Post by ninetyninenine »

cristiancavalli wrote: post #624
ninetyninenine wrote: post #619

Earlier quoted context omitted.

I don’t think you solved it otherwise you’d know that what I mean by variation is similar to how calculus is a variation of addition. Yea it involves addition but the solution is far more complicated. Think of it like this counting islands exists in the training data in the same way addition exists. The solution to this problem builds off of counting islands in the same way calculus builds off of addition. No trainin…

Yep and yep. Did it on two models and by myself — you know if you ask them to cite similar problems (and their sources) I’ll think you’ll quickly realize how derivative your question is in both question and solution. Given that you’re now accusing me of arguing in bad faith despite the fact I’ve listened to you repeat the same point with the only proof being “this question is a head scratcher for me; must be for ever…

Yeah I know you claimed to solve it. I’m saying I don’t believe you and I think you’re a liar. There’s various reasons why the biggest one is that you think the solution is “generalizable” from counting islands (it’s not).

That’s not the point though. The point is I have metrics on this. Roughly 50 interviews only one guy got it. So you make the claim the solution is generalize-able well then prove your claim then. I have metrics that support my claim. Where’s yours?

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#628

Post by ninetyninenine »

cristiancavalli wrote: post #626
consumer451 wrote: post #625

Earlier quoted context omitted.

[flagged]

> I keep repeating myself because you seem unable to accept information. I responded to your post with my thoughts and my own reframing of the question to try and open the conversation; you entirely ignored this in your response. Maybe you think this info is new to me but it’s not and it doesn’t prove anything for many more reasons than just the ones I cited in my prior response. > Not sure if you are trolling now. N…

Reasoning is not understood even among humans. We only know the black box definition in the sense that whatever we are doing it is reasoning.

If an LLM arrives at the same output a human does given input and the output is sufficiently low probability to happen by random chance or association then it fits the term reasoning insofar as the maximum extent in which we understand it.

Given that we don't know what's going on the best bar is simply my matching input and output and making sure it's not using memory or pattern matching or random chance. There are MANY prompts that meet this criteria.

Your thoughts and claims are to be honest just flat out wrong. It’s just made up because not only do you know what the model is doing internally you don’t even know what you or any other human is doing. Nobody knows. So I don’t know why you think your claims have any merit. They don’t and neither do you.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#629

Post by owl_vision »

lithocarpus wrote: post #516
AlienRobot wrote: post #509

Earlier quoted context omitted.

You know, these days I think the abstracts are generated by LLMs too. And the paper. Or at least it uses something like Grammarly. If things keep going this ways typos are going to be a sign of academic integrity.

A proper LLM will include realistic rates of typos eventually. ;)

Amen, Eliza wins.

The humans' mistakes of irrational response still boggles my mind.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#630

Post by owl_vision »

nmca wrote: post #221

(while I work at OAI, the opinion below is strictly my own) I feel like the current version is fairly hazardous to students and might leave them worse off. If I offer help to nontechnical friends, I focus on: - look at rate of change, not current point - reliability substantially lags possibility, by maybe two years. - adversarial settings remain largely unsolved if you get enough shots, trends there are unclear - ig…

my english teacher reminded us the same. +1
Post reply on HN