Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

thebullshitmachines.com

Post reply on HN
Print view View original

511–520 of 652 posts

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#511

Post by bccdee »

Loughla wrote: post #506
Mistletoe wrote: post #478

Earlier quoted context omitted.

I’d take the Wikipedia answer any day. Millions of eyes on each article vs. a black box with no eyes on the outputs.

Even Wikipedia is a problem though. There are so many pages now that self-reference is almost impossible to detect. Meaning, the citation of a statement made on Wikipedia that uses an outside article for reference, which is an article that was originally written using that very Wikipedia article as its own citation. It's all about trust. Trust the expert, or the crowd, or the machine. They're all able to be gamed.

False equivalence. "Nothing is perfectly unreliable, therefore everything is (broadly) unreliable, therefore everything is equally unreliable." No, some sources are substantially more reliable than others.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#512

Post by cristiancavalli »

ninetyninenine wrote: post #502
cristiancavalli wrote: post #493

Earlier quoted context omitted.

No I can “prove” it — look at any number of cases where LLMs can’t even do basic value comparisons despite being claimed as super intelligent. You can try and say well that’s a limitation of the technology and then I would reply — yes and that’s why I would say it’s not reasoning according the original human definition. Also you have yet to produce any evidence of reasoning and claiming you can over and over again do…

LLMs can reason they just don’t always reason. That’s the claim everyone makes. That is a human definition if it reasoned one time correctly. That is the colloquial definition. Someone who has brain damage can reason correctly on certain subjects and incorrectly on other subjects. This is an immensely reasonable definition. I’m not being pedantic or out of line here when I say LLMs can reason while using this definit…

I still think the jury is out on this given that they seem to fail on obvious things which are trivially reasoned about by humans. Perhaps they reason differently at which point I would need to understand how this reasoning is different from a humans reasoning (perhaps biological reasoning more generally?) and then I would want to consider whether one ought to call it reasoning given its differences (if there are any at the time of sampling). I understand your claim I’m just not buying it based on the current evidence and my interacting with these supposed “super intelligences” every day. I still find these tools valuable, just unable to “reason” about a concept which makes me think, as powerful and meaning filled as language is, our assumption of reasoning might just be a trick of our brain reasoning through a more tightly controlled stochastic space and us projecting the concept of reasoning onto a system. I see the COT models contort and twist language in a simulacrum of “reasoning” but any high school English teacher can tell you there is a lot of text written that appears to logically reason but doesn’t actually do anything of the sort once read with the requisite knowledge in the subject matter.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#513

Post by tremon »

bccdee wrote: post #511
Loughla wrote: post #506

Earlier quoted context omitted.

Even Wikipedia is a problem though. There are so many pages now that self-reference is almost impossible to detect. Meaning, the citation of a statement made on Wikipedia that uses an outside article for reference, which is an article that was originally written using that very Wikipedia article as its own citation. It's all about trust. Trust the expert, or the crowd, or the machine. They're all able to be gamed.

False equivalence. "Nothing is perfectly unreliable, therefore everything is (broadly) unreliable, therefore everything is equally unreliable." No, some sources are substantially more reliable than others.

*perfectly reliable, but yes.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#514

Post by simianparrot »

ninetyninenine wrote: post #486
Miraste wrote: post #470

Earlier quoted context omitted.

Answering novel prompts isn't proof of reasoning, only pattern matching. A calculator can answer prompts it's never seen before too. If anything, I would come down on the reasoning side, at least for recent CoT models-but it's not a trivial question at all.

Then I'll come up with a prompt such that the answer can only be arrived at via reasoning. I only have to demonstrate this once to prove LLMs CAN reason.

Go ahead then.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#515

Post by meroes »

williamcotton wrote: post #405
meroes wrote: post #383

Earlier quoted context omitted.

My prior is reasoning is a conscious activity. There is a first person perspective. LLMs are so far removed mechanically from brains the idea they reason is not even remotely worth considering. Modeling neurons can be done with a series of pipes and flowing water, and that is not expected to give arise to consciousness either. Nor are nuerons and synapses likely to be sufficient for consciousness. You know how we ins…

LLMs are so far removed mechanically from brains the idea they reason is not even remotely worth considering. Jet planes are so far removed mechanically from a bird that the idea they fly is not even remotely worth considering.

You’re right that my argument depends upon there being a great physical distinction between brains and H100s or enough water flowing through troughs.

But since we knew properties of wings were major comments to flight dating back to beyond the myths of Pegasus or Icarus, we rightly connected the similarities in the flight case.

Yet while we have studied neurons and know the brain is apart of consciousness, we don’t know their role in consciousness like the wing’s for flight.

If you got a bunch if daisy chained brains and that started doing what LLMs do, I’d change my tune—because the physical substrates are now similar enough. Focusing on neurons, and their facsimilized abstractions, may be like thinking flight depending upon the local cellular structure of a wing, rather than the overall capability to generate lift, or any other false correlation.

Just because an LLM and a brain get to the same answer, doesn’t mean they got there the same way.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#516

Post by lithocarpus »

AlienRobot wrote: post #509
MetaWhirledPeas wrote: post #498

Earlier quoted context omitted.

Sounds fun, if only to compare it to the abstract.

You know, these days I think the abstracts are generated by LLMs too. And the paper. Or at least it uses something like Grammarly. If things keep going this ways typos are going to be a sign of academic integrity.

A proper LLM will include realistic rates of typos eventually. ;)

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#517

Post by Terr_ »

ninetyninenine wrote: post #486
Miraste wrote: post #470

Earlier quoted context omitted.

Answering novel prompts isn't proof of reasoning, only pattern matching. A calculator can answer prompts it's never seen before too. If anything, I would come down on the reasoning side, at least for recent CoT models-but it's not a trivial question at all.

Then I'll come up with a prompt such that the answer can only be arrived at via reasoning. I only have to demonstrate this once to prove LLMs CAN reason.

> Then I'll come up with a prompt such that the answer can only be arrived at via reasoning.

Dude, if you can formulate a question and prove an answer absolutely requires "reasoning" (defined how?) then you should drop everything and publish a paper on it immediately.

You'll have plenty of time to use your discovery to poke at LLMs after you secure your worldwide fame and recognition.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#518

Post by almosthere »

I cashed a check the other day with my name but it had the wrong address (at least an address my bank is not aware of). I asked google real quick - panicking it wouldn't go through. Google's AI came up and immediately told me to get the check re-issused, go through all this crazy hassle. First result after that, and all the results basically said - you'll be fine.

The check is cashed, and went through just fine. They only care about the name.

LLMs are bullshit machines for sure. That doesn't mean they have no value, but they can be wrong.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#519

Post by bccdee »

nmca wrote: post #314
jdlshore wrote: post #300

Earlier quoted context omitted.

I read the whole course. Lesson 16, “The Next-Step Fallacy,” specifically addresses your argument here.

The discourse around synthetic data is like the discourse around trading strategies — almost anyone who really understands the current state of the art is massively incentivised not to explain it to you. This makes for piss-poor public epistemics.

Yeah because if they explained that synthetic data causes model collapse, their stock valuation would shrink.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#520

Post by daveguy »

ninetyninenine wrote: post #507
more-nitor wrote: post #491

Earlier quoted context omitted.

your claim (hypothesis): LLMs can reason your evidence: "it works with these inputs I tried!" ...hmm seems you're not quite versed in basic mathematical proofs?

Seems you’re not well versed in basic English. If I can reason it doesn’t mean I’m always reasoning or constantly reasoning or if I know how to do reasoning for every prompt. It just means it’s possible. How narrow or how wide that possibility is, is orthogonal to the claim itself. Please employ logic here. Ok math guy. Imagine I said numbers can be divided. The claim is true even though there is a number that can’t…

If it's only reasoning randomly how do you know when anything has been reasoned properly vs just a generated simulation of reasonable text?
Post reply on HN