Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

thebullshitmachines.com

Post reply on HN
Print view View original

531–540 of 652 posts

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#531

Post by Terr_ »

jvanderbot wrote: post #258

> We (the authors of this website) have at times sought insight into the inner workings of an LLM by asking it “why did you just do that?” > But the LLM can’t tell us. It’s not a person. It doesn’t have the metacognitive abilities necessary to reflect on its past actions and report the motivations underlying them*. > With no clue why it did whatever it just did, the LLM is forced to guess wildly at a plausible explan…

> I am almost convinced that we ourselves are a narrator riding along inside an animal's mind

I think this is one of those "some truth, but not the whole truth" things. Yes, we trick ourselves, such as with mis-remembered reflex actions: "I felt it, it hurt, therefore I decided to move", even though the nerve-impulse speeds means your limb was moving before your brain even knew about the pain.

But the "narration" seems to be very important. We create stories to capture cause-and-effect about the world (unclear how much that requires language) and it seems to be beneficially adaptive. In fact this drive is so important that we do it even when we abstractly know it's wrong, like when flipping 50/50 coins and imagining a particular coin is luckier than another, or that you're on a "hot streak", or "now that other outcome is overdue."

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#532

Post by oblio »

falcor84 wrote: post #425
oblio wrote: post #421

Earlier quoted context omitted.

Your wife is one of the end products of cutthroat competition across several billion years so let's just say her general intelligence has a fair bit more validation than 20 years of research.

Sexual selection applies an evolutionary pressure against men who challenge women too much about the validity of their reasoning.

I was really, really trying to ignore the casual misogyny in OP's comment but you're really making this hard.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#533

Post by wruza »

AlienRobot wrote: post #508
wruza wrote: post #488

Earlier quoted context omitted.

it should have some checkboxes and numeric entries for some parameters, although I don't know what those parameters would be The only params they have are technical params. You may see these in various tgwebui tabs. Nothing really breathtaking, apart from high temperature (affects next token probability). Is generating natural language part of what an LLM is, or is this a separate program on top of what it does? They…

Thanks, that is very informative! I have heard about the tokenization process before when I tried stable diffusion, but honestly I can't understand it. It sounds important but it also sounds like a very superficial layer whose only purpose is to remove ambiguity, the important work being done by the next layer in the process. I believe part of the problem I have when discussing "AI" is that it's just not clear to me…

When I say LLMs, I mean literal large language models, like all of them in the general "Text-to-Text" && "Transformers" categories, loadable into text-generation-webui. Most people probably only have experience with cloud LLMs https://www.google.com/search?q=big+LLM+companies . Most cloud LLMs are based on transformers (but we don't know what they are cooking in secrecy) https://ai.stackexchange.com/questions/46288/are-there-any-n... . Copilot, Cursor and other frontends are just software that uses some LLM as the main driver, via standard API (e.g. tgwebui can emulate openai api). Connectivity is not a problem here, cause everything is really simple API-wise.

I have heard about the tokenization process before when I tried stable diffusion, but honestly I can't understand it. It sounds important but it also sounds like a very superficial layer whose only purpose is to remove ambiguity, the important work being done by the next layer in the process.

SD is special because it's actually two networks (or more, I lost track of SD tech), which are sort of synchronized into the same "latent space". So your prompt becomes a vector that basically points at the compressed representation of a picture in that space, which then gets decompressed by VAE. And enhanced/controlled by dozens of plugins in case of A1111 or Comfy, with additional specialized networks. I'm not sure how this relates to text-to-text thing, probably doesn't.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#534

Post by aerhardt »

uh_uh wrote: post #138
TheOtherHobbes wrote: post #106

Earlier quoted context omitted.

Many claims don't stand up to scrutiny, and some look suspiciously like training to the test. The Apple study was clear about this. LLMs and their related modal models lack the ability to abstract information from noisy text inputs. This is really obvious if you play with any of the art generators. For example - the understanding of basic prepositions just isn't there. You can't say "Put this thing behind/over/in fro…

> There is no abstracted concept of a "colour" in there. There's just a lot of imagery tagged with each colour name, and if you select a different colour you get a vector in a space pointing to different images. It has been observed in LLMs that the distance between embeddings for colors follows the same similarity patterns that humans experience - colors that appear similar to humans, like red and orange, are closer…

The difference is if I ask a 5 year old to re-draw the drawing in orange, they will understand exactly what I mean.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#535

Post by cristiancavalli »

UltraSane wrote: post #530
cristiancavalli wrote: post #454

Earlier quoted context omitted.

Disagree — proponents of this point still have yet to prove reasoning and other studies agree about “reasoning” being potentially fake/simulated: https://the-decoder.com/apple-ai-researchers-question-openai... Just claiming a capability does not make it true and we have 0 “proof” of original reasoning that can be proved coming from these models. Especially given the potential cheating in current SOTA benchmarks

When does a "simulation" of reasoning become so good it is no different than actual reasoning?

Love this question! Really touches on some epistemological roots and certainly a prescient question in these times. I can certainly see a theoretical where we could create this simulation in totality to our perspectives and then venture out into the universe to find that this modality of intelligence would be limited in its understanding of completely new empirical experiences/phenomenon that are outside our current natural definitions/descriptions. To add to this question: might we be similarly limited in our ability to perceive these alien phenomena? I would love to read a short story or treatise on this idea!

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#536

Post by ninetyninenine »

cristiancavalli wrote: post #512
ninetyninenine wrote: post #502

Earlier quoted context omitted.

LLMs can reason they just don’t always reason. That’s the claim everyone makes. That is a human definition if it reasoned one time correctly. That is the colloquial definition. Someone who has brain damage can reason correctly on certain subjects and incorrectly on other subjects. This is an immensely reasonable definition. I’m not being pedantic or out of line here when I say LLMs can reason while using this definit…

I still think the jury is out on this given that they seem to fail on obvious things which are trivially reasoned about by humans. Perhaps they reason differently at which point I would need to understand how this reasoning is different from a humans reasoning (perhaps biological reasoning more generally?) and then I would want to consider whether one ought to call it reasoning given its differences (if there are any…

They can fail at reasoning. But they can demonstrably succeed to.

So the the statement that they CAN reason is demonstrably true.

Ok if given a prompt where the solution can only be arrived at by reasoning and the LLM gets to the solution for that single prompt, then how can you say it can't reason?

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#537

Post by idunnoman1222 »

therein wrote: post #266
idunnoman1222 wrote: post #257

My 80s copy of the encyclopedia Britannica was riddled with errors, perhaps we will survive this post truth

The situation is closer to if we had 10,000 variants of Encyclopedia Britannica in 80s that all looked like distinct bodies of work, riddled with different errors while looking like they were written from scratch.

What is the difference to the end user? Our situation is better than it was yesterday not worse. If we had a genie who could appear out of nowhere and tell us the truth TM at any point that would kind of ruin the adventure .

Who reads the output of a book or Wikipedia or a website or an AI and thinks oh good now I know the core truth of this thing and I never need to update this knowledge ever again case closed

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#538

Post by wruza »

ninetyninenine wrote: post #504
wruza wrote: post #471

Earlier quoted context omitted.

You can prove that LLMs can reason by simply giving it a novel problem where no data exists and having it solve that problem They scan a hyperdimensional problem space whose facetness and capacity a single human is unable to comprehend. But there potentially exist a slice that corresponds to a problem that is novel to a human. LLMs are completely alien to us both in capabilities and technicalities, so talking about w…

Reasoning is an abstract term. It doesn’t need to be similar to human reasoning. It just needs to be able to arrive at the answer through a process. Clearly we used the term reasoning for many varied techniques. The term doesn’t narrow to specifically one form of “human” like reasoning only.

Oh, that is true. "It" doesn't have to do human reasoning, at all.

But we have to at least define "reasoning" for the given manifestation of "it". Otherwise it's just birdspeak. Because reasoning is "the action of thinking about something in a logical, sensible way", which has to happen somewhere if not finger-pointable, then at least somehow scannable or otherwise introspectable. Otherwise it's yet another omnidude in the sky who made it all so that you cannot see him, but there will be hints if you believe.

Anyway, we have to talk something specific, not handwavy. Even if you prove that they CAN reason for some definition of it, both the proof and the definition must have some predictive/scientific power, otherwise they are as useless as nil thought about it.

For example, if you prove that the reasoning is somehow embedded as a spatial in-network set of dimensions rather than in-time, wouldn't that be literally equivalent to "it just knows the patterns"? What would that term substitution actually achieve?

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#539

Post by ninetyninenine »

enragedcacti wrote: post #510
ninetyninenine wrote: post #468

Earlier quoted context omitted.

It’s stupid. You can prove that LLMs can reason by simply giving it a novel problem where no data exists and having it solve that problem. LLMs CAN reason. Whether it can’t reason is not provable. To prove that you have to give the LLM every possible prompt that it has no data for and effectively show it never reasons and gets it wrong all the time. Not only is the proof impossible but it’s already been falsified as…

LLMs CAN read minds. Whether it can’t read minds is not provable. Literally I invite people to post prompts and correct answers to ChatGPT where it is trivially impossible for it to have known what number you were thinking of. Every one of those examples falsifies the claim that LLMs can’t read minds.

ok prove it. I'm thinking of a number right now between 1-10,000. Show me the number the LLM guesses. You can definitively prove this statement for me.

It's a probability problem really. The range of a prompt has billions of possibilities. If it arrived at a correct answer within that range then the probability it got there without reasoning is miniscule.

Same with this mind reading thing. Prove it.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#540

Post by falcor84 »

oblio wrote: post #532
falcor84 wrote: post #425

Earlier quoted context omitted.

Sexual selection applies an evolutionary pressure against men who challenge women too much about the validity of their reasoning.

I was really, really trying to ignore the casual misogyny in OP's comment but you're really making this hard.

Well, for what it's worth, I believe that this evolutionary pressure works as strongly, or even more so, against women who challenge men about the validity of their reasoning.
Post reply on HN