Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

thebullshitmachines.com

Post reply on HN
Print view View original

601–610 of 652 posts

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#601

Post by mhast »

AlienRobot wrote: post #508
wruza wrote: post #488

Earlier quoted context omitted.

it should have some checkboxes and numeric entries for some parameters, although I don't know what those parameters would be The only params they have are technical params. You may see these in various tgwebui tabs. Nothing really breathtaking, apart from high temperature (affects next token probability). Is generating natural language part of what an LLM is, or is this a separate program on top of what it does? They…

Thanks, that is very informative! I have heard about the tokenization process before when I tried stable diffusion, but honestly I can't understand it. It sounds important but it also sounds like a very superficial layer whose only purpose is to remove ambiguity, the important work being done by the next layer in the process. I believe part of the problem I have when discussing "AI" is that it's just not clear to me…

If you want to get a better understanding of this I recommend playing around in the "chat playgrounds" on some of the engines.

The Google one allows for some free use before you have to pay for tokens. (Usually you can buy $5 worth of tokens as a minimum and that will give you more than you can use up with manual requests.)

https://aistudio.google.com/prompts/new_chat

This UI allows you to alter the system prompt (which is usually hidden from the user on eg ChatGPT) and change to different models and change parameters. And then you give it the chat input similar as any other site.

You can also install a program like "LM Studio" and that will allow you to download models (through the UI) and run locally on your own machine. This gives you a similar interface to what you see in the Google AI Studio but you run it locally. And with downloaded models. (The model you download is the actual LLM which is basically very large amount of parameters you combine with the input tokens to get the next token the system outputs.)

For a more fundamental introduction to what all these systems do there are a number of Computerphile videos which are quite informative. Unfortunately I can't find a good playlist of them all but here's one of the early ones. (Robert Miles is in many of them.) https://www.youtube.com/watch?v=rURRYI66E54

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#602

Post by reportgunner »

aidos wrote: post #13

This is amazing! I was speaking to a friend the other day who works in a team that influences government policy. One of the younger members of the team had been tasked with generating a report on a specific subject. They came back with a document filled with “facts”, including specific numbers they’d pulled from a LLM. Obviously it was inaccurate and unreliable. As someone who uses LLMs on a daily basis to help me bu…

Wait, the people who click phishing links now think AI output is facts ? Imagine my shock.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#603

Post by mhast »

s2radhak wrote: post #342

Fascinating. The article repeatedly makes the claim that “LLMs work by predicting likely next words in a string of text”. Yet there’s the seemingly contradictory implication that we don’t know how LLMs work (ie we don’t know their secret sauce). How does one reconcile this? They’re either fancy autocompletes, or magic autocompletes (in which case the magic qualifier seems more important in understanding what they are…

These system work by taking a list of tokens (basically words) and the "model" and send that to a function which returns one new token.

You add that new token to the list of tokens.

Repeat with the new list of tokens.

That's how these systems work. We don't know exactly how the model works (wrt input tokens) but even that is a simplification. It's not magic. Just maths that's too complex to understand trivially.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#604

Post by appleorchard46 »

I think this takes too much of a sort of top-down approach. Many broad statements are made about what LLMs are and aren't - some positive, some negative, some well-substantiated, some less so - when there needs to be a greater focus on fundamental knowledge of their inner workings and (im)practical uses.

The title being a question implies it will teach you to answer the question yourself. But it feels more like you're expected to enter with the belief that they're oracles, and this is here to convince you they're bullshit machines.

I don't care which one it is! It doesn't matter if we call what they do logical reasoning or not, because either way it doesn't help to give a full understanding of their actual capabilities.

Much of the language surrounding this course makes me think the intention is a grounded view. If that is the case though, this misses the mark. Rather than educate on the reality of the situation, I fear the use of it would only exacerbate the principles-first approach seen all too often in discussion around AI. More accurate principles than most perhaps, but principles nonetheless.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#605

Post by consumer451 »

cristiancavalli wrote: post #475
ninetyninenine wrote: post #468

Earlier quoted context omitted.

It’s stupid. You can prove that LLMs can reason by simply giving it a novel problem where no data exists and having it solve that problem. LLMs CAN reason. Whether it can’t reason is not provable. To prove that you have to give the LLM every possible prompt that it has no data for and effectively show it never reasons and gets it wrong all the time. Not only is the proof impossible but it’s already been falsified as…

Saying something again does not provide proof of its actual veracity. Writing it in caps does not make it true despite the increased emphasis. I default to skepticism in the face of unproven assertions: if one can’t prove that they reason then we must accept the possibility that they do not. There are myriad examples of these models failing to “reason” about something that would trivial for a child or any other human…

Here was my test at ChatGPT 3.5.[0] I made up a novel game, and it figured it out. The test is simple, but it made me doubt absolute arguments that LLMs are not able to reason, in some way.

There is a question at the end of that comment, would love to hear other options.

[0] https://news.ycombinator.com/item?id=35442147

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#606

Post by cristiancavalli »

ninetyninenine wrote: post #596
cristiancavalli wrote: post #544

Earlier quoted context omitted.

Given your set of theoreticals then I would concede, yes the model is reasoning. At that point, though, the world would probably be far more concerned with your finding of a question that can only be met via reasoning and would be uninfluenced or paralleled by any empirical phenomenon including written knowledge as a medium of transference. The core issue I see here is you being able to prove that the model is actual…

https://news.ycombinator.com/item?id=43011839

This looks neat but I don’t think it meets the standard for “reasoning only.” (Still not sure how you would prove that one) furthermore this looks to be fairly generalizable in pattern+form to other grid problems so i don’t think it also meets the bar for “not being in the training data.” We known these models can generalize somewhat based upon their training but not consistently and certainly not consistently well. Again I’m not making the claim that responding to a novel prompt is a sign of reasoning as other have pointed out a calculator can do that too.

Your quote: “This is a unique problem I came up with. It’s a variation on counting islands.” You then say: “ as I came up with it so no variation of it really exists anywhere else.”

So not sure what to take away from your text but I do think this is a variation of a well-known problem type so I would be pretty amazed if there was something very close to this in the training data. Given it’s an interview question and those are written about ad-nauseum I’m not surprised then that it was able to generalize to the provided case. The COT researchers did see the ability to generalize in some cases just not necessarily actually use the COT tokens to reason and/or failed on generalizing on variations which they thought it should have given its ability to generalize in others and the postulation that it was using reasoning and not just a larger corpus to pattern match with.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#607

Post by sgt101 »

remich wrote: post #565
sgt101 wrote: post #68

Earlier quoted context omitted.

They don't have any concept of falsehood either, so this is very different from a human making things up with the knowledge that they may be wrong.

But sometimes when humans make things up they also don't have the knowledge they may be wrong. It's like the reference to "known unknowns" and "unknown unknowns". Or Dunning-Kruger personified. Basically you have three categories: (1) Liars know something is false and have an intent to deceive (LLMs don't do this) (2) Bullshitters may not know/care whether something is false, but they are aware they don't know (3) Bu…

It's super interesting.

There are two levels...

The pZombie type level where we look at the LLM as if it were a black box and simply account for its behavior. At this level LLM's claim to have knowledge and also claim knowledge of their limited knowledge "I can't actually taste". So approached from this direction we are in (2) they have awareness that there are some things that they don't know, but this awareness doesn't prevent them from pretending to this knowledge.

If we consider it from the perspective of knowing what's happening inside LLM's then I think the picture is different. The LLM is doing next word prediction with constant compute time per token - the algorithm is quite clear. We know this is true because it runs on llama.cpp or mlx on our macbooks as well on the farms of B200's that we fear will destroy the atmosphere. So LLM's don't have any actual operational knowledge of the logic of their utterances (dunning kruger, dunning kruger...) What I mean is that the LLM can't/isn't analysing what it says, it's just responding to stimulus. Humans do do this as well - it's easy to just chatter away to other people like a canary, but humans also can analysis what they are saying and strategically manipulate the messages that they create. So I would say that LLMs cannot be concerned about what they do or don't know - the concern rests with us when we challenge them (or not) by asking "how can you know that chocolate tastes better than strawberry - you have never tasted either".

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#608

Post by ninetyninenine »

consumer451 wrote: post #605
cristiancavalli wrote: post #475

Earlier quoted context omitted.

Saying something again does not provide proof of its actual veracity. Writing it in caps does not make it true despite the increased emphasis. I default to skepticism in the face of unproven assertions: if one can’t prove that they reason then we must accept the possibility that they do not. There are myriad examples of these models failing to “reason” about something that would trivial for a child or any other human…

Here was my test at ChatGPT 3.5.[0] I made up a novel game, and it figured it out. The test is simple, but it made me doubt absolute arguments that LLMs are not able to reason, in some way. There is a question at the end of that comment, would love to hear other options. [0] https://news.ycombinator.com/item?id=35442147

My thread has been voted down and it’s getting stale. The few remaining people are biased towards there point of view and are unlikely to entertain anything that will trigger a change in their established world view.

Most people will use this excuse to avoid responding to or even looking at your link here. It is compelling evidence.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#609

Post by ninetyninenine »

cristiancavalli wrote: post #606
ninetyninenine wrote: post #596

Earlier quoted context omitted.

https://news.ycombinator.com/item?id=43011839

This looks neat but I don’t think it meets the standard for “reasoning only.” (Still not sure how you would prove that one) furthermore this looks to be fairly generalizable in pattern+form to other grid problems so i don’t think it also meets the bar for “not being in the training data.” We known these models can generalize somewhat based upon their training but not consistently and certainly not consistently well.…

It’s a variation on a well known problem in the sense that I just added some unique rules to it.

The solution however is not a variation. It requires leaps of creativity that most people will be unable to solve. In fact I would argue this goes beyond just reasoning as you have to be creative and test possibilities to even arrive at a solution. It’s almost random chance that will get you there. Simple reasoning like logical reduction won’t let you arrive at a solution.

Additionally this question was developed to eliminate pattern matching that candidates use on software interviews. It was vetted and verified to not exist. No training data exists.

It definitively requires reasoning to solve. And it is also unlikely you solved it. ChatGPT o3 has solved it. Try it.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#610

Post by cristiancavalli »

ninetyninenine wrote: post #609
cristiancavalli wrote: post #606

Earlier quoted context omitted.

This looks neat but I don’t think it meets the standard for “reasoning only.” (Still not sure how you would prove that one) furthermore this looks to be fairly generalizable in pattern+form to other grid problems so i don’t think it also meets the bar for “not being in the training data.” We known these models can generalize somewhat based upon their training but not consistently and certainly not consistently well.…

It’s a variation on a well known problem in the sense that I just added some unique rules to it. The solution however is not a variation. It requires leaps of creativity that most people will be unable to solve. In fact I would argue this goes beyond just reasoning as you have to be creative and test possibilities to even arrive at a solution. It’s almost random chance that will get you there. Simple reasoning like l…

I did and I fail to see how you can make those guarantees given you given it as a n interview question? You’re able to the vet the training data of O3? I still don’t see how your answer could only be arrived at via reasoning and that it would take “leaps of creativity” to arrive at the correct answer? These all seem like value judgments not hard data or some proof that your question cannot be derived from the training data given you say it is a variation of. Seems like you have an interview question not “proof of reasoning” especially given the prior cited case of these models being able to generalize in some cases with enough data.

“And it is also unlikely you solved it” well I guess you overestimated your abilities on two counts today then.

> It’s a variation on a well known problem in the sense that I just added some unique rules to it.

> No training data exists.

No it definitely does but is a variation. You kinda just confirmed what we already knew. Given enough data about a thing these LLMs can generalize somewhat.

Post reply on HN