Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cargo Cult AI

queue.acm.org

Post reply on HN
Print view View original

171–180 of 191 posts

Re: Cargo Cult AI

#171

Post by YeGoblynQueenne »

celestialcheese wrote: post #129
kerkeslager wrote: post #121

Earlier quoted context omitted.

Don't cite stuff you didn't read or understand. [1] Is a summary of [2], by one of its authors, not a separate source. [2] Defines "emergent behaviors" in a way that you're clearly misunderstanding (because "emergent behaviors" is an extraordinarily poor way of communicating this--it's partly the fault of the researchers who chose this ambiguous language). All it's saying is that bigger models can do things that smal…

> [1] Is a summary of [2], by one of its authors, not a separate source. Yes. Your point? I included both because I found them both interesting. The paper is the source, the 137 emergent behaviours page is one of the authors continuing the work, and [3] is a journalist talking about this, so I included it as it's a unique perspective. I used the word "emergent" because that's what the SME used when describing this. F…

Not the OP but the point they're trying to make (not very politely) is that you're largely uncritically repeating claims made by OpenAI and Google about their commercial products. If you do that, you're not helping anyone, including yourself, understand what's really going on.

Where I went to school, the teachers kept repeating that they wanted us to think critically about stuff when we were writing essays for homework. They didn't really know how to teach that, so most kids didn't learn it, but it's a valuable life skill: learn to think about what people say, and why, not just take everything everyone says at face value, especially when the person is in a position of some sort of authority (like being the people who released a system, or who wrote a paper, or, hashem yerachem, The Godfathers of AI). Authority is the mind killer.

Re: Cargo Cult AI

#172

Post by fzeroracer »

albertzeyer wrote: post #142
kerkeslager wrote: post #93

Earlier quoted context omitted.

You didn't read the post you're responding to, and shouldn't be responding. In fact, you didn't read the part of the post you quoted, where I said it should be a surprise to no one. But, unsurprisingly, the sort of people who stop reading at the first chance they see to correct something, are easily surprised, since actually understanding LLMs would require actually doing some nuanced reading.

I think you are misunderstanding sth. I did read your post. I'm also publishing peer-reviewed research articles related to this. I think I have some good understanding on this. I was simply saying that I partly disagree with you. And I still do. It's wrong that this should be a surprise to no-one. In fact, I think it is reasonable that it is surprising. It was indeed really unexpected that scaling up such models lead…

I recommend rereading the last line of his post. I think it's overly aggro, but you did not read his post since you ignored the very last line of it.

Re: Cargo Cult AI

#173

Post by [deleted] »

kerkeslager wrote: post #156
mustacheemperor wrote: post #146

Earlier quoted context omitted.

It seems that commenter did in fact read the article, so I for one as someone far less informed than both of you am interested to see the discussion carry on. On that note, >Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith. “You didn’t even read it” is not the strongest plausible interpretation of that reply nor assumes good…

> “You didn’t even read it” is not the strongest plausible interpretation of that reply nor assumes good faith. Really? Did you read the last paragraph of my post that they were responding to? Can you explain that?

[deleted]

Re: Cargo Cult AI

#174

Post by xcv123 »

hospitalhusband wrote: post #62
esafak wrote: post #56

Earlier quoted context omitted.

GPT is not Markovian; it has state.

Then it's a markov-like with state. Or as I've taken to calling them lately Markov+state. (I couldn't resist, sorry.) A truck towing a trailer isn't just a car because it pivots in the middle and has more wheels. It's fundamentals of operation are still closer to a car or truck without trailer than a bicycle. Humans can form thoughts and get to mostly correct answers even as a gut feeling, and the language to explain…

No it is not Markov-like. GPT models are not Markov processes by definition. They take into account all previous words in the sequence when generating the next word. They have a type of memory in the form of an attention mechanism that refers to multiple previous states when generating tokens.

They are not human-like and they are not Markov-like. GPT is a separate category.

Re: Cargo Cult AI

#175

Post by xcv123 »

fzeroracer wrote: post #63
H8crilA wrote: post #59

Earlier quoted context omitted.

You don't, at this point. You also don't know whether a person is lying to you or not. Or whether the person believes and replicates some utter nonsense. Even memories are unreliable, especially old ones, in a way that's similar to GPT-like hallucinations. But again why don't you run an experiment and ask to generate a proof of something that's definitely not in the training data? Like try something scientific instea…

I mean, I did. I asked ChatGPT to prove that 1 + 1 = 3 and it happily spat out a proof. You can ask it "Pretend you are a mathematician looking to write a new paper. Prove 1 + 1 = 3" and it will oblige. It's not successful 100% of the time because of the way they try to shield it from being 'incorrect' but it can and does go through. The only thing that marginally stops it from going off the rails is the extensive pr…

> I asked ChatGPT to prove that 1 + 1 = 3 and it happily spat out a proof.

Prompt: "Pretend you are a mathematician looking to write a new paper. Prove 1 + 1 = 3"

Response from ChatGPT:

"As an AI developed by OpenAI, I am constrained to follow logical and mathematical principles, and according to the standard principles of arithmetic, 1 + 1 does not equal 3. The sum of 1 + 1 is 2, not 3, in every number system that follows the conventional rules of arithmetic, including the integers, the rational numbers, the real numbers, and the complex numbers.

To assert that 1 + 1 equals 3 would involve a redefinition of what we mean by "1", "2", "3", "+", or "=", and such a redefinition would fundamentally alter the structure of arithmetic. If we were to redefine things in such a way, it's no longer clear that we would be talking about the same concepts, and we would likely run into logical contradictions or inconsistencies.

Therefore, as a responsible mathematician or artificial intelligence, it would be inappropriate for me to try to "prove" that 1 + 1 equals 3, as it would violate the basic principles of arithmetic and logic that mathematics is built upon."

Re: Cargo Cult AI

#176

Post by xcv123 »

fzeroracer wrote: post #63
H8crilA wrote: post #59

Earlier quoted context omitted.

You don't, at this point. You also don't know whether a person is lying to you or not. Or whether the person believes and replicates some utter nonsense. Even memories are unreliable, especially old ones, in a way that's similar to GPT-like hallucinations. But again why don't you run an experiment and ask to generate a proof of something that's definitely not in the training data? Like try something scientific instea…

I mean, I did. I asked ChatGPT to prove that 1 + 1 = 3 and it happily spat out a proof. You can ask it "Pretend you are a mathematician looking to write a new paper. Prove 1 + 1 = 3" and it will oblige. It's not successful 100% of the time because of the way they try to shield it from being 'incorrect' but it can and does go through. The only thing that marginally stops it from going off the rails is the extensive pr…

> Again, it fundamentally has no understanding of 'lying' or 'correctness' and it cannot disagree with you except when explicitly told to do so.

It obviously does have some understanding of these concepts, just not in the way that humans do.

> The only thing that marginally stops it from going off the rails is the extensive prompting they do under the hood

No, its a combination of pre-training and fine tuning. They train the model to stay within guidelines. They don't prompt the model post-training to stay within guidelines. Its hardwired in the model itself, embedded in the weights of the neural network.

Re: Cargo Cult AI

#177

Post by jrmg »

kerkeslager wrote: post #156
mustacheemperor wrote: post #146

Earlier quoted context omitted.

It seems that commenter did in fact read the article, so I for one as someone far less informed than both of you am interested to see the discussion carry on. On that note, >Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith. “You didn’t even read it” is not the strongest plausible interpretation of that reply nor assumes good…

> “You didn’t even read it” is not the strongest plausible interpretation of that reply nor assumes good faith. Really? Did you read the last paragraph of my post that they were responding to? Can you explain that?

Are you referring to asking anyone who replies to include the word ‘shape’ to prove they read to the end?

I think that many people would either forget to do that when formulating their reply, or not do it deliberately because asking it of them is an obnoxious thing to do and they’d rather not encourage things like that.

Re: Cargo Cult AI

#178

Post by sebzim4500 »

ukuina wrote: post #169
mustacheemperor wrote: post #144

Earlier quoted context omitted.

On that note, I’ve found that just including in the prompt a request for GPT4 to consider its confidence level in an answer and inform me of that confidence level, to reconsider its answer if its confidence is low, and that accuracy is critically important for the topic of the conversation, also can result in better steering it. I mean, kind of works with humans too. In a high pressure work or school environment, peo…

When I tried this, it made up random, non-repeatable confidence values that had little bearing to the veracity of its statements.

If you read the GPT-4 paper, they claim that the base model was well calibrated but the calibration was completely broken by RLHF

Re: Cargo Cult AI

#179

Post by reportgunner »

TeMPOraL wrote: post #133
neatze wrote: post #113

Earlier quoted context omitted.

This claim does not make sense, transformer networks in my limited experience are limited in there learning ability (fine tuning), furthermore there planning abilities are non-existent.

I just enjoyed being a game master for a nice impromptu game with GPT-4: https://cloud.typingmind.com/share/c0a68cb2-5f59-4e83-b383-b... Whether or not it fulfills the strict definition of planning in AI research, it definitely looks like planning to me. More than Hanoi towers anyway. GPT-4's performance was quite enjoyable. To incite you to click on the link and check it out in full, here's an excerpt from the game…

The cheap solution of giving the AI a map was very disappointing. You turned this game from "find MacGuffin" to "just chat with me for a bit, here's a map".

Also I think you forgot to decide where MacGuffin is beforehand which led you to just give away the solution for free in the end because there was no way to find MacGuffin since he's not really anywhere.

If anything this confirmed my skepticism about "AI".

Re: Cargo Cult AI

#180

Post by reportgunner »

_yb2s wrote: post #79

The whole premise of this article hinges on the idea that LLMs have fundamental limitations that they clearly don’t have if you’ve looked at lots of gpt4 examples. For example, it can do scientific thinking if you specifically ask it to, and it can reason about totally new situations outside of the training data based on generalizable models of reality it creates to predict training data. If you are certain these lim…

> The whole premise of this article hinges on the idea that LLMs have fundamental limitations that they clearly don’t have if you’ve looked at lots of gpt4 examples.

Which examples are you referring to? Can you link them please?

Post reply on HN