Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cargo Cult AI

queue.acm.org

Post reply on HN
Print view View original

151–160 of 191 posts

Re: Cargo Cult AI

#151

Post by sebzim4500 »

neatze wrote: post #112
_yb2s wrote: post #79

The whole premise of this article hinges on the idea that LLMs have fundamental limitations that they clearly don’t have if you’ve looked at lots of gpt4 examples. For example, it can do scientific thinking if you specifically ask it to, and it can reason about totally new situations outside of the training data based on generalizable models of reality it creates to predict training data. If you are certain these lim…

> clearly don’t have if you’ve looked at lots of gpt4 examples for example, can you fine tune GPT to play chess at ELO 1600 ? If you don't know answer, you are in for surprise.

Given the lc0 policy network plays at a 2000+ strength on its own, I would expect that with enough finetuning gpt4 would be able to play way above 1600 strength.

It's possible that finetuning would basically be training a new network from scratch and the resulting network would forget everything apart from chess. It would be a really interesting experiment, GPT-2 is probably too small but I think llama-7B might be sufficient.

Re: Cargo Cult AI

#152

Post by bmc7505 »

sebzim4500 wrote: post #147
bmc7505 wrote: post #140

Earlier quoted context omitted.

Although I think their hearts are in the right place, AI safety researchers are primarily driven by irrational instincts and misjudge the promise and perils of artificial intelligence. If humanity chooses to turn away from God and sacrifice each other worshipping false idols, that will be our fault alone, whether or not the technology exists to hasten our demise. We do have thousands of Einsteins today wielding untol…

For the record, if you had told me earlier in the conversation that your faith in your convictions arise from your religious faith you would have saved us both a fair amount of time. Obviously, your arguments are never going to be convincing to someone who does not share your beliefs.

You asked about my beliefs and I engaged your question in good faith. Good day, Sebastian!

Re: Cargo Cult AI

#153

Post by TeMPOraL »

sandworm101 wrote: post #150
TeMPOraL wrote: post #145

Earlier quoted context omitted.

Here is me putting GPT-4 in a vaguely described maze, giving it an underspecified goal, making it a player in a game, myself acting as DM: https://cloud.typingmind.com/share/c0a68cb2-5f59-4e83-b383-b... I don't think GPT-4 is memorizing solutions here. I can see extrapolation and some degree of imagination in there, but of course you could say it's memorizing higher-level patterns. At some point though, you have to c…

A text maze. It is effectively brute-forcing a choose-your-own-adventure book. An aware mind would escape the maze by just skipping ahead to read the good parts of the book. That's what I did.

Not necessarily. It depends on what the mind wants.

You skipped ahead because you were impatient - or efficient - and didn't value the game anywhere near as much as the reward. GPT-4 just played along as instructed.

Now, if you were a player in a real RPG game, you'd probably play along with the story DM gave - because you'd care about the process more than about getting to the end.

Re: Cargo Cult AI

#154

Post by [deleted] »

celestialcheese wrote: post #129
kerkeslager wrote: post #121

Earlier quoted context omitted.

Don't cite stuff you didn't read or understand. [1] Is a summary of [2], by one of its authors, not a separate source. [2] Defines "emergent behaviors" in a way that you're clearly misunderstanding (because "emergent behaviors" is an extraordinarily poor way of communicating this--it's partly the fault of the researchers who chose this ambiguous language). All it's saying is that bigger models can do things that smal…

> [1] Is a summary of [2], by one of its authors, not a separate source. Yes. Your point? I included both because I found them both interesting. The paper is the source, the 137 emergent behaviours page is one of the authors continuing the work, and [3] is a journalist talking about this, so I included it as it's a unique perspective. I used the word "emergent" because that's what the SME used when describing this. F…

[deleted]

Re: Cargo Cult AI

#155

Post by tunesmith »

fsckboy wrote: post #64
tunesmith wrote: post #44

Earlier quoted context omitted.

There's absolutely no reason to be "brutally" honest. It's entirely possible to be respectful, clear, and concise all at the same time. And yes, as easy it is to read that sentence without the intensifier, it's also easy to write it without it as well.

> There's absolutely no reason to be "brutally" honest. You mean to say there's no absolute reason to be "brutally" honest, because then you can see there's no absolute reason to be smotheringly polite either. (did you read the brief piece I linked?) There absolutely is a reason to say what springs to your mind, it's quick and efficient, and that's something that people who quickly come up with quality thoughts prize…

Yup, I read it, and I read it months ago too, it's come up a few times.

What I'm saying is that it's absolutely possible to quickly and efficiently say those quality thoughts that springs to mind, in a respectful manner. And yes, it is a skill people should learn because it gives you a superset of advantages compared to if you don't. You can talk to a wider slice of people, and learn more from them as well. It's not very time-consuming or laborious, either. If I'm able to communicate my thoughts to you in a respectful manner, why would I change to be brutal about it? Once you know both, you tend to see that the brutality, aside from its other disadvantages, is simply superfluous.

Re: Cargo Cult AI

#156

Post by kerkeslager »

mustacheemperor wrote: post #146
kerkeslager wrote: post #134

Earlier quoted context omitted.

> Please don't comment on whether someone read an article. "Did you even read the article? It mentions that" can be shortened to "The article mentions that". This guideline is likely one of the main reasons Hacker News comments are so simultaneously overconfident and undereducated. If you want HN to be a safe place for people interrupting informed conversation with whatever nonsense pops into their head, fine, but I…

It seems that commenter did in fact read the article, so I for one as someone far less informed than both of you am interested to see the discussion carry on. On that note, >Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith. “You didn’t even read it” is not the strongest plausible interpretation of that reply nor assumes good…

> “You didn’t even read it” is not the strongest plausible interpretation of that reply nor assumes good faith.

Really? Did you read the last paragraph of my post that they were responding to? Can you explain that?

Re: Cargo Cult AI

#157

Post by xcv123 »

agalunar wrote: post #104
xcv123 wrote: post #65

Earlier quoted context omitted.

> Although LLMs are incredible feats of engineering, they're useless scientifically https://blogs.nvidia.com/blog/2022/09/20/bionemo-large-langu...

LLMs can be useful tools for conducting scientific research, in much the way that ordinary computer programs, or desk calculators, or slide rules are useful for conducting scientific research. I meant that (insofar as I am aware) they are not useful as models that we can study to understand the nature of human intelligence.

> They are not useful as models that we can study to understand the nature of human intelligence

Correct, but they were not explicitly designed to replicate human cognitive processes. That was never the goal. There is no pretense of them being a cognitive model that we can scientifically study.

Re: Cargo Cult AI

#158

Post by agalunar »

xcv123 wrote: post #157
agalunar wrote: post #104

Earlier quoted context omitted.

LLMs can be useful tools for conducting scientific research, in much the way that ordinary computer programs, or desk calculators, or slide rules are useful for conducting scientific research. I meant that (insofar as I am aware) they are not useful as models that we can study to understand the nature of human intelligence.

> They are not useful as models that we can study to understand the nature of human intelligence Correct, but they were not explicitly designed to replicate human cognitive processes. That was never the goal. There is no pretense of them being a cognitive model that we can scientifically study.

Agreed, but I get the impression many people mistakenly believe that they are.

Re: Cargo Cult AI

#159

Post by burnished »

throwaway675309 wrote: post #149
gyrovagueGeist wrote: post #116

Earlier quoted context omitted.

"It doesn't stop being magic just because you know how it works" - Terry Prachett

Just because we have a pithy quote doesn't detract from the fact that we shouldn't go all starry eyed and bandy about terms which obfuscate rather than explore the mechanics of how things work. Calling something "black magic" is too close to all of the anthropomorphism bullshit that seems to travel like a miasma around LLMs.

Magic is a garbage explanation for the function of a system, but the original context was some one expressing dismay at people being impressed and 'the thing resembles magic' is a great explanation for that.

Re: Cargo Cult AI

#160

Post by neatze »

sebzim4500 wrote: post #151
neatze wrote: post #112

Earlier quoted context omitted.

> clearly don’t have if you’ve looked at lots of gpt4 examples for example, can you fine tune GPT to play chess at ELO 1600 ? If you don't know answer, you are in for surprise.

Given the lc0 policy network plays at a 2000+ strength on its own, I would expect that with enough finetuning gpt4 would be able to play way above 1600 strength. It's possible that finetuning would basically be training a new network from scratch and the resulting network would forget everything apart from chess. It would be a really interesting experiment, GPT-2 is probably too small but I think llama-7B might be su…

I am not sure how to go about it, one way would be is to prompt it as MDP (seems like not fair), another way just prompt only as movies history (eg. I guess GPT will have to learn some form of MDP representation).

Most interesting aspect to me is if model can learn not make illegal moves ?

Post reply on HN