Live data from Hacker News

Telling GPT-4 you're scared or under pressure improves performance

aimodels.substack.com

251–255 of 255 posts

Re: Telling GPT-4 you're scared or under pressure improves performance

#251

Earlier quoted context omitted.

You didn't really address what og_kalu brought up. Which is that, it's possible that the model learns human like thinking, because that's the best way to accurately predict the human response itself. I generally agree with you, but still, I do think that this is the current question. What it is the model is learning that it then uses for predictions? Because you're assuming it's learning some purely token correlation…

I wasnt getting the sense it was worthwhile to engage, as my views werent being accurately understood. By I can address this. The meaning of words is, roughly, states of the world. If I say, "pass me the salt" that is satisfied if you, in fact, pass me the salt. If I say, "that tree is green" this is true if that tree which we are both talking about has the property of causing a perceptual state "seeming green" in bo…

I hear you, but I'm not sure it truly resolved the question of the premise.

You claim the data isn't there to learn human like thinking. But it's not substantiated. It's possible the sum total of all human writing does encode the core reasoning logic and function of humans, and that ML models could learn it from that.

You also seem to claim that without agency you cannot have real intelligence or human reasoning. But AI can be given agency, and it's already being worked on. And when it comes to tastes, convictions, etc., it's also something you can impart by just randomly seeding the AI a particular way which leans it towards certain preferences.

I do agree with you, the current AI models don't have our capacity to have a self driven learning process. They can't think of experiments to conduct to gather the missing data they think would help them know things with more certainty. We're able to learn and infer concurrently, and the two feed into each other almost in real time, and we have the capability to look for data, test hypothesis, etc., all in real time again.

The part I'm not convinced here either though, is that this can't be achieved with LLMs either.

Re: Telling GPT-4 you're scared or under pressure improves performance

#252

Earlier quoted context omitted.

> Is there some predictive power that we gain by reducing LLM skills to mere token production side effects? Yes. We can infer from this that out of sample hallucinations will be closer to the desired productions when measured by token productions metrics than by any other more informative domain relevant metrics. Which is exactly the case. If you ask llm to solve a math problem it haven't seen its response will be cl…

>If you ask llm to solve a math problem it haven't seen its response will be closer to the desired solution in its linguistic form rather than in mathematical meaning. This isn't true. Wild how people will confidently say nonsense about things they obviously haven't actually tested. GPT-4 can manage arbitrary arithmetic calculation closer to the real value than you could ever do without an external tool.

I don't mean calculations. I mean math problems.

Like those: https://www.mathschool.com/locations/andover/news/prepare-fo...

Re: Telling GPT-4 you're scared or under pressure improves performance

#253

Earlier quoted context omitted.

>If you ask llm to solve a math problem it haven't seen its response will be closer to the desired solution in its linguistic form rather than in mathematical meaning. This isn't true. Wild how people will confidently say nonsense about things they obviously haven't actually tested. GPT-4 can manage arbitrary arithmetic calculation closer to the real value than you could ever do without an external tool.

I don't mean calculations. I mean math problems. Like those: https://www.mathschool.com/locations/andover/news/prepare-fo...

ChatGPT-4 is quite capable of solving problems like that.

Here's it solving the last two problems on the Grade 7-8 Russian Math Olympiad from the linked page.

https://chat.openai.com/share/e90d4711-aa38-45c5-97e7-a9a04e...

https://chat.openai.com/share/0c9c579a-ca9f-4aa6-bac6-624372...

The two answers (84 and 225,792) agree with the answer key. I only gave it the questions and didn't give it the PDF either, so it didn't cheat and just read the answer off the answer key in the PDFs.

Re: Telling GPT-4 you're scared or under pressure improves performance

#254
post #192

So much disagreement in this thread over statements like LLMs “just predict the next token”. The thing to keep in mind - we know very little about how the human mind works. Thinking that somehow humans are special and then making conclusions about LLMs truly work based on that it just pointless IMO. Who knows. Maybe consciousness itself is a next word predictor. All we know now is that LLMs exhibit some emergent beha…

I'd like to read about your team's learnings on LLMs during the last months. Did you write any article/HN comment about it?

Re: Telling GPT-4 you're scared or under pressure improves performance

#255
post #192

So much disagreement in this thread over statements like LLMs “just predict the next token”. The thing to keep in mind - we know very little about how the human mind works. Thinking that somehow humans are special and then making conclusions about LLMs truly work based on that it just pointless IMO. Who knows. Maybe consciousness itself is a next word predictor. All we know now is that LLMs exhibit some emergent beha…

I'd like to read about your team's learnings on LLMs during the last months. Did you write any article/HN comment about it?

Some clues in my comments. But for the most part, I can’t share. We’ve built some cool stuff, but not enough to be “moaty” at the moment.
Post reply on HN