Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Human mathematicians are being outcounterexampled

xenaproject.wordpress.com

Post reply on HN
Print view View original

101–110 of 265 posts

Re: Human mathematicians are being outcounterexampled

#101

Post by NitpickLawyer »

am17an wrote: post #98
NitpickLawyer wrote: post #96

Earlier quoted context omitted.

> about what I'd expect pre-generative-AI counterexample search systems to be able to find. The difference today (and the reason why everyone is excited about it) is that the same system that does advanced math can write poetry, play an above average game of chess, code frontend/backend stuff and do cybersec. These are not "expert systems", nor are they trained for each task individually. That's the catch. > just ind…

> nor are they trained for each task individually. They are explicitly trained for each task individually.

They are not. Pretraining just dumps every piece of content in the mix, and only has one objective - next token prediction. And you can get pretty good results even with base models, you just have to manage context differently. Later stages (mid, post training) involve RL that "surfaces" the right "traces" out of the pre-training. But they are not trained individually, as we used to do.

Re: Human mathematicians are being outcounterexampled

#102

Post by am17an »

NitpickLawyer wrote: post #101
am17an wrote: post #98

Earlier quoted context omitted.

> nor are they trained for each task individually. They are explicitly trained for each task individually.

They are not. Pretraining just dumps every piece of content in the mix, and only has one objective - next token prediction. And you can get pretty good results even with base models, you just have to manage context differently. Later stages (mid, post training) involve RL that "surfaces" the right "traces" out of the pre-training. But they are not trained individually, as we used to do.

Have you looked at the what data companies (e.g. Scale, Mercor) hire for? Why do you think Meta records their employees every keystroke/mousestroke/eye-movement?

EDIT: just re-read your comment. I don't think you have a good understanding here, no offence.

Re: Human mathematicians are being outcounterexampled

#103

Post by chowells »

ATMLOTTOBEER wrote: post #72
chowells wrote: post #41

Earlier quoted context omitted.

It's elegant if all you're concerned with is whether a conjecture is true or false. Answered, move along! But mathematics is not a collection of facts. Mathematics is the study of abstraction. And what do you learn from a single data point? What can you abstract from that? That's why just being a counterexample isn't really interesting. There has to be more than "counterexample" for there to be something to abstract.…

You’re smuggling in a frame here which isn’t obviously true: that mathematics is not just a collection of facts

You have a weird definition of "smuggling". I say it outright. Because I follow it up with a description of the actual practice of math: it's the study of abstraction. That's not a frame. That's just what math is.

Re: Human mathematicians are being outcounterexampled

#104

Post by NitpickLawyer »

am17an wrote: post #102
NitpickLawyer wrote: post #101

Earlier quoted context omitted.

They are not. Pretraining just dumps every piece of content in the mix, and only has one objective - next token prediction. And you can get pretty good results even with base models, you just have to manage context differently. Later stages (mid, post training) involve RL that "surfaces" the right "traces" out of the pre-training. But they are not trained individually, as we used to do.

Have you looked at the what data companies (e.g. Scale, Mercor) hire for? Why do you think Meta records their employees every keystroke/mousestroke/eye-movement? EDIT: just re-read your comment. I don't think you have a good understanding here, no offence.

None taken, but it would be odd, since I've been training GOFAI models since 2010s and have had LMs in production since before chatgpt came out (before RLHF), so I think I have a pretty good understanding :) But I'm always open to learning.

I think the misunderstanding comes from "individually". You are thinking about diverse datasets, but that's not what individually means in this context. In ML individually trained means that for each task you prepare an architecture, dataset and eval and train that model on that data. And each model has its own objective that you train for. In LMs the objective is singular, for every data point - next token prediction. And, importantly, you train on every datapoint, the more diverse the better, but not independently. The cool thing is that training on diverse datasets improves scores on other downstream tasks, while the training objective is the same.

So you can have a training run on common crawl + programming that improves scores on logic puzzles, or common crawl + novels that improves scores on planning tasks. But the important thing is that it's all trained together, not independently.

Re: Human mathematicians are being outcounterexampled

#105

Post by am17an »

NitpickLawyer wrote: post #104
am17an wrote: post #102

Earlier quoted context omitted.

Have you looked at the what data companies (e.g. Scale, Mercor) hire for? Why do you think Meta records their employees every keystroke/mousestroke/eye-movement? EDIT: just re-read your comment. I don't think you have a good understanding here, no offence.

None taken, but it would be odd, since I've been training GOFAI models since 2010s and have had LMs in production since before chatgpt came out (before RLHF), so I think I have a pretty good understanding :) But I'm always open to learning. I think the misunderstanding comes from "individually". You are thinking about diverse datasets, but that's not what individually means in this context. In ML individually trained…

I think we're saying the same thing? I said they are explicitly trained on these tasks, not that they are some separate models during programming RL, or business tasks RL.

> The cool thing is that training on diverse datasets improves scores on other downstream tasks, while the training objective is the same.

Maybe for some language modelling tasks, i.e. it learns some internal representation that is transferable. However I would find it quite odd if a model becomes good at Bio while not explicitly going through Bio training.

Re: Human mathematicians are being outcounterexampled

#106

Post by anitil »

angry_octet wrote: post #55
tchalla wrote: post #24

Earlier quoted context omitted.

I was once in a presentation for a math PhD thesis. During the thesis, the evaluator of the thesis noticed a flaw in their proof. The student understood and then asked “What now?” The evaluator prof simply shrugged.

A recording of a car crash: discovering on live radio/podcast that the central tenet of your book is wrong, and amateurishly so. Naomi Wolf 'death recorded' on BBC[1], skip to 5:51. After this the book was pulped and she had some sort of psychotic break during COVID and allied with ultra-right and COVID denialist loonies. [1] https://www.bbc.com/news/av/world-us-canada-48639663

I wasn't sure if I would be able to listen again - it's so hard to hear.

Re: Human mathematicians are being outcounterexampled

#107

Post by NitpickLawyer »

am17an wrote: post #105
NitpickLawyer wrote: post #104

Earlier quoted context omitted.

None taken, but it would be odd, since I've been training GOFAI models since 2010s and have had LMs in production since before chatgpt came out (before RLHF), so I think I have a pretty good understanding :) But I'm always open to learning. I think the misunderstanding comes from "individually". You are thinking about diverse datasets, but that's not what individually means in this context. In ML individually trained…

I think we're saying the same thing? I said they are explicitly trained on these tasks, not that they are some separate models during programming RL, or business tasks RL. > The cool thing is that training on diverse datasets improves scores on other downstream tasks, while the training objective is the same. Maybe for some language modelling tasks, i.e. it learns some internal representation that is transferable. Ho…

> I said they are explicitly trained on these tasks, not that they are some separate models during programming RL, or business tasks RL.

You said:

> > nor are they trained for each task individually.

> They are explicitly trained for each task individually.

And that's the main misunderstanding.

Collins says:

individually in American English (ˌɪndəˈvɪdʒuəli, ˌɪndəˈvɪdʒəli) adverb 1. as an individual or individuals rather than as a group; one at a time; separately; singly

Which is precisely what LMs don't do (in contrast to previous "AI" models, which did do that). They are trained on every datapoint at the same time. So long as we agree on that, I think we are saying the same thing :)

Re: Human mathematicians are being outcounterexampled

#110

Post by am17an »

NitpickLawyer wrote: post #107
am17an wrote: post #105

Earlier quoted context omitted.

I think we're saying the same thing? I said they are explicitly trained on these tasks, not that they are some separate models during programming RL, or business tasks RL. > The cool thing is that training on diverse datasets improves scores on other downstream tasks, while the training objective is the same. Maybe for some language modelling tasks, i.e. it learns some internal representation that is transferable. Ho…

> I said they are explicitly trained on these tasks, not that they are some separate models during programming RL, or business tasks RL. You said: > > nor are they trained for each task individually. > They are explicitly trained for each task individually. And that's the main misunderstanding. Collins says: individually in American English (ˌɪndəˈvɪdʒuəli, ˌɪndəˈvɪdʒəli) adverb 1. as an individual or individuals rat…

Your username indeed checks out
Post reply on HN