Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Terence Tao on O1

mathstodon.xyz

Post reply on HN
Print view View original

451–460 of 527 posts

Re: Terence Tao on O1

#451

Post by baq »

ColinWright wrote: post #450
jameshart wrote: post #449

‘Able to make the same creative mathematical leaps as Terence Tao’ seems like a pretty high bar to be setting for AI. This is like when you’re being interviewed for a programming job and the interviewer explains some problem to you that it took their team months to figure out, and then they’re disappointed you can’t whiteboard out the solution they came up with in 40 minutes without access to google.

My experience of working with people like Terence Tao, and being nowhere near their standard, is that they are looking for any kind of creativity. Everything is accepted, and it doesn't have to be "at their level". Having read what he's saying there, and with my experience, I think your characterisation is inaccurate. And having been at the talk he gave for the IMO earlier this year he is impressed with some of the i…

I wonder what the creative spark even is in the context of an autoregressive transformer.

Perhaps it’s an ability to confabulate facts into the context window which are not present in the training data but which are, in the context of maths, viable hypotheses? Every LLM can generate bullshit, but maybe we just need the right bullshit?

Re: Terence Tao on O1

#452

Post by nybsjytm »

jameshart wrote: post #449

‘Able to make the same creative mathematical leaps as Terence Tao’ seems like a pretty high bar to be setting for AI. This is like when you’re being interviewed for a programming job and the interviewer explains some problem to you that it took their team months to figure out, and then they’re disappointed you can’t whiteboard out the solution they came up with in 40 minutes without access to google.

> ‘Able to make the same creative mathematical leaps as Terence Tao’ seems like a pretty high bar to be setting for AI.

There's no need to try to infer this kind of high bar, because what he says is actually very specific and concrete: "Here the result was mildly disappointing ... Essentially the model proposed the same strategy that was already identified in the most recent work on the problem (and which I restated in the blog post), but did not offer any creative variants of that strategy." Crucially the blog post in question was part of his input to ChatGPT.

Otherwise, he's been clear that while he anticipates a future where it is more useful, at present he only uses AI/ChatGPT for bibliography formatting and for writing out simple "Hello World" style code. (He is, after all, a mathematician and not a coder.) I've seen various claims online that he's using ChatGPT all the time to help with his research and, beyond the coding usage, that just seems to not be true.

(However, it's fair to say that "able to help Terence Tao with research" is actually a high bar.)

Re: Terence Tao on O1

#453

Post by drzzhan »

ColinWright wrote: post #450
jameshart wrote: post #449

‘Able to make the same creative mathematical leaps as Terence Tao’ seems like a pretty high bar to be setting for AI. This is like when you’re being interviewed for a programming job and the interviewer explains some problem to you that it took their team months to figure out, and then they’re disappointed you can’t whiteboard out the solution they came up with in 40 minutes without access to google.

My experience of working with people like Terence Tao, and being nowhere near their standard, is that they are looking for any kind of creativity. Everything is accepted, and it doesn't have to be "at their level". Having read what he's saying there, and with my experience, I think your characterisation is inaccurate. And having been at the talk he gave for the IMO earlier this year he is impressed with some of the i…

That's interesting. I am young so I don't know what actually is creativity. Could you explain that part for me?

Re: Terence Tao on O1

#454

Post by AndrewKemendo »

jameshart wrote: post #449

‘Able to make the same creative mathematical leaps as Terence Tao’ seems like a pretty high bar to be setting for AI. This is like when you’re being interviewed for a programming job and the interviewer explains some problem to you that it took their team months to figure out, and then they’re disappointed you can’t whiteboard out the solution they came up with in 40 minutes without access to google.

This is precisely the first thing I thought

If arguably the person with the highest IQ currently living, is impressed but still not fully satisfied that a computer doesn’t give Nobel prize winning mathematical reasoning I think that’s a massive metric itself

So what then should the first year maths PhD think? I believe Tao obliquely addresses this with his previous post with effectively “o1 is almost as good as a grad student”

Re: Terence Tao on O1

#455

Post by kmacdough »

ColinWright wrote: post #450
jameshart wrote: post #449

‘Able to make the same creative mathematical leaps as Terence Tao’ seems like a pretty high bar to be setting for AI. This is like when you’re being interviewed for a programming job and the interviewer explains some problem to you that it took their team months to figure out, and then they’re disappointed you can’t whiteboard out the solution they came up with in 40 minutes without access to google.

My experience of working with people like Terence Tao, and being nowhere near their standard, is that they are looking for any kind of creativity. Everything is accepted, and it doesn't have to be "at their level". Having read what he's saying there, and with my experience, I think your characterisation is inaccurate. And having been at the talk he gave for the IMO earlier this year he is impressed with some of the i…

Right, Terrance was hoping it would have something new to think about it, some new perspective, right or wrong. GPTs have the ability to process insane amounts of information across all branches of math, science and art. This ability eclipses that of the most motivated intellectuals such as Terrance. It is thus a little disappointing that it was unable to find anything in its vast knowledgebase to apply a new lens to the problem.

Re: Terence Tao on O1

#458

Post by faizshah »

jameshart wrote: post #449

‘Able to make the same creative mathematical leaps as Terence Tao’ seems like a pretty high bar to be setting for AI. This is like when you’re being interviewed for a programming job and the interviewer explains some problem to you that it took their team months to figure out, and then they’re disappointed you can’t whiteboard out the solution they came up with in 40 minutes without access to google.

This has been observed by more people than just Terence Tao. Try using chatgpt to program something of higher complexity than tutorial code or write a basic blog post, it lacks creativity and the code is poorly designed.

Re: Terence Tao on O1

#459

Post by perching_aix »

drzzhan wrote: post #453
ColinWright wrote: post #450

Earlier quoted context omitted.

My experience of working with people like Terence Tao, and being nowhere near their standard, is that they are looking for any kind of creativity. Everything is accepted, and it doesn't have to be "at their level". Having read what he's saying there, and with my experience, I think your characterisation is inaccurate. And having been at the talk he gave for the IMO earlier this year he is impressed with some of the i…

That's interesting. I am young so I don't know what actually is creativity. Could you explain that part for me?

A couple days ago I saw a tweet that described how to remove an element from an array in O(1) time instead of O(n). The key to it was identifying that for the purpose the given array was being used for, it could be unordered / not fully ordered, and it would not be an issue.

This way, it was possible to simply replace the element with the last array element, then decrease the size of the array by one. I'd say that's pretty creative: whoever came up with this was able identify what can be traded off to make the previously impossible, possible, unlocking new scales and possibilities.

In practice, I'd say creativity is often being able to manifest people's qualia in some unprecedented way. For example, say you're experimenting in your DAW, and discover a pretty cool sound. You identify the ways it can be used to emote and then utilize it in a work. If you really stumbled upon a sound that a lot of people find as emotive as you did, you just did something creative: it's as if you translated the qualia of an emotion into sound.

This qualia to manifestation is what's behind creativity in all of senses of the word I believe. In my previous example, discovering that orderedness is not actually a strict requirement, and (ab)using that to significantly alter the scaling of such an action is creative, because it undoes the notion that orderedness is a requirement. It goes against what's natural, but in a way that becomes extremely natural and indispensable once realized.

I think, in that way, current AIs are trained to be uncreative, since being creative inherently requires experimentation that is unaligned with the normal.

Re: Terence Tao on O1

#460

Post by ilrwbwrkhv »

faizshah wrote: post #458
jameshart wrote: post #449

‘Able to make the same creative mathematical leaps as Terence Tao’ seems like a pretty high bar to be setting for AI. This is like when you’re being interviewed for a programming job and the interviewer explains some problem to you that it took their team months to figure out, and then they’re disappointed you can’t whiteboard out the solution they came up with in 40 minutes without access to google.

This has been observed by more people than just Terence Tao. Try using chatgpt to program something of higher complexity than tutorial code or write a basic blog post, it lacks creativity and the code is poorly designed.

Even for like basic Rust programs it ties itself into countless borrow checker issues and cannot get out of it, both the OpenAI as well as Sonnet (Anthropic).

It doesn't really get logic still, but it does small edits well when the code is very clear.

I think this will always remain a problem. Because it can never shut up, it keeps making stuff up and "hallucinate" (works normally, just incorrectly) to dig itself further and further into a hole.

Autocomplete on steroids is what peak AI will look like till the time we can crack consciousness and AGI (which the modern versions are nothing even close to).

Post reply on HN