Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ten advances in mathematics and theoretical computer science

openai.com

Post reply on HN
Print view View original

691–700 of 1001 posts

Re: Ten advances in mathematics and theoretical computer science

#691

Post by alasano »

J_Shelby_J wrote: post #651
dominotw wrote: post #562

Earlier quoted context omitted.

> but I’ve noticed Fable to be quite a big step up there what did you notice ?

I've noticed that out of all LLMs I've ever used that Fable is the MOST LLM; the text it produces is abomination. It's impressive how much I hate it. It is such an awful writer - it assumes the reader has zero context and therefore gives every single bit of context and detail - which is nice if you're writing a legal document I suppose. But it uses, niche, $10 words to describe every facet of everything it's discussi…

Just praised Fable in another comment but what you're saying is also insanely true.

I literally roll my eyes and cringe quite often at its output pretty much daily.

I don't like to overload my sessions with skills but I've been using a "write-normal" skill I made just to have it rewrite outputs that particularly piss me off.

https://gist.github.com/alasano/1c734fa055231a5defcfd213217e...

I'm sure there's a million of these skills out there, but this one is tailored to the stuff that makes me mad in particular.

Re: Ten advances in mathematics and theoretical computer science

#692

Post by annzabelle »

throwaway27448 wrote: post #675
bryan0 wrote: post #662

Earlier quoted context omitted.

this gets more nuanced because "the sigmoids won't save you": https://www.astralcodexten.com/p/the-sigmoids-wont-save-you

If the sigmoid is incorrect it's certainly more correct than the exponential. > https://www.astralcodexten.com/p/the-sigmoids-wont-save-you The conclusion of this article seems to be "you should give ai the benefit of the doubt against all reason". Barf

The author of that post is a prominent Bay Area "rationalist," who have had a quasi-theistic relationship with the concept of all-powerful AIs for a couple decades now.

Re: Ten advances in mathematics and theoretical computer science

#693

Post by throwaway27448 »

conformist wrote: post #683
throwaway27448 wrote: post #675

Earlier quoted context omitted.

If the sigmoid is incorrect it's certainly more correct than the exponential. > https://www.astralcodexten.com/p/the-sigmoids-wont-save-you The conclusion of this article seems to be "you should give ai the benefit of the doubt against all reason". Barf

Isn’t the point more “it’s easy to fall into the trap to believe that predicting when the sigmoid is going to bend is possible and the right heuristic is to instead extrapolate locally”? That aside, I’d question whether applying the Lindy effect in particular to something that’s not really a life expectancy but more a growth rate is credible… or perhaps a bit circular since it “assumes away” the ceiling.

Nobody in this thread is trying to predict when the sigmoid is going to bend. Perhaps they should

Re: Ten advances in mathematics and theoretical computer science

#694

Post by subygan »

throwaway27448 wrote: post #634
sothatsit wrote: post #535

People argue whether we are at y-5, y, or y+5, meanwhile we seem to be on a y=2^x exponential that keeps delivering more and more impressive results. The most interesting question to me is what will be consumed by the exponential like math seems to be undergoing, and what won’t. Writing has been quite stubborn, but I’ve noticed Fable to be quite a big step up there. How about politics? Will we develop new ways to let…

Sigmoidal, not exponential. It would be insane to assume an exponential curve

in the absence of a stalling signal. it's better to assume exponential and work backwards than hope the next bottleneck is impossible.

Re: Ten advances in mathematics and theoretical computer science

#695

Post by gpm »

curt15 wrote: post #688
sothatsit wrote: post #535

People argue whether we are at y-5, y, or y+5, meanwhile we seem to be on a y=2^x exponential that keeps delivering more and more impressive results. The most interesting question to me is what will be consumed by the exponential like math seems to be undergoing, and what won’t. Writing has been quite stubborn, but I’ve noticed Fable to be quite a big step up there. How about politics? Will we develop new ways to let…

Another interesting question is why the frontier labs are piling on pure maths, which has little direct economic value compared to something like law or improving the efficiency of their own models? How much OpenAI and Anthropic are paying to serve these models for ordinary users is the elephant in the room. A cynical take is that the frontier labs are trying their best to pump up their pre-IPO valuation through flas…

Because it's a tool in search of a use case (or many use cases) and mathematics is the most natural use case for it. Mathematics is by definition the art of putting words on a page in a rigorously defined "correct manner" (i.e. in the form of a valid logical argument, a proof) and all LLMs do is put words on pages and evaluating if they're good words is by far easiest when there is a strict definition of right and wrong.

Re: Ten advances in mathematics and theoretical computer science

#696

Post by danielmarkbruce »

curt15 wrote: post #688
sothatsit wrote: post #535

People argue whether we are at y-5, y, or y+5, meanwhile we seem to be on a y=2^x exponential that keeps delivering more and more impressive results. The most interesting question to me is what will be consumed by the exponential like math seems to be undergoing, and what won’t. Writing has been quite stubborn, but I’ve noticed Fable to be quite a big step up there. How about politics? Will we develop new ways to let…

Another interesting question is why the frontier labs are piling on pure maths, which has little direct economic value compared to something like law or improving the efficiency of their own models? How much OpenAI and Anthropic are paying to serve these models for ordinary users is the elephant in the room. A cynical take is that the frontier labs are trying their best to pump up their pre-IPO valuation through flas…

It's one of the few areas where you can verify results. That fits nicely into training models. They aren't just making judgement calls on what would be nice, it's "what can we do?".

Re: Ten advances in mathematics and theoretical computer science

#697

Post by VladVladikoff »

plaidfuji wrote: post #674

Any computable problem will eventually fall to computers. LLMs have made math proofs more computable, in the sense that a computer can both generate potential solutions and check the validity of its solutions on its own, with a reasonable chance of converging on something correct. I assume this was already doable to some extent, but it seems like it’s now exponentially easier. That still doesn’t mean that all math is…

Would be great to see them solve Yang-Mills and Mass Gap.

Re: Ten advances in mathematics and theoretical computer science

#698

Post by plif »

tyre wrote: post #548
sothatsit wrote: post #535

People argue whether we are at y-5, y, or y+5, meanwhile we seem to be on a y=2^x exponential that keeps delivering more and more impressive results. The most interesting question to me is what will be consumed by the exponential like math seems to be undergoing, and what won’t. Writing has been quite stubborn, but I’ve noticed Fable to be quite a big step up there. How about politics? Will we develop new ways to let…

We will get much better at manipulation and better at people “writing” things to justify their own feelings. What’s new about LLMs is that you can scalably manipulate people individually. It used to be that you could either have scale (speeches, tweets, interviews, website, etc.) or individual engagement (replying to mail/tweets/town hall questions.) Now you can pull the history and preferences of an individual, then…

Not new about LLMs. Targeted ads / big data is this.

Another degree of capability, yes. But we have been trending here for a long time.

Re: Ten advances in mathematics and theoretical computer science

#699

Post by metanonsense »

5555watch wrote: post #501
evenhash wrote: post #408

Earlier quoted context omitted.

> Whilst current models can't 'intuit' and come up with conjectures People keep saying this. Why? Surely the AI can complete the prompt “Generate new research questions based on these observations”? When I read the reasoning traces of coding models they are constantly asking themselves questions and attempting to answer them.

I like the illustration that the models are working on a convex hull of known information. Filling gaps with linear combinations of known facts and results. They can't exit the hull until the "intuition" starts spawning points outside the convex hull.

I think this is only "statistically" true in the sense that training is based on facts and not non-facts (except maybe with the ingestion of flat-earthers literature ;-). The existence of hallucinations in a bare transformer shows that the convex hull is not about information but about text, so the limit may more be "possible linear combinations of text", which allows for much extrapolation and counterfactuals. True creativity may be one reinforcement learning mid-training goal away that rewards novelty over correctness.
Post reply on HN