Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An OpenAI model has disproved a central conjecture in discrete geometry

openai.com

Post reply on HN
Print view View original

831–840 of 1001 posts

Re: An OpenAI model has disproved a central conjecture in discrete geometry

#831

Post by _zoltan_ »

esikich wrote: post #567
dawnerd wrote: post #517

Earlier quoted context omitted.

Personally I found 5.5 a massive step back from 5.4. Both of them still use way too many fallbacks and unnecessary checks, especially if you're having it output php. It's fine if you're just one person and checking everything and able to catch and correct. But it's really bad when you have a team all using it, not checking the output and trusting it's output leading to spaghetti code. Technically works, but very mess…

Idk man, I think at this point, if you can't get good code out of frontier models, you're doing something wrong. Plenty of resources out there for you to familiarize yourself with the workflows if you can be bothered.

100%. this is what I tell people who fail at using these models. the model isn't the problem anymore.

Re: An OpenAI model has disproved a central conjecture in discrete geometry

#832

Post by _zoltan_ »

bigstrat2003 wrote: post #600
Supermancho wrote: post #572

Earlier quoted context omitted.

>That's literally my job... Since you’re not in a unique position, I can confidently state that your comparison of LLMs to jr developers seems unfounded. Today, LLMs produce code that is superior to junior developer code by an order of magnitude. Notably, they demonstrate consistent syntax, clear separation of concerns, strong test coverage, organizational rigor, idiomatic API usage, and the ability to generate and m…

LLMs absolutely do not exceed the abilities of junior devs. They don't even meet that bar, let alone exceed it. Junior devs are capable of getting syntax right without someone going "hey you messed that up". LLMs are not. Junior devs get basic logic right. LLMs do not. Comparing an LLM to a senior developer is an absolute joke.

they 100% exceed even medior developers.

who cares about syntax? who cares about iteration? what I care about are _results_, which they can produce at the end. do you check your human colleagues how many iterations they do before committing/showing their work to anybody? no. why should you set such a bar then for your LLM?

Re: An OpenAI model has disproved a central conjecture in discrete geometry

#833

Post by cma »

turzmo wrote: post #785
raincole wrote: post #280

I like how everyone laughed when OpenAI said their models will have "PhD-Level Intelligence" and now the goalpost has been moved to if AI can create new math (i.e., not PhD-Level, but Leibniz/Euler/Galois level.)

Not denying that these advances are impressive, but it is important to consider that this is a cherry-picked result. This doesn’t mean that AI can now be expected to do problems of similar or lower difficulty, but that it happened to work well on one problem. What you won’t see is how many others they had to try to get this result.

Earlier of their systems have solve other Erdos problems that people had worked on, this one was more monumental and had had a lot more prior effort that didn't solve, but this isn't a one-off.

Re: An OpenAI model has disproved a central conjecture in discrete geometry

#835

Post by isolli »

Question:

The conjecture was about an upper bound for the maximum number of pairs. It has been disproven.

Was the Erdos problem the conjecture itself, or was it about the actual maximum number of pairs? (In which case it will probably never be solved.)

The problem is defined in the narrow version here: https://www.erdosproblems.com/90

Re: An OpenAI model has disproved a central conjecture in discrete geometry

#837

Post by iLoveOncall »

varenc wrote: post #425
iLoveOncall wrote: post #347

Absolutely no proof that any LLM actually found the result, and just a mention of an "internal model". Served to you by one of the biggest liars in the world. Why would anyone believe this to be true even for a split second?

This has been an unsolved open problem for 80 years. What you're suggesting is that someone connected to Open AI solved this very hard math problem, but then rather than taking credit for it, falsely attributed it to AI? The point of having an AI solve an unsolved problem, is to make it very clear that the insight must have come from the AI and wasn't in the training data. Sure, it's possible OpenAI had access to som…

> What you're suggesting is that someone connected to Open AI solved this very hard math problem, but then rather than taking credit for it, falsely attributed it to AI?

I'm suggesting that OpenAI invested a lot of resources and money in having someone (or a group of people) disprove this conjecture, so they could claim their LLM disproved it. Yes.

I'm not sure why you're surprised by this, given that everything that Altman has said in the past has turned out to be a lie.

The fact that they gave an EDITED (even rewritten, from the PDF itself) chain of thought is just further proof. Why not give the raw one alongside? No reason at all, except if it doesn't exist.

Re: An OpenAI model has disproved a central conjecture in discrete geometry

#838

Post by killerstorm »

davebren wrote: post #398
mooreat wrote: post #366

Earlier quoted context omitted.

I currently operate under the assumption that humans are at most as powerful as Turing Machines. And from what I understand these models internally are modeling increasingly harder and larger DFAs, so they're at least as powerful as regular languages. Assuming humans are more powerful than regular languages I could maybe agree that these methods may not eventually yield entirely human like intelligence, but just bett…

Well yeah there is likely an equivalence between computability and epistemology, but I'm not sure it matters when comparing LLM intelligence to human intelligence. There is clearly a missing link that prevents the LLM from reaching beyond its training data the way humans do.

The act of discovery is usually associated with "abductive reasoning", i.e. finding a novel pattern in data.

Usually people point out that humans are more sample efficient: they might notice a novel pattern in a handful of samples, whereas training NN might require take millions.

However a claim that LLMs fundamentally cannot do abductive reasoning at all is not warranted - we don't see a clear cut, it just looks like the way LLMs do it is less efficient.

Re: An OpenAI model has disproved a central conjecture in discrete geometry

#839

Post by bayindirh »

biztos wrote: post #731
teiferer wrote: post #727

Earlier quoted context omitted.

> Every day, it grows harder and harder to contain a mental map of recent relevant progress by simple virtue of the amount being produced. And by opening the door to LLM-generated results, you'll see greater and greater amounts without any hope of ever navigating this field again without machine help. It's a little like a software project which more and more gets extended by a AI agents with less and less review by h…

I'm no mathematician but it seems like if this happens, we get to a quite intriguing place as a species. Say we achieve interstellar travel, but nobody actually knows how it works. Or we cure cancer, but the "cure" requires a microrobotic implant, and it runs as a blackbox AI, and only the other AIs can make one, and there's no guarantee they will know how to make one tomorrow. Or we solve global warming but it requi…

Welcome to the "Hyperion Cantos".

The book doesn't deviate from what you have envision, or the future you envision doesn't deviate from the book, I may say.

Re: An OpenAI model has disproved a central conjecture in discrete geometry

#840

Post by vessenes »

Quentak wrote: post #813

I'd like to know how many tokens in total went into solving this problem. Have they talked about this? It matters whether they got this result in 10 million tokens or 10 billion. Whether it's closer to 1 human working on this for 1 year or 1000 humans for 1 year. The news feels different when the probability of one AI run solving this is 1 in a thousand vs 1 a million. Approximately I'm asking about the amount of mon…

Good q. You can see that tantalizing graph where compute is displayed on a log axis and pass@1 goes up to like 50/60%. I’d love to know just how much compute is encoded on that axis.

I guess you can get some estimate from the excerpted CoT, but that CoT might be backed by quite a lot of parallel compute.

Post reply on HN