> For years, AI researchers have used math as a proving ground for their models.
For years?
Mathematicians issue warning as AI rapidly gains ground
241–250 of 366 posts
Re: Mathematicians issue warning as AI rapidly gains ground
#242Another mathematician already predicted this, but you didn't listen. His name was Theodore Kaczynski. It's time to reap what you've sown.
The complete historical illiterate? He didn't even understand the hunter-gatherers he so idolized.
Re: Mathematicians issue warning as AI rapidly gains ground
#243Earlier quoted context omitted.
Sounds like yet another example of how AI is kneecapping industries from the bottom by "removing the barrier to entry" but really just removing the training path by doing the work itself with no guidance for juniors.
We are on tiny 1-5T parameter models with local power stations. We can reach Q models just by throwing resources at it. That’s a million times current B models.
what is a B model vs. a Q model? what do these letters mean?
Re: Mathematicians issue warning as AI rapidly gains ground
#244Earlier quoted context omitted.
We are on tiny 1-5T parameter models with local power stations. We can reach Q models just by throwing resources at it. That’s a million times current B models.
what is a B model vs. a Q model? what do these letters mean?
B Billion parameter, T trillion, Q Quadrillion.
Re: Mathematicians issue warning as AI rapidly gains ground
#245Anyone else draw similarities with this and the artists and authors who complained when gen ai first came out. I think a lot of people don't realise the disruption ai will cause to many industries, until its directly impacting them, basically personal fable at scale ( https://en.wikipedia.org/wiki/Personal_fable ).
The artists will be fine and AI will liberate them. It's the engineers and mathematicians who are walking into the blade. They built their entire sense of self on being the best optimizers in the room and optimization is the first thing the machines take. Their whole identity was a number going up. Now watch it go to zero.
I cant speak for engineers, but as a mathematician I wholeheartedly disagree with everything you claim in your comment. Almost none of the mathematicians that I know care about the optimization aspect of mathematics: the pursuit of optemizing constants in theorems and providing minor technical improvements is mostly seen as pointless unless there are significant new mathematical insights that fuel the improvement. I think most mathematicians rather build their identity around providing actual understanding of problems using mathematics and improving society's understanding of mathematical problems.
Of course AI threatens this too, but the threat is of a much lesser degree. One could even argue that AI is helpful here with getting mathematicians to the 'frontier of knowledge' as AI is usually good in combining ideas from different fields.
Re: Mathematicians issue warning as AI rapidly gains ground
#246Earlier quoted context omitted.
Instead of going into a long technical argument of why your description of LLMs is flawed, I'll go straight to the point, because people keep moving the goal posts. What exact problem would need to be solved by LLMs to convince you that they DO discover novel solutions?
I'm more interested why you think my understanding is flawed honestly. I thought I distilled it decently well in two sentences. The bottom line is, in this hyperdimensional space you can find relationships that are not easily distinguished by human minds, but the corpus is still fixed, a llm can't truly know anything beyond its training data.
> Think of it as 1000+ ways to think of that word/token
I assume you used 1000 because that's in the ballpark of the vector size. But these are not independent scalars, like each might store a certain property. Just like in 2D you can have 4 quadrants (or subdivide further), with a vector of size 1000 you can encode an insane amount of meaning.
> Those meanings are baked in at training time. So, LLMs might be able to cross-reference them and solve a class of problems that flew under our radar, but can't come up with revolutionary theories that were never in the training set.
There's a lot of jumping to conclusions here, but I'll try to answer more generally.
This idea of how LLMs work is mostly to build an intuition, like with a CNN you'd say imagine a layer does edge detection, and so on. And to some degree you can detect those kinds of behavior, but a NN is a VERY general architecture. It needn't work like you say, it can calculate any function and running under a loop and a scratchpad (basically an agent) is turing complete.
Even ignoring that, this part is misleading
> Those meanings are baked in at training time.
Being baked in at training time does not mean it didn't build novel meanings at training time.
This is even more significant when you take into account post training RL.
A simple proof that transformers can generate novel, superhuman solutions, is that you can build a transformer based chess bot, feed it 0 human games, and train it with RL until it can beat any human, completely novel and unconstrained by human gameplay (because it would've never seen it).
You can do that with any task that's verifiable, like coding or math.
(Also as a separate fact, as long as a task is easier to verify than solve (basically always), you have somewhat of a million monkeys with a typewriter, and with temperature sampling the model might eventually stumble it's way onto a solution.)
Re: Mathematicians issue warning as AI rapidly gains ground
#247Earlier quoted context omitted.
We are on tiny 1-5T parameter models with local power stations. We can reach Q models just by throwing resources at it. That’s a million times current B models.
Is this a known or quantifiable thing? I thought that the limit had already been determined i.e. the existing models top out and at some point it doesn't matter how much time or energy you let the model consume, it won't improve the result. And with regards to training parameters, I thought we were equally limited there, e.g. the existing models can't benefit from a larger parameter space. I was under the impression…
The existing models "top out" not because they don't get better, but because it is uneconomical.
What we do know is that a model "tops out" wrt training data - that is, for a model of a given size, there's only so much training data you can squeeze into the set before you stop seeing gains. But conversely it means that if you already have a model of say 1 Ttok that is "trained to capacity", then a model of 2 TTok needs roughly twice as much training data to fully utilize all those weights. Which means that the cost of training it is not 2x but 4x (twice as many params x twice as many tokens). And then of course serving it is 2x more expensive, but even with optimal training the gains aren't 2x. So it very quickly becomes uneconomical.
A good example of that kind of model is (was) GPT-4.5. The prices and the consequent lack of demand show why companies don't really do that sort of thing anymore.
But no, there's no evidence of a plateau as such. I'm not sure what "evidence that we have not reached a plateau" would even look like.
Re: Mathematicians issue warning as AI rapidly gains ground
#248Anyone else draw similarities with this and the artists and authors who complained when gen ai first came out. I think a lot of people don't realise the disruption ai will cause to many industries, until its directly impacting them, basically personal fable at scale ( https://en.wikipedia.org/wiki/Personal_fable ).
I’m curious, do writers and authors still really care about AI? I think by now most people are completely put off by AI slop, the value of AI writing or image generation is basically zero So I suspect that the cloud will pass on math too, initial demos get extrapolated and people get worried but in the end slop is slop and serious people aren’t getting replaced or even threatened.
There have been numerous studies by now showing that most people cannot reliably distinguish "slop" from the real thing, and that many genuinely prefer the slop even.
Re: Mathematicians issue warning as AI rapidly gains ground
#249Much of math (or science) research has the strange quality of being mostly curiosity-driven, but having giant benefits that occasionally spin out to the public. Some questions are more urgent and practical. My feeling is that the more directly practical a question is, the more likely the research community is to support AI usage in that question. The annoying thing about recent AI advances is that they target questio…
I think nuance gets lost in these conversations. Your distinction between the practical and the theoretical is important. Practicality is important - everything we do is a matter of practicality of means or method , even how we pursue theoretical ends - but two points. First, there is more to life than the practical. Some truths are known for their own sake, even if they also tell us about still more profound truths…
I'm not sure why you're being downvoted, but I agree with all of your points. Those aren't things that pretty lend themselves well to mathematical modelling. But... there is a marginal field of math that does apply to this: statistics. The first two cases are somewhat special: - It may be daily obvious that an API is terrible, and that the replacement is not. If API 1 takes 1 sec to call, and API 2 takes 100ms to call, straightforward choice without stats. - provisioning can be dangerous. While not really a stats problem, you do need to have a quite elegant model of what is getting refactored, and how to know when to invalidate those cache entries. For the rest of the examples you provided, you're making changes that may make the problem better, may have no effect, or may make the problem worse. You completely need to use statistics to determine whether or not changes like those are honestly having an effect. Performance analysis is part math and part art, and without the math background, you're likely going to be spinning your wheels a bunch. Beyond stats, fields like queuing theory are going to make a massive breakthrough when you're doing performance breakthrough in distributed systems.
Re: Mathematicians issue warning as AI rapidly gains ground
#250Earlier quoted context omitted.
> AI (in this form) will never be able to solve things we truly cannot solve yet. Argument?
The strongest argument for this is structural: what LLMs are. In a brutal simplistic way: each token is represented in a high dimensional vector. LLMs operate on them. They are the true, underlying meaning of the token for the LLM. Think of it as 1000+ ways to think of that word/token. Those meanings are baked in at training time. So, LLMs might be able to cross-reference them and solve a class of problems that flew…
I don't see how any of this follow. Yes, the LLMs will learn the "meaning" (here narrowly defined as relative configuration in the embedding space) of vectors that correspond to tokens in whatever tokenizer is used to feed into them. But that vector space is not discrete, and nothing precludes the model from internally operating on other vectors that it never saw in training, based on how they relate to those vectors which it did see.