I was not refuted sufficiently a couple of years ago.
I claimed "training is open boundary" etc.
Can AI do maths yet? Thoughts from a mathematician
211–220 of 364 posts
Re: Can AI do maths yet? Thoughts from a mathematician
#212Earlier quoted context omitted.
If you looked at how the average accountant spent their time before the arrival of the digital spreadsheet, you might have predicted that automated calculation would make the profession obsolete. But it didn't. This time could be different, of course. But I'll need a lot more evidence before I start telling people to base their major life decisions on projected technological change. That's before we even consider tha…
It's important to base life decisions on very real technological change. We don't know what the change will be, but it's coming. At the very least, that suggests more diverse skills. We're all usually (but not always) better off, with more productivity, eventually, but in the meantime, jobs do disappear. Robotics did not fully displace machinists and factory workers, but single-skilled people in Detroit did not do we…
If AI reaches this level socioeconomic impact is going to be so immense, that choosing what subject you study will have no impact on your outcome - no matter what it is - so it's a pointless consideration.
Re: Can AI do maths yet? Thoughts from a mathematician
#213I was not refuted sufficiently a couple of years ago. I claimed "training is open boundary" etc.
Like as a few years ago, I just boringly add again "you need modeling" to close it.
Re: Can AI do maths yet? Thoughts from a mathematician
#214Earlier quoted context omitted.
The only thing I've consistently had issues with while using AI is graphs. If I ask it to put some simple function, it produces a really weird image that has nothing to do with the graph I want. It will be a weird swirl of lines and words, and it never corrects itself no matter what I say to it. Has anyone had any luck with this? It seems like the only thing that it just can't do.
You're doing it wrong. It can't produce proper graphs with it's diffusion style image generation. Ask it to produce graphs with python and matplotlib. That will work.
And works very well - it made me a nice general "draw successively accurate Fourier series approximations given this lambda for coefficients and this lambda for the constant term". PNG output, no real programming errors (I wouldn't remember if it had some stupid error, I'm a python programmer). Even TikZ in LaTeX isn't hopeless (although I did ending up reading the tikz manual)
Re: Can AI do maths yet? Thoughts from a mathematician
#215Earlier quoted context omitted.
You know, "You're using it wrong" is usually meant to carry an ironic or sarcastic tone, right? It dates back to Steve Jobs blaming an iPhone 4 user for "holding it wrong" rather than acknowledging a flawed antenna design that was causing dropped calls. The closest Apple ever came to admitting that it was their problem was when they subsequently ran an employment ad to hire a new antenna engineering lead. Maybe it's…
It's not an LLM. You're simply asking too much of it. It doesn't work the way you want it to, sorry.
Tell Wolfram. They're the ones who've been advertising it for years, well before LLMs were a thing, using English-language prompts like these examples: https://www.pcmag.com/news/23-cool-non-math-things-you-can-d...
The problem has always been that you only get good answers if you happen to stumble on a specific question that it can handle. Combining Alpha with an LLM could actually be pretty awesome, but I'm sure it's easier said than done.
Re: Can AI do maths yet? Thoughts from a mathematician
#216In other news we’ve discovered life (our bacteria) on mars
Just joking
Re: Can AI do maths yet? Thoughts from a mathematician
#217Earlier quoted context omitted.
The reddit thread is ... interesting (direct link[1]). It seems to be a debate among mathematicians some of whom do have access to the secret set. But they're debating publicly and so naturally avoiding any concrete examples that would give the set away so wind-up with fuzzy-fiddly language for the qualities of the problem tiers. The "reality" of keeping this stuff secret 'cause someone would train on it is itself bi…
It's not about training directly on the test set, it's about people discussing questions in the test set online (e.g., in forums), and then this data is swept up into the training set. That's what makes test set contamination so difficult to avoid.
>> It's not about training directly on the test set, it's about people discussing questions in the test set online
Don't kid yourself. There are 10's of billions of dollars going into AI. Some of the humans involved would happily cheat on comparative tests to boost investment.
Re: Can AI do maths yet? Thoughts from a mathematician
#218I just spent a few days trying to figure out some linear algebra with the help of ChatGPT. It's very useful for finding conceptual information from literature (which for a not-professional-mathematician at least can be really hard to find and decipher). But in the actual math it constantly makes very silly errors. E.g. indexing a vector beyond its dimension, trying to do matrix decomposition for scalars and insisting…
It reliably fails also basic real analysis proofs, but I think this is not too surprising since those require a mix of logic and computation that is likely hard to just infer from statistical likelihood of tokens
Re: Can AI do maths yet? Thoughts from a mathematician
#219Earlier quoted context omitted.
It's not an LLM. You're simply asking too much of it. It doesn't work the way you want it to, sorry.
Tell Wolfram. They're the ones who've been advertising it for years, well before LLMs were a thing, using English-language prompts like these examples: https://www.pcmag.com/news/23-cool-non-math-things-you-can-d... The problem has always been that you only get good answers if you happen to stumble on a specific question that it can handle. Combining Alpha with an LLM could actually be pretty awesome, but I'm sure it…
Before LLMs exploded nobody really expected WA to perform well at natural language comprehension. The expectations were at the level of "an ELIZA that knows math".
Re: Can AI do maths yet? Thoughts from a mathematician
#220Earlier quoted context omitted.
Wolfram Alpha is better at actually doing math, but far worse at explaining what it’s doing, and why.
I wish there was a way to tell Chatgpt where it has made a mistake, with a single mouse click.
What's surprising to me is that this would surely be in OpenAI's interests, too -- free RLHF!
Of course there would be the risk of adversaries giving bogus feedback, but my gut says it's relatively straightforward to filter out most of this muck.