People struggle to get math, too.
AI language models are struggling to “get” math
81–90 of 201 posts
Re: AI language models are struggling to “get” math
#82That's because they're not modelling anything. The shocking thing about current AI models is that just sort of repeating and copying from memory what you've heard and seen gets you 97% of the way to imitating a person.* They still need to generate actual models somewhere to create consistency; so many generated images with one eye completely different from the other, or three arms, or fingers that grow into their cel…
This is factually wrong, both in terms of quantity and quality.
Current AI models are not "just sort of repeating and copying from memory". This is just an incorrect characterization of how they work and how they perform.
AI skeptics often say things like this then backpedal with something like "Well they aren't really repeating what they heard, but their generative model is just a slightly more sophisticated version of repeating what they've heard." But this weaker claim is also true of humans. It's certainly the case that >97% percent of what humans say is "just repeating and copying" in the same sense.
Re: AI language models are struggling to “get” math
#83Are there any general purpose models that are good at learning math? I mainly know basic feed-forward neural nets, but I don't think they do well outside their training region. Math, of course, has an infinite training region.
From my (limited) experience with the advanced ML models, they can "do basic math", but they make amateur mistakes with basic things - which indicates they don't actually know addition , but they are good at looking at patterns in existing language. I would assume that state-of-the-art ML models could "convert a word problem into an equation", then feed that equation into a 30 year-old graphing calculator to "do the…
> "convert a word problem into an equation"
Isn't this a huge step? It's not a minor detail remaining to be solved, but possibly the largest step!
Re: AI language models are struggling to “get” math
#84How much of this is just "AI is bad at everything", but in the math case, it's easier for the lay person to tell . It's all just passable garbled nonesense that the reader (goes to lengths) to interept based on their prior knowledge, which is not expressed in the syntax of what these systems output. In the case of mathematics, we're far less willing to "BS away" the interpretive failures. But if we were equally deman…
Have you ever used Github CoPilot? It does a lot of useful work, automating away rote typing in programming. Have you tried Dall-E or Stable Diffusion? They make good looking images. This comment seems completely unmoored from where the state of the art is right now.
Re: AI language models are struggling to “get” math
#85Earlier quoted context omitted.
Musicians don’t spit out an album in one sitting and they’re highly trained in theory. They get bored and tired of a process and take breaks. They come up with an album of loops composed together over time. AIs state will forever be constrained to the limits of human cognition and behavior as that’s what it’s trained on. I read published research all year. Circular reasoning. Tautology. It’s all over PhD thesis. Ther…
I think it's deeply depressing that AI has been sold as something even capable of modelling anything humans do; and quite depressing that this comment exists. "AI" is just taking `mean()` over our choice of encodings of our choice of measurements of our selection of things we've created. There is as much "alike humans" in patterns in tree bark. AI is an embarrassingly dumb procedure, incapable of the most basic homol…
I lack your confidence, and find it a bit religious.
> The idea that `mean(424,34324,223123,3424,....)` even has any revelance to us at all is quite absurd.
Most of what I say to anyone is exactly this.
When I'm about to give anyone any information, I look back at all of the relevant past information that I can recall (through word and sensory association, not by logic, unless I have a recollection of an associated internal or external dialog that also used logical rules.) I multiply those by strength of recollection and similarity of situation (e.g. can I create a metaphor for the current situation from the recalled one?). I take the mean, then I share it, along with caveats about the aforementioned strength of recollection and similarity of situation.
This is what it feels like I actually do. Any of these steps can be either taken consciously or by reflex. It's not hidden.
> I think it's deeply depressing that AI has been sold as something even capable of modelling anything humans do
This is a bizarre position. All computers ever do is model things that humans do. All a computer consists of is a receptacle for placing human will that will continue to apply that will after the human is removed. They are a way of crystallizing will in a way that you can sustain it with things (like electricity) other than the particular combination of air, water, food, space, pressure, temperature, etc. that is a person. An overflow drain is a computer that models the human will. An automatic switch/regulator is the basic electrical model of human will, and a computer is just a bunch of those stitched together in a complementary way.
Re: AI language models are struggling to “get” math
#86The article is actually about how they are getting good at it :)
Re: AI language models are struggling to “get” math
#87I genuinely wonder if we will find there are some inherent tradeoffs to knowledge and understanding such that if we ever have machines that can “think like humans” they would in practice run into human-like cognition limits: ie such machines would be “bad at math” in the same way humans are “bat at math” compared to conventional computers.
Even if neural networks were fundamentally incompatible with conventional computation, I don't see why you couldn't augment a neural network with a conventional ALU to do the numerical computations. This is exactly what humans do with pencil and paper - it's just a bit too slow.
Either the language model would need to know what it's doing or the host program would have to know what the AI is doing. Both seem out of reach. The latter seems more doable since you could hack something up for simple scenarios, but you'd effectively have to match the capabilities of the neural network in a classical way to handle every case (which would render using a neural net moot).
Re: AI language models are struggling to “get” math
#88Earlier quoted context omitted.
AI is bad at music also. Even the state of the art transformer models can't produce more than a few seconds of coherent melodic phrases.
AI is bad at Audio. AI can do MIDI fine.
MIDI is extraordinarily expressive and is likely used to sequence a large majority of music produced within the last three decades. A lot of the instruments you hear are synthesizers or samplers running directly from MIDI. There is a lot more to what MIDI can do, and is used for, than the conception most people have from "canyon.mid" or old website background music. If an AI can do MIDI just fine then it's an extremely small leap to doing audio just fine.
Re: AI language models are struggling to “get” math
#89That's because they're not modelling anything. The shocking thing about current AI models is that just sort of repeating and copying from memory what you've heard and seen gets you 97% of the way to imitating a person.* They still need to generate actual models somewhere to create consistency; so many generated images with one eye completely different from the other, or three arms, or fingers that grow into their cel…
This is factually wrong, both in terms of quantity and quality. Current AI models are not "just sort of repeating and copying from memory". This is just an incorrect characterization of how they work and how they perform. AI skeptics often say things like this then backpedal with something like "Well they aren't really repeating what they heard, but their generative model is just a slightly more sophisticated version…
> Current AI models are not "just sort of repeating and copying from memory". This is just an incorrect characterization of how they work and how they perform.
You say this, but don't explain how. Because this is exactly what they are doing.
> AI skeptics often say things like this
I'm not really an AI skeptic. I think that we're very close to AI being indistinguishable from people. There are clearly problems that need to be solved, but I think the hardest problem was accepting the fact that humans are largely just copying and realizing that would be enough to get you 97% of the way there, especially if you gave a machine far more to copy than a human could consume.
> then backpedal with something like "Well they aren't really repeating what they heard, but their generative model is just a slightly more sophisticated version of repeating what they've heard." But this weaker claim is also true of humans. It's certainly the case that >97% percent of what humans say is "just repeating and copying" in the same sense.
Maybe I'm not expressing myself clearly, but it seems that you're just repeating my comment with a sneer. Agreeing angrily?
Re: AI language models are struggling to “get” math
#90How much of this is just "AI is bad at everything", but in the math case, it's easier for the lay person to tell . It's all just passable garbled nonesense that the reader (goes to lengths) to interept based on their prior knowledge, which is not expressed in the syntax of what these systems output. In the case of mathematics, we're far less willing to "BS away" the interpretive failures. But if we were equally deman…
AI is bad at music also. Even the state of the art transformer models can't produce more than a few seconds of coherent melodic phrases.
That's wrong, and shows how ignorant you are of SOTA techniques for music generation. They are far ahead of that.