Live data from Hacker News

AI language models are struggling to “get” math

spectrum.ieee.org

121–130 of 201 posts

Re: AI language models are struggling to “get” math

#121
post #84

Earlier quoted context omitted.

Have you ever used Github CoPilot? It does a lot of useful work, automating away rote typing in programming. Have you tried Dall-E or Stable Diffusion? They make good looking images. This comment seems completely unmoored from where the state of the art is right now.

Math follows a completely different approach with respect to how machine-learning AIs do their thing. Reason derives its strength in having a few primitives and creating new assertions through the transformation of symbols by following precise rules (which is how algorithms work). In ML-based AIs, everything is imprecise and probabilistic, and this kind of generation gets its strength from building recognizable from…

Automated theorem proving is the same problem as “complete and label the diagram”, which image generation is okay at.

Work in progress for sure, though.

Re: AI language models are struggling to “get” math

#123

Earlier quoted context omitted.

Language models aren't built for math. Their improvement/training cycles aren't sensitive to the exactness and rule-based nature of mathematical language, plus there are probably a lot of bad/misleading examples of math in the source data. You'd have to be unrealistically pessimistic to call what GPT-3 and other huge language models produce "nonsense".

It's not that they were not built for math, but more like verification is hard. But it's hard for humans as well. A large generative model + a fast verifier could do wonders. AlphaGo was built on that - the model can propose moves, but you can verify who won in the end. There are some code generation models that write their own tests as well, or use externally provided tests to verify their solutions. The DeepMind ma…

I disagree. It is that they were not built for math. While brain analogies are shittier than most people assume, this is like trying to do math in your head without being allowed to think through calculations.

Re: AI language models are struggling to “get” math

#124
post #116
post #97

Earlier quoted context omitted.

I don't see a lot of progress in AudioLM compared to results from 2018: https://storage.googleapis.com/magentadata/papers/maestro/in... What's more interesting and concerning - listen carefully to the first piano continuation example from AudioLM, notice the similarity of the last 7 seconds to Moonlight sonata: https://youtu.be/4Tr0otuiQuU?t=516 I'm afraid we will see a lot of this with music generation models in the…

There are quite simple tricks to avoid repetition/copying in NNs, e.g. by (1) training a model to predict the "popularity" of the main model's outputs and penalizing popular/copied productions by backpropping through that model so as to decrease the predicted popularity, or (2) by conditioning on random inputs (LLMs can be prompted with imaginary "ID XXX" prefixes before each example to mitigate repetitions), or (3)…

I'm not sure what you mean by "backpropping through that model so as to decrease the predicted popularity". During training, we train a model to literally reproduce famous chunks of music exactly as they are in the training set. We can also learn to predict popularity at the same time, but we can't backpropagate anything that will reduce popularity, because this would directly contradict the main loss objective of exact reproduction.

Having said that, I think the idea of predicting popularity is good - we can use it for filtering already generated chunks during post-training evaluation phase.

I don't think the other two methods you suggest would help here, we want to generate while conditioning on famous pieces, and we don't want to increase temperature if we want to generate conservative, but still high quality pieces.

It's true that we (humans) are less sensitive to plagiarism in the text output, but even for LLMs it is a problem when it tries to generate something highly creative, such as poetry. I personally noticed multiple times a particular beautiful poetry phrases generated by GPT-2 only to google it and find out they were copied verbatim from a human poem.

Re: AI language models are struggling to “get” math

#125
> “I think there’s this notion that humans doing math have some rigid reasoning system—that there’s a sharp distinction between knowing something and not knowing something,” says Ethan Dyer, a machine-learning expert at Google. But humans give inconsistent answers, make errors, and fail to apply core concepts, too. The borders, at this frontier of machine learning, are blurred.

This part resonates with me. There was a time when I could calculate congruent modulo problems with exponents, but I couldn’t do it step by step, I could only “hallucinate” in a fuzzy way to arrive at the solution, somehow like recalling the solution from memory.

When we have to explain our reasoning we can’t think the same way. It’s like thinking with a debugger attached.

Re: AI language models are struggling to “get” math

#127

Earlier quoted context omitted.

can you actually share what "current AI models" are then? Not trying to be rude, but you just said "na ah" and then refused to argument any position.

Current LLMs are "modeling" something according to pretty much any sense of the word "model". In the technical, computational linguistics sense, LLMs are language models that give a conditional posterior distribution over sentences. Given some (constrained) context, the model tells you the posterior distribution over sentences in or around that context. In the nontechnical, layman sense of the word, they are a system…

A more layman way to describe it that avoids too much over simplification is that these learning models try to group things and apply probabilities to sequences of groupings.

E.g.; A word is a grouping of letters, try to find the sequences of letters with the highest probabilities.

A phrase is a grouping of words, with punctuation marks. Try to find the sequences of words with the highest probabilities.

A sentence is a grouping of phrases. Try to find the highest probability sequences.

A paragraph is a sequence of sentences. And so on and so on.

Within very narrow domains (specific writing styles, say technical or legal writing), these models can be very accurate, since the sequencing of words into phrases, and phrases into sentences, and sentences into paragraphs etc., are very predictable. People call this kind of predictable sequencing a 'style', and it aids us in understanding text more quickly. More generally across all domains, it's much harder to accurately predict these sequences, because AI identify the 'style' of a text, purely from the text itself. No context surrounding the text is given to the AI, and so it guesses.

For example:

a political press release, will be written in one style of writing. And a company marketing press release will be written in a slightly different style of writing. As humans, we can easily distinguish between what is commercial marketing, and what is political, because we are given that information upfront. In latin (the choice language for some mathematicians and logicians for historical reasons), we have the information 'a priori'. A learning algorithm, isn't given that information up front, and must determine only from the text itself, whether it is more likely to be a marketing release selling some product, and therefore it should adopt a certain language style, or a political release selling an ideology and therefore should adopt a slightly different language style.

When we don't know the right answer, and have no way to determine it, the solution that most computers are programmed to adopt is a minimax solution, i.e., minimise the maximum possible error. It does this by sort of mixing and matching both marketing and political styles.

When a human reads it, sometimes it looks very strange and funny. Usually this is because it has some distinguishing feature, that we can immediately recognise as placing it as either a political or marketing document, i.e., a company name, a political party, a corporate or political letterhead, a famous person's name etc. The computer naturally doesn't know who Donald Trump is, since we haven't taught it who or what a Trump is, so it doesn't give it any precedence over any other word on the page. Actually, in the case of Donald Trump, I bet if you took the dates off of all of his tweets, even humans would have a hard time distinguishing if they were political or commercial in nature.

Re: AI language models are struggling to “get” math

#128
post #109

Earlier quoted context omitted.

sure, but co-pilot is mostly just copying code (see, for example, the issue with it producing quake source code). If you think of AI as a dial from sample(data) to mean(data), then as the dial is turned towards the mean() you get more "generic" results, but also more garbled ones. Copilot is more like a search engine, having turned the dial more towards sample(). The real invention of the NN is simply to provide that…

> It's all just passable garbled nonesense that the reader (goes to lengths) to interept based on their prior knowledge, which is not expressed in the syntax of what these systems output. > It's still little more than a fancy search. I feel like the goalposts have been moved between your two comments. CoPilot is obviously not producing garbled nonsense, and it's also not just printing the top result from StackOverflo…

As someone who uses co-pilot daily - it often does print garbled nonsense (semantic nonsense, not syntactic nonsense - if you get my meaning).

Re: AI language models are struggling to “get” math

#129
post #84

Earlier quoted context omitted.

Have you ever used Github CoPilot? It does a lot of useful work, automating away rote typing in programming. Have you tried Dall-E or Stable Diffusion? They make good looking images. This comment seems completely unmoored from where the state of the art is right now.

sure, but co-pilot is mostly just copying code (see, for example, the issue with it producing quake source code). If you think of AI as a dial from sample(data) to mean(data), then as the dial is turned towards the mean() you get more "generic" results, but also more garbled ones. Copilot is more like a search engine, having turned the dial more towards sample(). The real invention of the NN is simply to provide that…

I think you are missing the conditional, contextual nature of language models. They mix things in coherent ways, they adapt to the request. Google doesn't create new things when they don't exist, and the pre-written code examples on the internet will never adapt to your needs.

But I agree with you that everything they do seems intelligent because 'intelligence' was in the training data. Not much different from us, if you raise a human removed from society (take his intelligent training data away) he will not accomplish almost anything on his own.

Re: AI language models are struggling to “get” math

#130
post #112

Earlier quoted context omitted.

It’s already being done and will only get better: https://twitter.com/sergeykarayev/status/1569377881440276481

I suspect it's not solved, because solving this (beyond some trick/toy examples) is essentially solving General AI.

It's unclear what you mean by "solved". Even a human can't turn every arbitrary problem into code to solve, but we still consider humans "generally intelligent".

GPT3 can't turn as many problems into code as I can, but it can do some, and GPT4 (or whatever) will be able to do more, etc.

Post reply on HN