Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A new Google model is nearly perfect on automated handwriting recognition

generativehistory.substack.com

Post reply on HN
Print view View original

251–260 of 328 posts

Re: A new Google model is nearly perfect on automated handwriting recognition

#251

Post by gf000 »

baq wrote: post #199
jstummbillig wrote: post #40

Earlier quoted context omitted.

I remain confused but still somewhat interested as to a definition of "novel", given how often this idea is wielded in the AI context. How is everyone so good at identifying "novel"? For example, I can't wrap my head around how a) a human could come up with a piece of writing that inarguably reads "novel" writing, while b) an AI could be guaranteed to not be able to do the same, under the same standard.

If the model can map an unseen problem to something in its latent space, solve it there, map back and deliver an ultimately correct solution, is it novel? Genuine question, ‘novel’ doesn’t seem to have a universally accepted definition here

Good question, though I would say that there may be different grades of novelty.

One grade might be your example, while something like Gödel's incompleteness theorems or Einstein's relativity could go into a different grade.

Re: A new Google model is nearly perfect on automated handwriting recognition

#252

Post by CamperBob2 »

cxvrfr wrote: post #189
CamperBob2 wrote: post #90

Earlier quoted context omitted.

How'd you do at the International Math Olympiad this year?

How would you do multiplying 10000 pairs of 100 digit numbers in a limited amount of time? We don't anthropomorphize calculators though...

One problem for your argument is that transformer networks are not, and weren't meant to be, calculators. Their raw numerical calculating abilities are shaky when you don't let them use external tools, but they are also entirely emergent. It turns out that language doesn't just describe logic, it encodes it. Nobody expected that.

To see another problem with your argument, find someone with weak reasoning abilities who is willing to be a test subject. Give them a calculator -- hell, give them a copy of Mathematica -- and send them to IMO, and see how that works out for them.

Re: A new Google model is nearly perfect on automated handwriting recognition

#253

Post by gf000 »

famouswaffles wrote: post #89
imiric wrote: post #64

Earlier quoted context omitted.

Here's a thought experiment: if modern machine learning systems existed in the early 20th century, would they have been able to produce an equivalent to the theory of relativity? How about advance our understanding of the universe? Teach us about flight dynamics and take us into space? Invent the Turing machine, Von Neumann architecture, transistors? If yes, why aren't we seeing glimpses of such genius today? If we'v…

>Here's a thought experiment: if modern machine learning systems existed in the early 20th century, would they have been able to produce an equivalent to the theory of relativity? How about advance our understanding of the universe? Teach us about flight dynamics and take us into space? Invent the Turing machine, Von Neumann architecture, transistors? Only a small percentage of humanity are/were capable of doing any…

> Only a small percentage of humanity are/were capable of doing any of these. And they tend to be the best of the best in their respective fields.

A definite, absolute and unquestionable no, and a small, but real chance is absolutely different categories.

You may wait for a bunch of rocks to sprout forever, but I would put my money on a bunch of random seeds, even if I don't know how they were kept.

Re: A new Google model is nearly perfect on automated handwriting recognition

#254

Post by CamperBob2 »

cxvrfr wrote: post #190
CamperBob2 wrote: post #139

Earlier quoted context omitted.

Yeah, that must be it.

Well being able to extrapolate solutions to "novel" mathematical exercises based on a very large sample of similar tasks in your dataset seems like a reasonable explanation. Question is how well it would do if it was trained without those samples?

Gee, I don't know. How would you do at a math competition if you weren't trained with math books? Sample problems and solutions are not sufficient unless you can genuinely apply human-level inductive and deductive reasoning to them. If you don't understand that and agree with it, I don't see a way forward here.

A more interesting question is, how would you do at a math competition if you were taught to read, then left alone in your room with a bunch of math books? You wouldn't get very far at a competition like IMO, calculator or no calculator, unless you happen to be some kind of prodigy at the level of von Neumann or Ramanujan.

Re: A new Google model is nearly perfect on automated handwriting recognition

#255

Post by dr_dshiv »

throwout4110 wrote: post #72
throwup238 wrote: post #61

Earlier quoted context omitted.

I use the Portal de Archivos Españoles [1] for Spanish colonial documents. Each country has their own archive but the Spanish one has the most content (35 million digitized pages) The hard part is knowing where to look since most of the images haven’t gone through HRT/OCR or indexing so you have to understand Spanish colonial administration and go through the collections to find stuff. [1] https://pares.cultura.gob.e…

Want to collab on a database and some clustering and analysis? I’m a data scientist at FAIR with an interest in antiquarian docs and books

Hit me up, if you can. I’m focused on neolatin texts from the renaissance. Less than 30% of known book editions have been scanned and less than 5% translated. And that’s before even getting to the manuscripts.

https://Ancientwisdomtrust.org

Also working on kids handwriting recognition for https://smartpaperapp.com

Re: A new Google model is nearly perfect on automated handwriting recognition

#257

Post by vintermann »

ninetyninenine wrote: post #222
vintermann wrote: post #192

Earlier quoted context omitted.

I don't know, that's commendable self-insight, it's true of lots and lots of people but there are few who would admit it!

I am unique. Totally. It is not like HN is flooded with cognition or psychology or IQ articles every other hour. Not at all. And whenever one shows up, you do not immediately get a parade of people diagnosing themselves with whatever the headline says. Never happens. You post something about slow thinking and suddenly half the thread whispers “that is literally me.” You post something about fast thinking and the othe…

Ah, so you were just attempting sarcasm?

Re: A new Google model is nearly perfect on automated handwriting recognition

#258

Post by HarHarVeryFunny »

CamperBob2 wrote: post #254
cxvrfr wrote: post #190

Earlier quoted context omitted.

Well being able to extrapolate solutions to "novel" mathematical exercises based on a very large sample of similar tasks in your dataset seems like a reasonable explanation. Question is how well it would do if it was trained without those samples?

Gee, I don't know. How would you do at a math competition if you weren't trained with math books? Sample problems and solutions are not sufficient unless you can genuinely apply human-level inductive and deductive reasoning to them. If you don't understand that and agree with it, I don't see a way forward here. A more interesting question is, how would you do at a math competition if you were taught to read, then lef…

> A more interesting question is, how would you do at a math competition if you were taught to read, then left alone in your room with a bunch of math books?

But that isn't how an LLM learnt to solve math olympiad problems. This isn't a base model just trained on a bunch of math books.

The way they get LLMs to be good at specialized things like math olympiad problems is to custom train them for this using reinforcement learning - they give the LLM lots of examples of similar math problems being solved, showing all the individual solution steps, and train on these, rewarding the model when (due to having selected an appropriate sequence of solution steps) it is able itself to correctly solve the problem.

So, it's not a matter of the LLM reading a bunch of math books and then being expert at math reasoning and problem solving, but more along the lines "of monkey see, monkey do". The LLM was explicitly shown how to step by step solve these problems, then trained extensively until it got it and was able to do it itself. It's probably a reflection of the self-contained and logical nature of math that this works - that the LLM can be trained on one group of problems and the generalizations it has learnt works on unseen problems.

The dream is to be able to teach LLMs to reason more generally, but the reasons this works for math don't generally apply, so it's not clear that this math success can be used to predict future LLM advances in general reasoning.

Re: A new Google model is nearly perfect on automated handwriting recognition

#259

Post by jstummbillig »

jofla_net wrote: post #244
terminalshort wrote: post #143

Earlier quoted context omitted.

If a LLM had written Linux, people would be saying that it isn't novel because it's just based on previous OS's. There is no standard here, only bias.

Cept its not made Linux (in the absence of it). At any point prior to the final output it can garner huge starting point bias from ingested reference material. This can be up to and including whole solutions to the original prompt minus some derivations. This is effectively akin to cheating for humans as we cant bring notes to the exam. Since we do not have a complete picture of where every part of the output comes f…

> Cept its not made Linux (in the absence of it).

Neither did you (or I). Did you create anything that you are certain your peers would recognize as more "novel" than anything a LLM could produce?

Re: A new Google model is nearly perfect on automated handwriting recognition

#260

Post by fragmede »

hinkley wrote: post #131
fragmede wrote: post #99

Earlier quoted context omitted.

hinkley wrote: > We got the results back. You are a horrible person. I’m serious, that’s what it says: “Horrible person.” > We weren’t even testing for that. joshstrange then wrote: > If you want to listen to the line from Portal 2 it's on this page (second line in the section linked): https://theportalwiki.com/wiki/GLaDOS_voice_lines_(Portal_2) ... as if the fact that the words that hinkley wrote are from a popular…

So if two sentences that make no sense to you sandwich one that does, you should totally accept the middle one at face value. K.

Yes. You chose to repeat those words in that sequence in that place. You could have said anything else in the whole wide world, but you chose to use a quote from an ancient video game stating that someone was horrible. Sorry if I'm being autistic and taking things too literally again, working on having social skills was a different thread from today.
Post reply on HN