Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI isn’t outthinking mathematicians, it’s out-remembering them

davidepiffer.com

Post reply on HN
Print view View original

321–330 of 545 posts

Re: AI isn’t outthinking mathematicians, it’s out-remembering them

#321

Post by smj-edison »

doginasuit wrote: post #316

A superhuman working memory is exactly how I've been describing the LLM advantage. Paired with the unreliability of its reasoning and judgment, it is what makes AI a supplement to human intelligence, not a replacement. On the other side, LLMs make random mistakes and wrong choices and they have a bias toward writing more code instead of less. You can make up for this to some degree by running another LLM against thei…

I also find them bad at what I call "abstraction compression." They're really bad at noticing when a helper function is needed, when some structure they repeated five times slightly differently can become a struct, when a whole section of code can be encapsulated in a simpler design.

I'm lucky that for my side project (an interpreter) I've written all the code myself, so I've built up its design in my mind over the past year, and so as I mull over what I'm writing I start coming up with simpler designs.

Interestingly using Opus 5 (and LLMs in general) has made me worse at this, since I don't feel the pain of writing something over and over again. On the other hand, I don't really want to implement a whole stdlib, so I have it write more of the auxiliary code. The hardest thing is that I have to manually manage the context, which is painful when I personally remember every helper function and why. I have to remember to keep the list of helper functions updated, which is irritating.

Re: AI isn’t outthinking mathematicians, it’s out-remembering them

#322

Post by fma »

grahamburger wrote: post #210
hibikir wrote: post #198

I suspect that a lot about what we call being very intelligent is ultimately out-remembering people around us. I think of all the times in my software career when I did something that others considered very high performance, it either came down to either having more energy than others at tackling a problem they thought was more trouble than it was worth, or just bringing back random knowledge from previous jobs or se…

People have told me I was smart since I was a kid, but I can't remember for shit. I had a thought when I was fairly young that the only reason I was (maybe, sometimes) outperforming others intellectually is that I was habitually compensating for my poor memory by working things out on the fly, while others could rely more on rote memorization. Anyway, takes all kinds I guess!

I remember for math tests in high school/college it was easier for me to derive formulas than to remember it.

Re: AI isn’t outthinking mathematicians, it’s out-remembering them

#323

Post by Razengan »

bogzz wrote: post #247
Razengan wrote: post #138

That's exactly what the strength of AI is, no? Reading all the world's knowledge and recalling it in an instant and seeing where it applies. Precisely perfect for replacing lawyers, if nothing else..

For instance, Opus 5 yesterday critiqued my resume and mentioned a ridiculous little detail-- my phone number area code didn't match the state in which I currently work (I know, I should anonymize but I couldn't be bothered). It failed to notice that I both worked in (previously) and studied in the state of my area code. This was just two pages of text, set to highest effort. Why would you want to replace your lawyer…

> Why can't we be fascinated by emergent properties of intelligence without immediately jumping 10 steps into the future

Because, when computers first came out, and filled entire rooms, people predicted they'd eventually shrink down to fit the crevice of your bum and everyone on the planet would have them

When the first cars came out, people predicted that within 10 years, city streets would no longer be drowning in horse dung

Do people who rant and rail against every new technology in its infancy, actively choose to forget history, or simply didn't learn about the many similar instances in the past?

Re: AI isn’t outthinking mathematicians, it’s out-remembering them

#324

Post by vatsachak »

whateverboat wrote: post #319

It was duuring my PhD I realised that thinking is essentially 1. remembering all the different information to remember all the tricks 2. trying all the different tricks in the problem 3. optimizing deciding which to try based on different information 4. trying random things to discover patterns (and hence new tricks) 5. explaining your tricks to others so that they can do the first 4 steps independently and come up w…

LLMs are enormously good 1. and 2. But humans are much better at the rest.

I don't want to read another LLM documentation ever again. The information to text ratio is way too small.

Re: AI isn’t outthinking mathematicians, it’s out-remembering them

#325

Post by Waterluvian »

hibikir wrote: post #198

I suspect that a lot about what we call being very intelligent is ultimately out-remembering people around us. I think of all the times in my software career when I did something that others considered very high performance, it either came down to either having more energy than others at tackling a problem they thought was more trouble than it was worth, or just bringing back random knowledge from previous jobs or se…

The amount of times I’ve invented something genius in software dev to later discover it has a name…

Re: AI isn’t outthinking mathematicians, it’s out-remembering them

#326

Post by aetherspawn »

hibikir wrote: post #198

I suspect that a lot about what we call being very intelligent is ultimately out-remembering people around us. I think of all the times in my software career when I did something that others considered very high performance, it either came down to either having more energy than others at tackling a problem they thought was more trouble than it was worth, or just bringing back random knowledge from previous jobs or se…

In line with the other comment on this post: I also am classified as “brilliant” by people and IQ tests. My country gave me an award for being a great inventor. But my memory is worse than anyone I know.

I can’t remember the lyrics to any songs, or the lines from any movies, or what I’m doing this week. I don’t think it’s a strong hypothesis.

My wife has a nearly eidetic memory, so marrying her was a game changer for counteracting mine :)

Re: AI isn’t outthinking mathematicians, it’s out-remembering them

#327

Post by Aeolun »

grahamburger wrote: post #210
hibikir wrote: post #198

I suspect that a lot about what we call being very intelligent is ultimately out-remembering people around us. I think of all the times in my software career when I did something that others considered very high performance, it either came down to either having more energy than others at tackling a problem they thought was more trouble than it was worth, or just bringing back random knowledge from previous jobs or se…

People have told me I was smart since I was a kid, but I can't remember for shit. I had a thought when I was fairly young that the only reason I was (maybe, sometimes) outperforming others intellectually is that I was habitually compensating for my poor memory by working things out on the fly, while others could rely more on rote memorization. Anyway, takes all kinds I guess!

I think the fact that ‘just work it out on the fly’ is a viable response to ‘I cannot remember things’ is what makes the intelligence right?

Re: AI isn’t outthinking mathematicians, it’s out-remembering them

#328

Post by raducu »

hibikir wrote: post #198

I suspect that a lot about what we call being very intelligent is ultimately out-remembering people around us. I think of all the times in my software career when I did something that others considered very high performance, it either came down to either having more energy than others at tackling a problem they thought was more trouble than it was worth, or just bringing back random knowledge from previous jobs or se…

> out-remembering people around us.

Doing an angular -> react refactor with Opus 5 at work.

Amazed at how good it is but also amazed at how dumb it is after I told it I want a very basic and specific code review:do the file names match the contents? is one functionality/concern split in multiple files? do the variable names match what they do? does a method read nicely?

Anyway, I was very surprised at how bad it can be at this.

I suspect it's because the LLM simply does not focus well enough on a single method.

My gut modelling is that the critical files are quite small, some 4k LOC and the LLM tries to read them all in one go and my requirement is surprisingly difficult but the LLM does not realize it, loads all the input into context and simply cannot find a good signal per token/group of tokens.

I guess I can do it because I literally cannot fit 4k LOC in my head and I actually read only tens of lines but I can reeealy dig deep in those lines.

The LLM can remember lots, it has 1M context window, was trained with needle in the haystack problems, but it can severely underestimate how much computation is required with TASTE/universal computation type tasks.

Yes, LLMs are incredible when it comes to delivering well defined outcomes, it's evident they were RL trained very well. But they are incredibly bad at understanding nuance in a text.

It's incredible how agentic coding benchmaxing seems to correlate with writing good poetry but at the same time finding good names for files and variables and wiring beautiful code does not.

Same with getting basic details of who said what in an "adversarial" conversation between 2 humans -- LLMs are incredibly bad at it, with all their "almost discovering new maths" skills

Re: AI isn’t outthinking mathematicians, it’s out-remembering them

#329

Post by Fr0styMatt88 »

I often think “what will ‘maintainable’ code look like in the future?” given this kind of thing. Much of ‘good coding’ is about optimising the codebase for workability for a human intellect with human working memory.

Though having said that, from the studies I’ve come across it seems like LLMs tend to generate more verbose code but perform better over the long-term when the code is maintained and not allowed to sprawl all over the place.

I’m mainly using AI for tools development and in that context I often wonder if I’ve just developed a career-long habit of over-abstracting. Like, the tools work without all the fuss I might have put in at the beginning and you just iterate and evolve as you go.

Re: AI isn’t outthinking mathematicians, it’s out-remembering them

#330

Post by stevenalowe »

and yet very little access to common sense :)

the article cites the ability to keep a ton of details in working memory as an advantage, I'm not so sure that it is - perhaps it is quality over quantity; a compression of everything known into a smaller set of interlocking patterns should provide a more useful generalization (if correct, of course). The human's ability to perceive 7-10 concepts might just be the tips of the icebergs composed of a gazillion micro-concepts, i.e. our working memory is enormous, but not consciously accessible

also, the older neural capacity estimates are way wrong [1]: dendrites also compute

[1] https://www.yahoo.com/news/science/articles/neuroscientists-...

Post reply on HN