Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fitting an elephant with four non-zero parameters

arxiv.org

Post reply on HN
Print view View original

21–30 of 155 posts

Re: Fitting an elephant with four non-zero parameters

#21

Post by jampekka »

maitola wrote: post #16

I love the ironic side of the article. Perhaps they should add the reason for it, from Fermi's and Neumann's. When you are building a model of reality in Physics, If something doesn’t fit the experiments, you can’t just add a parameter (or more) variate it and fit the data. The model should have zero parameters, ideally, or the least possible, or, even at a more deeper level, the parameters should emerge naturally fr…

This was mentioned in the first paragraph of the paper. The paper is mostly humoristic.

That said, the wisdom of the quip has been widely lost in many fields. In many fields data is "modeled" with huge regression models with dozens of parameters or even neural networks with billions of parameters.

> In 1953, Enrico Fermi criticized Dyson’s model by quoting Johnny von Neumann: “With four parameters I can fit an elephant, and with five I can make him wiggle his trunk.”[1]. This quote is intended to tell Dyson that while his model may appear complex and precise, merely increasing the number of parameters to fit the data does not necessarily imply that the model has real physical significance.

Re: Fitting an elephant with four non-zero parameters

#22

Post by mananaysiempre »

btown wrote: post #6
dheera wrote: post #4

I wish there was more humor on arXiv. If I could make a discovery in my own time without using company resources I would absolutely publish it in the most humorous way possible.

There's plenty of humor on arXiv, and that's part of why it's so incredible! Some lists: https://academia.stackexchange.com/questions/86346/is-it-ok-... https://www.ellipsix.net/arxiv-joke-papers.html

Joke titles and/or author lists are also quite popular, e.g. the Greenberg, Greenberger, Greenbergest paper[1], a paper with a cat coauthor whose title I can’t seem to recall (but I’m sure there’s more than one I’ve encountered), or even the venerable, unfortunate in its joke but foundational in its substance Alpher, Bethe, Gamow paper[2]. Somewhat closer to home, I think computer scientist Conor McBride[3] is the champion of paper titles (entries include “Elimination with a motive”, “The gentle art of levitation”, “I am not a number: I am a free variable”, “Clowns to the left of me, jokers to the right”, and “Doo bee doo bee doo”) and sometimes code in papers:

  letmeB this (F you) | you == me = B this
                      | otherwise = F you
  letmeB this (B that)            = B that
  letmeB this (App fun arg)       = letmeB this fun `App` letmeB this arg
(Yes, this is working code; yes, it’s crystal clear in the context of the paper.)

[1] https://arxiv.org/abs/hep-ph/9306225

[2] https://en.wikipedia.org/wiki/Alpher%E2%80%93Bethe%E2%80%93G...

[3] http://strictlypositive.org/

Re: Fitting an elephant with four non-zero parameters

#23

Post by bee_rider »

Ya know, in academic writing I tend to struggle with making it sound nice and formal. I try not to use the super-stilted academic style, but it is still always a struggle to walk the line between too loose and too jargony.

Maybe this sort of thing would be a really good tradition. Everyone must write a very silly article with some mathematical arguments in it. Then, we can all go forward with the comfort of knowing that we aren’t really at risk of breaking new grounds in appearing unserious.

It is well written and very understandable!

Re: Fitting an elephant with four non-zero parameters

#24

Post by dilawar »

maitola wrote: post #16

I love the ironic side of the article. Perhaps they should add the reason for it, from Fermi's and Neumann's. When you are building a model of reality in Physics, If something doesn’t fit the experiments, you can’t just add a parameter (or more) variate it and fit the data. The model should have zero parameters, ideally, or the least possible, or, even at a more deeper level, the parameters should emerge naturally fr…

Hmm..

Hodgin and Huxley did ground-breaking work on squid's giant axon and modelled neural activity. They had multiple parameters extracted from 'curve fitting' of recorded potential and injected currents which were much later mapped to sodium channels. Similarly, another process to potassium channels.

I woudnt worry too much having multiple parameters -- even four when 3 can't just explain the model.

Re: Fitting an elephant with four non-zero parameters

#29

Post by elijahbenizzy »

visarga wrote: post #15
elijahbenizzy wrote: post #13

This is humorous (and well-written), but I think its more than that. I'm always making the joke (observation) that ML (AI) is just curve-fitting. Whether "just curve-fitting" is enough to produce something "intelligent" is, IMO, currently unanswered, largely due to differing viewpoints on the meaning of "intelligent". In this case they're demonstrating some very clean, easy-to-understand curve-fitting, but it's reall…

> Whether "just curve fitting" is enough to produce something "intelligent" is, IMO, currently unanswered Continual "curve fitting" to the real world can create intelligence. What is missing is not something inside the model. It's missing a mechanism to explore, search and expand its experience. Our current crop of LLMs ride on human experience, they have not largely participated in creating their own experiences. Th…

There are a whole bunch of assumptions here. But sure, if you view the world as a closed system, then you have a decision as a function of inputs:

1. The world around you 2. The experiences within your (really, the past view of the world around you) 3. Innateness of you (sure, this could be 2 but I think it's also something else) 4. The experience you find + the way you change yourself to impact (1), (2), and (3)

If you think of intelligence as all of these, then you're making the assumption that all that's required for (2), (3), and (4) is "agentic systems", which I think skips a few steps (as the author of an agent framework myself...). All this is to say that "what makes intelligence" is largely unsolved, and nobody really knows, because we actually don't understand this ourselves.

Post reply on HN