Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pure-Python symbolic regression that rediscovered Kepler's law from 8 data point

github.com

Post reply on HN
Print view View original

11–20 of 30 posts

Re: Pure-Python symbolic regression that rediscovered Kepler's law from 8 data point

#11

Post by sinuhe69 »

And I wonder if somebody has tried with the available galactic data and see if the genetic programming can come up with a better formula than MOND or Einstein's general relativity.

For simple problems as Kepler's law, a quick detour on Desmos will show a perfect fit for power law instantly. In general, there are many important criteria for a better curve fitting (for ex. independent, normal distributed residuals), not just R, so I hope the author has/will incorporate them into the search to create a more robust result.

Re: Pure-Python symbolic regression that rediscovered Kepler's law from 8 data point

#12

Post by nyrikki »

ziofill wrote: post #10

The battery example makes no sense: capacity_SOH ≈ 0.913 − 0.352 · tanh( cycle^((temperature/cycle)^0.485) ) I understand this fits the data, but exponents should be dimensionless, what is temperature/cycle?

I am not saying this result is correct, but you don't have unit safety in python at all

In this example temperature would be a magnitude, not a unitful value.

At least in the ISO 8000whatever convention where a value is the product of a unit and a magnitude like most people are use to.

Here is a Terry Tao post with more information[0] on why the convention is there, but as he mentions, in differential geometry and Clifford/Geometric Algebra you do things like add vectors to scalers all the time.

[0] https://terrytao.wordpress.com/2012/12/29/a-mathematical-for...

Re: Pure-Python symbolic regression that rediscovered Kepler's law from 8 data point

#13

Post by [deleted] »

nyrikki wrote: post #12
ziofill wrote: post #10

The battery example makes no sense: capacity_SOH ≈ 0.913 − 0.352 · tanh( cycle^((temperature/cycle)^0.485) ) I understand this fits the data, but exponents should be dimensionless, what is temperature/cycle?

I am not saying this result is correct, but you don't have unit safety in python at all In this example temperature would be a magnitude , not a unitful value. At least in the ISO 8000whatever convention where a value is the product of a unit and a magnitude like most people are use to. Here is a Terry Tao post with more information[0] on why the convention is there, but as he mentions, in differential geometry and C…

[deleted]

Re: Pure-Python symbolic regression that rediscovered Kepler's law from 8 data point

#15

Post by ziofill »

nyrikki wrote: post #12
ziofill wrote: post #10

The battery example makes no sense: capacity_SOH ≈ 0.913 − 0.352 · tanh( cycle^((temperature/cycle)^0.485) ) I understand this fits the data, but exponents should be dimensionless, what is temperature/cycle?

I am not saying this result is correct, but you don't have unit safety in python at all In this example temperature would be a magnitude , not a unitful value. At least in the ISO 8000whatever convention where a value is the product of a unit and a magnitude like most people are use to. Here is a Terry Tao post with more information[0] on why the convention is there, but as he mentions, in differential geometry and C…

Your answer makes no sense either

Re: Pure-Python symbolic regression that rediscovered Kepler's law from 8 data point

#16

Post by [deleted] »

aesthesia wrote: post #9
sade_95 wrote: post #8

Earlier quoted context omitted.

You're right, and I'd go further: the inductive bias isn't incidental, it's the whole product. Short trees over {+, *, sqrt, exp, ...} plus a parsimony penalty is basically Occam's razor made executable. A bias-free learner can't generalize at all (no free lunch), so the honest question is whether this particular bias matches the domain. For physics it has an unreasonably good track record though that's the mystery o…

Thank you, Claude.

[deleted]

Re: Pure-Python symbolic regression that rediscovered Kepler's law from 8 data point

#17

Post by sade_95 »

ziofill wrote: post #10

The battery example makes no sense: capacity_SOH ≈ 0.913 − 0.352 · tanh( cycle^((temperature/cycle)^0.485) ) I understand this fits the data, but exponents should be dimensionless, what is temperature/cycle?

Fair point, and nyrikki has it right: the engine never sees units. Everything is normalized to dimensionless magnitudes before the search, so that formula is an empirical fit on pure numbers, not a physical law. I should say that more clearly in the README.

Enforcing dimensional consistency during the search is a known extension (AI Feynman does a version of it) and honestly it's a good roadmap item, a dimensionally sound form would be more trustworthy.

Re: Pure-Python symbolic regression that rediscovered Kepler's law from 8 data point

#18

Post by sade_95 »

sinuhe69 wrote: post #11

And I wonder if somebody has tried with the available galactic data and see if the genetic programming can come up with a better formula than MOND or Einstein's general relativity. For simple problems as Kepler's law, a quick detour on Desmos will show a perfect fit for power law instantly. In general, there are many important criteria for a better curve fitting (for ex. independent, normal distributed residuals), no…

Funny you ask, I actually spent a weekend on exactly that. Took the SPARC rotation curve data (~2700 points, 150+ galaxies), built the radial acceleration relation from the raw files and let the engine search. Honest result: it recovered a0 around 1.2e-10 m/s2 like the published fits, and found a different functional form (a log-parabola) that fits exactly as well as the MOND interpolating function, difference of 0.0004 dex. But "exactly as well" is the problem: at the current scatter the data can't tell the forms apart. Which I guess is why the interpolating function debate never dies. No new physics from me, but it was a good way to hit the real wall of the field.

And agreed on residuals, R2 alone is weak. Hold-out validation is already in there, proper residual diagnostics are a fair ask for the roadmap.

Re: Pure-Python symbolic regression that rediscovered Kepler's law from 8 data point

#19

Post by srean »

sade_95 wrote: post #8
srean wrote: post #7

Earlier quoted context omitted.

This is not surprising at all and depends on the inductive bias hardcoded in the search. There are infinite number of curves that agree on those 8 points and deviate from Kepler 's law everywhere else. On such 'trajectories' this algorithm would have performed badly.

You're right, and I'd go further: the inductive bias isn't incidental, it's the whole product. Short trees over {+, *, sqrt, exp, ...} plus a parsimony penalty is basically Occam's razor made executable. A bias-free learner can't generalize at all (no free lunch), so the honest question is whether this particular bias matches the domain. For physics it has an unreasonably good track record though that's the mystery o…

I would argue that Kepler's success influenced the choice of the inductive bias. In that case the claim that Kepler took years what this can do in seconds is not an unbiased position to take.

I do see a great value in conjecturing possible solutions that needs to be verified with domain specific knowledge.

Recall that Ptolemaic epicycles were a great fit, in fact a better fit than Copernicus's heliocentric model. This makes me wary of deep NNs in Physics.

Re: Pure-Python symbolic regression that rediscovered Kepler's law from 8 data point

#20

Post by sade_95 »

srean wrote: post #19
sade_95 wrote: post #8

Earlier quoted context omitted.

You're right, and I'd go further: the inductive bias isn't incidental, it's the whole product. Short trees over {+, *, sqrt, exp, ...} plus a parsimony penalty is basically Occam's razor made executable. A bias-free learner can't generalize at all (no free lunch), so the honest question is whether this particular bias matches the domain. For physics it has an unreasonably good track record though that's the mystery o…

I would argue that Kepler's success influenced the choice of the inductive bias. In that case the claim that Kepler took years what this can do in seconds is not an unbiased position to take. I do see a great value in conjecturing possible solutions that needs to be verified with domain specific knowledge. Recall that Ptolemaic epicycles were a great fit, in fact a better fit than Copernicus's heliocentric model. Thi…

You are right, and I concede the point. I chose the operators and the parsimony rule knowing already that laws like Kepler exist. So "Kepler took 10 years" is a bit of theater, the honest claim is only: if a short law exists in this basis, the search finds it fast. Kepler had to invent the idea that such law exists at all, and fight Tycho's data with no computer. Not the same job.

For the epicycles, for me this is exactly the argument for parsimony pressure. Epicycles fit better because you can always add one more circle, same as adding parameters in a NN. They lose on description length, and that is the only defense I have too. And it is not perfect: on noisy data my tool produces its own epicycles, formulas that fit very well and mean nothing. One battery model it gave me predicted the battery heals itself after cycle 264. Great fit, zero physics. So yes, a conjecture machine that a domain expert must verify. No more than that.

Post reply on HN