Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pure-Python symbolic regression that rediscovered Kepler's law from 8 data point

github.com

Post reply on HN
Print view View original

21–30 of 30 posts

Re: Pure-Python symbolic regression that rediscovered Kepler's law from 8 data point

#21

Post by gus_massa »

ziofill wrote: post #10

The battery example makes no sense: capacity_SOH ≈ 0.913 − 0.352 · tanh( cycle^((temperature/cycle)^0.485) ) I understand this fits the data, but exponents should be dimensionless, what is temperature/cycle?

I've seen crazy stuff for heat exchangers and other stuff used in factories.

When the temperature difference is small, everithing is linear and you get nice formulas.

When the difference of temperature is big and you have liquids with convection and turbulence, you only get empirical formulas with weird exponents. The correct method would be to give the constants with the correct units, but it's usual to specify unit to measure the data instead, and just enter the numbers in the formula.

After a short search in Google, I got this example https://www.researchgate.net/figure/Equations-used-to-obtain...

Re: Pure-Python symbolic regression that rediscovered Kepler's law from 8 data point

#22

Post by sade_95 »

gus_massa wrote: post #21
ziofill wrote: post #10

The battery example makes no sense: capacity_SOH ≈ 0.913 − 0.352 · tanh( cycle^((temperature/cycle)^0.485) ) I understand this fits the data, but exponents should be dimensionless, what is temperature/cycle?

I've seen crazy stuff for heat exchangers and other stuff used in factories. When the temperature difference is small, everithing is linear and you get nice formulas. When the difference of temperature is big and you have liquids with convection and turbulence, you only get empirical formulas with weird exponents. The correct method would be to give the constants with the correct units, but it's usual to specify unit…

This is exactly the niche I had in mind, thank you for the concrete example. Where theory works (small deltas, linear regime) nobody needs my tool. Where turbulence starts, engineers already live with empirical correlations and weird exponents since one century, Nusselt-Reynolds style. A GP search is just a systematic way to produce this kind of correlation, with a Pareto front instead of one formula.

And your point about units is the industrial reality: the convention "measure in these units, then use the numbers" is exactly what the tool does internally. A proper dimensional mode stays on my roadmap, but I feel less bad about it now.

Re: Pure-Python symbolic regression that rediscovered Kepler's law from 8 data point

#23

Post by srean »

sade_95 wrote: post #20
srean wrote: post #19

Earlier quoted context omitted.

I would argue that Kepler's success influenced the choice of the inductive bias. In that case the claim that Kepler took years what this can do in seconds is not an unbiased position to take. I do see a great value in conjecturing possible solutions that needs to be verified with domain specific knowledge. Recall that Ptolemaic epicycles were a great fit, in fact a better fit than Copernicus's heliocentric model. Thi…

You are right, and I concede the point. I chose the operators and the parsimony rule knowing already that laws like Kepler exist. So "Kepler took 10 years" is a bit of theater, the honest claim is only: if a short law exists in this basis, the search finds it fast. Kepler had to invent the idea that such law exists at all, and fight Tycho's data with no computer. Not the same job. For the epicycles, for me this is ex…

I agree and I vehemently share your concern about profusion of tweakable parameters in the model.

There is some misconception in the wild about epicycles models that need not be shared by you specifically. There weren't that many epicycles per orbiting body, but every orbiting body had a few that had to be 'trained' specifically for them.

My fear, and I suspect yours too is that good curve fits done one at a time with mathematical models that are universal approximators (*) rarely, if at all lead to causally explanatory models. In Physics it's the latter that we seek.

(*) Epicycloidal models are a form of Fourier analysis and are a class of universal approximators for periodic trajectories.

Re: Pure-Python symbolic regression that rediscovered Kepler's law from 8 data point

#24

Post by nyrikki »

ziofill wrote: post #15
nyrikki wrote: post #12

Earlier quoted context omitted.

I am not saying this result is correct, but you don't have unit safety in python at all In this example temperature would be a magnitude , not a unitful value. At least in the ISO 8000whatever convention where a value is the product of a unit and a magnitude like most people are use to. Here is a Terry Tao post with more information[0] on why the convention is there, but as he mentions, in differential geometry and C…

Your answer makes no sense either

That is because the way you were introduced to the concepts was targeted at didactic convenience and incremental teaching curricula. Pedagogical efficiency is important but care is also needed when mapping concepts that are accepted as a priori in a specific domain to universal beliefs.

Physics uses the properties of unitful objects to help avoid errors, it is a convention and not (by itself) a fundamental truth.

If your world is physically realizable one, the benefits make this choice a no brainier, but it is still a convention with some convenient outcomes.

A unitful quantity is actually a tuple, specifically (value = quantity x unit)

In computer science and programming, assuming a priori that individual scalers are only useful in the context of units, or that they will behave in the same way can cause you real issues.

Be careful with conventions and assumptions, use them where they serve you well and avoid them where they block you from finding useful mechanical means to solve problems.

Re: Pure-Python symbolic regression that rediscovered Kepler's law from 8 data point

#25

Post by sade_95 »

srean wrote: post #23
sade_95 wrote: post #20

Earlier quoted context omitted.

You are right, and I concede the point. I chose the operators and the parsimony rule knowing already that laws like Kepler exist. So "Kepler took 10 years" is a bit of theater, the honest claim is only: if a short law exists in this basis, the search finds it fast. Kepler had to invent the idea that such law exists at all, and fight Tycho's data with no computer. Not the same job. For the epicycles, for me this is ex…

I agree and I vehemently share your concern about profusion of tweakable parameters in the model. There is some misconception in the wild about epicycles models that need not be shared by you specifically. There weren't that many epicycles per orbiting body, but every orbiting body had a few that had to be 'trained' specifically for them. My fear, and I suspect yours too is that good curve fits done one at a time wit…

Thanks for the epicycles clarification, I did not know each body had its own fitted set. This makes the parallel with overfitting even stronger.

And I think we agree on the real problem: a good fit on one problem, made with a universal approximator, is not an explanation. It can be just a compressed description, maybe with a cause behind, maybe not. I cannot fix this, but I added this week a small thing to at least see it: a bootstrap stability check. You resample the data, refit, and look how often the same form comes back. High frequency does not prove anything causal. But low frequency is a good signal that you are only looking at a Fourier-type fit, not a law. The decision stays with the user, the tool only makes it visible.

Re: Pure-Python symbolic regression that rediscovered Kepler's law from 8 data point

#26

Post by sade_95 »

sinuhe69 wrote: post #11

And I wonder if somebody has tried with the available galactic data and see if the genetic programming can come up with a better formula than MOND or Einstein's general relativity. For simple problems as Kepler's law, a quick detour on Desmos will show a perfect fit for power law instantly. In general, there are many important criteria for a better curve fitting (for ex. independent, normal distributed residuals), no…

[dead]

Re: Pure-Python symbolic regression that rediscovered Kepler's law from 8 data point

#27

Post by sade_95 »

ziofill wrote: post #10

The battery example makes no sense: capacity_SOH ≈ 0.913 − 0.352 · tanh( cycle^((temperature/cycle)^0.485) ) I understand this fits the data, but exponents should be dimensionless, what is temperature/cycle?

[dead]

Re: Pure-Python symbolic regression that rediscovered Kepler's law from 8 data point

#28

Post by srean »

sade_95 wrote: post #20
srean wrote: post #19

Earlier quoted context omitted.

I would argue that Kepler's success influenced the choice of the inductive bias. In that case the claim that Kepler took years what this can do in seconds is not an unbiased position to take. I do see a great value in conjecturing possible solutions that needs to be verified with domain specific knowledge. Recall that Ptolemaic epicycles were a great fit, in fact a better fit than Copernicus's heliocentric model. Thi…

You are right, and I concede the point. I chose the operators and the parsimony rule knowing already that laws like Kepler exist. So "Kepler took 10 years" is a bit of theater, the honest claim is only: if a short law exists in this basis, the search finds it fast. Kepler had to invent the idea that such law exists at all, and fight Tycho's data with no computer. Not the same job. For the epicycles, for me this is ex…

> a good fit on one problem, made with a universal approximator, is not an explanation. It can be just a compressed description,

I couldn't have said it better.

Re: Pure-Python symbolic regression that rediscovered Kepler's law from 8 data point

#29

Post by gus_massa »

sade_95 wrote: post #25
srean wrote: post #23

Earlier quoted context omitted.

I agree and I vehemently share your concern about profusion of tweakable parameters in the model. There is some misconception in the wild about epicycles models that need not be shared by you specifically. There weren't that many epicycles per orbiting body, but every orbiting body had a few that had to be 'trained' specifically for them. My fear, and I suspect yours too is that good curve fits done one at a time wit…

Thanks for the epicycles clarification, I did not know each body had its own fitted set. This makes the parallel with overfitting even stronger. And I think we agree on the real problem: a good fit on one problem, made with a universal approximator, is not an explanation. It can be just a compressed description, maybe with a cause behind, maybe not. I cannot fix this, but I added this week a small thing to at least s…

Too late, but I hope you see it. In the original post you said:

> That's Kepler's Third Law (T² ∝ a³), which took Kepler ~10 years to find in 1618. GP_ELITE found it in ~3 seconds.

In future post in HN I recommend to avoid that kind of comparisons, because we all know he did that on paper and didn't even have a good set of functions to use a brute force aproach. It feels like unnecessary linkbait and make people write unfriendly comments. Probably it's better:

fake quote> That's Kepler's Third Law (T² ∝ a³), GP_ELITE found it in ~3 seconds.

Re: Pure-Python symbolic regression that rediscovered Kepler's law from 8 data point

#30

Post by sade_95 »

gus_massa wrote: post #29
sade_95 wrote: post #25

Earlier quoted context omitted.

Thanks for the epicycles clarification, I did not know each body had its own fitted set. This makes the parallel with overfitting even stronger. And I think we agree on the real problem: a good fit on one problem, made with a universal approximator, is not an explanation. It can be just a compressed description, maybe with a cause behind, maybe not. I cannot fix this, but I added this week a small thing to at least s…

Too late, but I hope you see it. In the original post you said: > That's Kepler's Third Law (T² ∝ a³), which took Kepler ~10 years to find in 1618. GP_ELITE found it in ~3 seconds. In future post in HN I recommend to avoid that kind of comparisons, because we all know he did that on paper and didn't even have a good set of functions to use a brute force aproach. It feels like unnecessary linkbait and make people writ…

Thanks for the advice; you're absolutely right ,it was just to grab attention.
Post reply on HN