A 'secret weapon' that has served me very well for learning classifiers is to first learn a good linear classifier. I am almost hesitant to give this away (kidding). Use the non-thresholded version of that linear classifier output as one additional feature-dimension over which you learn a decision tree. Then wrap this whole thing up as a system of boosted trees (that is, with more short trees added if needed). One of…
There are decision trees for what you want do do.
Oblique Decision trees, Model Trees. (M5 Trees for example), Logistic Model Trees (LMT) or Hierarchical Mixture of Experts (HME).
A 'secret weapon' that has served me very well for learning classifiers is to first learn a good linear classifier. I am almost hesitant to give this away (kidding). Use the non-thresholded version of that linear classifier output as one additional feature-dimension over which you learn a decision tree. Then wrap this whole thing up as a system of boosted trees (that is, with more short trees added if needed). One of…
There are decision trees for what you want do do. Oblique Decision trees, Model Trees. (M5 Trees for example), Logistic Model Trees (LMT) or Hierarchical Mixture of Experts (HME).
Yes.
I mention restricted oblique trees in passing in my original comment. In my experience, oblique trees tend to add considerable complexity, the others more so. Of course whether the complexity is warranted or not will depend on the dataset.
The merit of what I used is in its simplicity. Any simple ML library would have a linear classifier and a tree learner.
Super easy to implement, train, maintain, debug. One to two person team can handle this fine.
Isn’t that exactly how humans (and even animals) operate?
Human societies look for actual major correlations and establish classifications. Except with scientific-minded humans, we often also want, to know the why behind the correlations. David Hume got involved w that… https://brainly.com/question/50372476
Let me ask a provocative question. What, ultimately, is the difference between knowledge and bias?
If you sum up experimental physics into one heuristic it is “avoid fooling yourself with assumptions” - I left physics over a decade ago, but I feel confident that physicists still work hard to understand what they observe and don’t let LLMs have all the fun. If there’s one field of science where the scientists are legitimately allowed to go all the way back to basics, it’s elementary particle physics.
In general I would agree. I think it holds true at the highest levels. What worries me is the noticeable uptick of presentations of the sort -- look ma better fit ... deep neural nets. These are mostly by more junior folks, but not necessarily. I have been in the audience in many. These and the uptick in research proposals funded by providers of infra for such DNNs. I have been in the audience of many. A charitable r…
Again, said as someone out of the fray, let’s hope it self-corrects. Physics is a very community driven field, and the young must try new things, and be allowed to, it is part of progress. It is when the seniors surrender the standard of quality they carry, we have trouble.
And here, indeed, particle physics can be uniquely vulnerable - given the complexity and economics of the research, it is hard to falsify claims made with new methods if the established researchers cave too easily.
I worked (professionally) on a product a few years ago based upon decision tree and random forest classifiers. I had no background in the math and had to learn this stuff which has payed dividends as llms and AI have become hyped. This is one of the best explanations I've seen and has me super nostalgic for that project.
Gonna try to cook up something personal. It's amazing how people are now using regression models basically all the time and yet no-one uses these things on their own.
In general I would agree. I think it holds true at the highest levels. What worries me is the noticeable uptick of presentations of the sort -- look ma better fit ... deep neural nets. These are mostly by more junior folks, but not necessarily. I have been in the audience in many. These and the uptick in research proposals funded by providers of infra for such DNNs. I have been in the audience of many. A charitable r…
Again, said as someone out of the fray, let’s hope it self-corrects. Physics is a very community driven field, and the young must try new things, and be allowed to, it is part of progress. It is when the seniors surrender the standard of quality they carry, we have trouble. And here, indeed, particle physics can be uniquely vulnerable - given the complexity and economics of the research, it is hard to falsify claims…
Amen.
To support your point of view, I haven't encountered this in particle physics. It's from other branches. I am not a Physicist myself, happened to be in a position to observe funding requests, request for collaborations.
I worked (professionally) on a product a few years ago based upon decision tree and random forest classifiers. I had no background in the math and had to learn this stuff which has payed dividends as llms and AI have become hyped. This is one of the best explanations I've seen and has me super nostalgic for that project. Gonna try to cook up something personal. It's amazing how people are now using regression models…
I worked on a product which was the best ID reader in the world at the time 25 years ago. The OCR engine was based on Decision tree and "Random Forest" (I suspect the name did exist) with only 3 trees. It was very effective as a secret weapon of the competitiveness. I tried to train a NN with a framework called SNNS(Stuttgart Neural Network Simulator) as the 4th tree complement to the existing 3.
Today, hand writing OCR is a "hello world" sample in Tensorflow.
> (ii) linear functions are terrible where equi-label regions have a partitioned structure. Could you explain what "equi-label regions having a partitioned structure" mean?
I missed a word "recursively", that I have edited in my original comment now. Consider connected regions in the domain that have the same label. Much like countries on a political map. The situation where this has a short description in terms of recursive subdivision of space, is what I am calling a partitioned structure. It's really rather tautological.
"recursively partitioned" sounds like a fractal to me. Not sure what you really mean.