41–50 of 67 posts
Is each architecture given one set of random weights? Or is the architecture of the net tested against a bunch of random weights so that it performs well independently of the weights?
Is each architecture given one set of random weights? Or is the architecture of the net tested against a bunch of random weights so that it performs well independently of the weights?
From the paper :
(1) An initial population of minimal neural network topologies is created
(2) each network is evaluated over multiple rollouts, with a different shared weight value assigned at each rollout
(3) networks are ranked according to their performance and complexity
(4) a new population is created by varying the highest ranked network topologies, chosen probabilistically through tournament selection
How is this different from genetic programming?
How was early Machine Learning different from statistics? New names makes things exciting for people to oick up. Who wants to estimate multinomial regression when you can learn a shallow softmax activated neural network! Its all about creating hype.
How is this different from genetic programming?
How is this different from genetic programming?
How was early Machine Learning different from statistics? New names makes things exciting for people to oick up. Who wants to estimate multinomial regression when you can learn a shallow softmax activated neural network! Its all about creating hype.
Some of the very early work in machine learning, in the 1950's and '60s was not statistical. The first "artificial neuron", the Pitts & McCulloch neuron, from 1938 was a propositional logic circuit. Arthur Samuel's 1952 checkers-playing programs used a classical minimax search with alpha-beta pruning.
Machine learning in the '70s and '80s was for the most part not statistical, but logic-based, in keeping with the then-current trend for logic-based AI. Early algorithms did not use gradient descent or other statistical methods and the models they learned were sets of logic rules, and not the parameters of continuous functions.
For instance, a lot of work from that time focused on learning decision lists and decision trees, the latter of which are best remembered today. The focus on rules probably followed from the realisation of the problems with knowledge acquisition for expert systems, that were the first big success of AI.
You can find examples of machine learning research from those times in the work of researchers like Ryszard Michalski, Ross Quinlan (know for C4.5 and IDR and the first-order inductive learner FOIL), (the) Stuart Russel, Tom Mitchell, and others.
Unpopular quote from my image and video processing professor - “The only problem with machine learning is that the machine does the learning and you don’t.” While I understand that is missing a lot of nuance, it has stuck with me over the past few years as I feel like I am missing out on the cool machine learning work going on out there. There is a ton of learning about calculus, probability, and statistics when doin…
For me, the exciting part is to understand how the "blackbox" can learn and to find a representation of the data that makes this box to learn. For instance, I've been working in users profiling and it's been a challenge to find which features and in which representation allow the model to learn. It's fantastic when you make a little change in a feature (for instance, use the median instead of the mean) and your model…
This illustrates the black-box aspect of it. You changed something and the results are affected but you don't know why. Median has a built-in implicit filtering (it's not affected by extreme outliers like the mean), so it could simply be that you needed to filter your inputs. But won't know, because... black box.
Unpopular quote from my image and video processing professor - “The only problem with machine learning is that the machine does the learning and you don’t.” While I understand that is missing a lot of nuance, it has stuck with me over the past few years as I feel like I am missing out on the cool machine learning work going on out there. There is a ton of learning about calculus, probability, and statistics when doin…
ML is more like growing crops than it is about "designing stuff". Growing crops is slow, and you don't know beforehand what the result will be. However, you can still throw a lot of science at growing crops ("plant breeding" is a science), and the same holds for engineering.
Unpopular quote from my image and video processing professor - “The only problem with machine learning is that the machine does the learning and you don’t.” While I understand that is missing a lot of nuance, it has stuck with me over the past few years as I feel like I am missing out on the cool machine learning work going on out there. There is a ton of learning about calculus, probability, and statistics when doin…
"what is exciting to you about your work?" Seeing the black boxes solve problems I know I could not solve manually. If you go into industry, expect to be spending the vast majority of your time wrangling data, integrating the black boxes into other software, and apologizing repeatedly to managers because you have no idea how long it will take to make your black box work, if it ever does. Trying to make plans around A…
Free career advice:
Don't be the boy in the parable of the emperor's clothing. You will be punished. Just smile and nod, say vaguely positive stuff, get your job references, and quickly find a new gig.
I'm pretty sure this was posted a while back (maybe a month or two)?
Unpopular quote from my image and video processing professor - “The only problem with machine learning is that the machine does the learning and you don’t.” While I understand that is missing a lot of nuance, it has stuck with me over the past few years as I feel like I am missing out on the cool machine learning work going on out there. There is a ton of learning about calculus, probability, and statistics when doin…
I’m most excited about what is now being called scientific machine learning, ie machine learning models explicitly structured to learn interpretable models that can be used to understand some scientific domain better. For example, I’m starting to work on using graph RNNs to study dynamical behavior of reinforcement learning agents and how it might give us insights into psychiatric disorders.