Reasoning from analogy is the wrong way of saying what people have in mind when they provide examples to explain what it is. To me, it sound more like reasoning form what already works. In most examples I have seen, it is mostly about solving a problem using existing solutions/packages available and add your own flavor to it. That's why the "analogy" thinking because that's how the idea is presented, people pitch the idea using other's solutions. But what Elon Musk is different from others is that his thinking is stemming from scientific research more than you think it is. It is asking about the right question by digging deep into the core of the problem until you cannot dig any further. For example, in Elon Musk's example of battery, you first start with a goal: how to build an electric cars that is cost effective. You break the goals down into smaller problems, and you could recognize that battery is the biggest contributing factor in providing the solution to this goal, then you ask, why is the cost of battery so high yet it yield so little? Then you break the problem down into further smaller problems such as the cost for each component that comprised the battery, then you keep reasoning from there until you can find a ground where you can start reasoning back up. Personally I think the name "First principle" is also misleading since it is not mainly about principles, but more about asking the right questions.
Elon Musk on First Principles
101–105 of 105 posts
Re: Elon Musk on First Principles
#102The so-called reason by analogy is just gradient descent to approach local optima in optmization. The "by first principle" is just providing an approximate (ideally convex) model and solve it analytically for global optima. The problem is generally very hard (#P hard), both for formulating the problem and solving it. Global optimum of course in theory is better, but the quality of your objective function and constrai…
I agree, but would like to add the aspect of uncertainty. When reasoning from analogy, you usually have good statistical knowledge about the properties of the problem. For example, the targetted product category may exist, and customer behavior is known, and therefore predicting what would happen in some nearby configuration is usually somewhat accurate. On the other hand, in the first principles case you need to hav…
With uncertainty you just change your objective function to an evaluation function of any possible scenario scaled with its probability. It doesn't change how optimization works. Again this formulation is only easy to solve in very limited context. For example in economics people have been working with the oversimplified supply-demand curves precisely because they are usually the dominating factors and in practice a sufficiently accurate model works just fine. This model only gives insights for why different ways of reasoning works, not actually providing the panacea.
The good thing about gradient descent is that you do NOT need to have a model, you just need to focus on a few parameters and figure out what is the direction for best improvement from a current relatively good point, where the other billions of parameters are already accounted for and assumed independent from the direction you are going.
Re: Elon Musk on First Principles
#103Earlier quoted context omitted.
I love this explanation. Do you happen to blog about optimization topics?
If you're asking waht I think you're asking, then what you're then looking for is called 'numerical analysis'. Specifically, the grandparent was describing this [1]. Here is an OCW link for a good primer in to various introductory numerical analysis processes for Engineering [2]. [1]: https://en.wikipedia.org/wiki/Gradient_descent [2]: http://ocw.mit.edu/courses/mechanical-engineering/2-993j-int...
Numerical Analysis is too broad for this topic. The precise category should be:
https://en.wikipedia.org/wiki/Mathematical_optimization
The hierarchy of topics in this area: http://en.wikipedia.org/wiki/Category:Mathematical_optimizat...
This is also why Google is paying millions of dollars for a quantum computer. Being able to solve complex optimization problems efficiently almost partially translates to access to higher intelligence.
http://googleresearch.blogspot.com/2013/05/launching-quantum...
Re: Elon Musk on First Principles
#104Earlier quoted context omitted.
Spot on. Some research to back you up: http://alumni.stanford.edu/get/page/magazine/article/?articl...
Do you know what the article means by: "why some become Muhammad Ali and others Mike Tyson"? I think it's contrasting the success of Ali with the (inferred) lack thereof of Tyson. Maybe? But that doesn't make any sense to me; the first paragraph of Tyson's Wikipedia page will tell you why. Unless it's talking about social success. In which case it seems out of context of the article.
I know very little about boxing, so I'm really not sure. My thought would be that though both were successful, perhaps Ali was an underdog who became successful where Tyson started successful but hit a plateau? I have no idea whether or not that maps to reality, that's just pattern matching. The other possibility is that the author of the article knows the same amount about boxing as I do, and chose an unfortunate analogy.
Re: Elon Musk on First Principles
#105Earlier quoted context omitted.
I agree, but would like to add the aspect of uncertainty. When reasoning from analogy, you usually have good statistical knowledge about the properties of the problem. For example, the targetted product category may exist, and customer behavior is known, and therefore predicting what would happen in some nearby configuration is usually somewhat accurate. On the other hand, in the first principles case you need to hav…
With uncertainty you just change your objective function to an evaluation function of any possible scenario scaled with its probability. It doesn't change how optimization works. Again this formulation is only easy to solve in very limited context. For example in economics people have been working with the oversimplified supply-demand curves precisely because they are usually the dominating factors and in practice a…
It seems you are assuming there are direct observations. Musk is talking about generating hypothetical observations from a model (the cost of the battery is bounded from below by the cost of the battery materials). This sort of bounding does not always work outside physics, because the uncertainties are so ill behaving.