111–120 of 123 posts
Earlier quoted context omitted.
> In a few years we will have an accurate digital twin of the world No, you will not, unless you redefine what "accurate" means.
Google has 3D models of cities nowadays. 20years ago we only had 2D maps. Why do you think this trend will not continue?
Earlier quoted context omitted.
I’m also skeptical, but after reading the explanation above, I am intrigued. Say I have a cube with 100 x 100 x 100 mesh cells inside, and ports on opposing faces. Given enough time, I can literally run through every possible combination of PEC and air for every cell and solve the FD form of maxwells equations, then save the results. Now, a user can ask my solver for any of those cases, and I simply pull the presolve…
>> But the beauty of neural networks is that they can be very good at generalizing from a partial sample of the problem space. That is really not the case. Neural nets generalise very poorly, hence the need for ever larger amounts of data: to overcome their lack of generalisation by attempting to cover as many "cases" as possible. Edit: when this subject comes up I cite the following article, by François Chollet, mai…
>the mapping from inputs to outputs performed by deep nets quickly stops making sense if new inputs differ even slightly from what they saw at training time
In my experience that isn't really true, if you have an appropriately designed net, training data which appropriately samples the problem space, and the net is not overtrained (overfit).
You can think of training data as representing points in high dimensional space. Like any interpolation problem, if you sample the space with the right density, you can get accurate interpolation results - and neural nets have another huge advantage, in that they learn highly nonlinear interpolation in these high d spaces. So the net may be unlikely to generalize to points outside of the sampled space - although now that I think of it I'm not sure of how nets handle extrapolation - but when you're dealing with space with thousands of dimensions (like each pixel in an image) you can still derive a ton of utility from the interpolation which effectively replaces hardcoded rules about the problem you're solving.
Earlier quoted context omitted.
The hype only started in 2015ish, how many years could you have been waiting for it to collapse?
Watson has been an IBM hype-brand for a decade. It is finally getting to the point that it can reliably achieve some parts of what I was sold on back in 2012.
Earlier quoted context omitted.
>Care to share some examples? Unfortunately, the only example I know of is confidential, with a provisional patent filed. But what I can tell you is that we are already solving non-trivial problems some 3-6 orders of magnitude faster than FDM/FEM. Not quite navier-stokes tier, but still in the realm of PDEs. Sorry I can't be more specific, such is the nature of cutting edge industry, as I'm sure you understand!
"Nullius in verba" - I'll believe it when I see it. And not sure I see the point in patenting such a thing. If it's as good as you say, in less than six months a research group in some country where US patents aren't held in very high regard will have duplicated the work and either sell it as a service or publish it in the scientific literature. They might not even mention knowing about the patent at all! Things have…
US patents can be used to prevent the sale or import of violating foreign products into the US market. It's still the largest market and US patents are still the lynchpin of any patent portfolio.
Simultaneous discovery does occur and primacy is difficult to prove. Most patent offices had first-to-file as the deciding rule, the USPTO adopted first-to-file a few years back to cut down on endless litigation.
Earlier quoted context omitted.
>> But the beauty of neural networks is that they can be very good at generalizing from a partial sample of the problem space. That is really not the case. Neural nets generalise very poorly, hence the need for ever larger amounts of data: to overcome their lack of generalisation by attempting to cover as many "cases" as possible. Edit: when this subject comes up I cite the following article, by François Chollet, mai…
Well...I think that take is a little overly cynical, and I disagree particularly with this: >the mapping from inputs to outputs performed by deep nets quickly stops making sense if new inputs differ even slightly from what they saw at training time In my experience that isn't really true, if you have an appropriately designed net, training data which appropriately samples the problem space, and the net is not overtra…
The actual problem is that this is very difficult, if not impossible, to know before training begins. Most of the time, the best that can be achieved is to train a model on whatever data you have and then painstakingly test it at length and at some cost, on the real-world inputs the trained model has to operate on.
Basically, it's very hard to know your sampling error.
Regarding interpolation and dense sampling etc, the larger the dimensionality of the problem the harder it gets to ensure your data is "dense", let alone that it covers an adequate region of the instance space. For example, the pixels in one image are a tiny, tiny subset of all pixels in all possible images- which is what you really want to represent. Come to that, the pixels in many hundred thousands of images are still a tiny, tiny subset of all pixels in all possible images. I find Chollet's criticism not cynical, but pragmatic and very useful. It's important to understand the limitations of whatever tool you're using.
>> although now that I think of it I'm not sure of how nets handle extrapolation
They don't. It's the gradient optimisation. Gets stuck to local minima, always has, always will. Maybe a new training method will come along at some point. Until then don't expect exrapolation.
Earlier quoted context omitted.
It doesn’t need to generalize, just do sophisticated interpolation. Basing the results on a dense sampling of the input space is exactly what I was suggesting.
Apologies for the misunderstanding. You said "generalizing from a partial sample of the problem space" and I thought you meant generalisation to unseen data from few examples, which is generally what we would all like to get from machine learnig models (but don't). But, if a neural net can't _extrapolate_ to unseen instances, I don't see how it can solve problems like the one you describe with any useful precision, a…
In the example I gave, everyone agrees that if you had long enough and enough processing power, you could solve every possible configuration, and store the results. Then you could instantaneously "solve" any problem.
Unfortunately, the problem I describe is a toy problem (too simple to be useful), and yet it would still take way way too long to solve all the possible configurations.
What if you solved some tiny fraction of the configurations though? That would be a sampling of the configuration space. Then a neural network could use that sampling to interpolate to the cases not solved. That would provide a significant speedup over actually solving the problem.
So the real question is what density you need to pre-solve the configuration space to make it work? It definitely depends on what accuracy you need in the solution, as well as how good you can do with the interpolation. If I said previously that gigantic numbers of examples are not needed, then I misspoke. I am sure they would be needed. Gigantic is vague though - is it the kind of number that can be rented from AWS, or is it the kind of number that would require civilization resources?
I have no idea if the math actually works out to make it a useful approach. All I am saying is that conceptually I can see that in some cases, it could be possible.
Earlier quoted context omitted.
Google has 3D models of cities nowadays. 20years ago we only had 2D maps. Why do you think this trend will not continue?
Collecting more data scales far better than processing more data into information. That's one bottleneck right there.
Getting a picture from the tip of a wind turbine blade is done with a drone in the best case, or a rope access technician in the worst (legacy) case. Analyzing this picture with AI to identify anomalies takes a few seconds. That’s one of my company’s use cases.
In the same way, ask the google maps team about what scales better: flying real airplanes to capture photographic data, or just processing this data to get 3D models...
Edit: typo
Earlier quoted context omitted.
> In a few years we will have an accurate digital twin of the world No, you will not, unless you redefine what "accurate" means.
Google has 3D models of cities nowadays. 20years ago we only had 2D maps. Why do you think this trend will not continue?
Earlier quoted context omitted.
I don't disagree that translation is far from mastered, but bear in mind that Google Translate isn't state of the art, mostly because of computational constraints, and 2016 GTranslate was even worse.
No true Scotsman? What is , then, the state of the art in 2020 machine translation?
[1] https://www.deepl.com/translator
[2] https://ai.googleblog.com/2019/10/exploring-massively-multil...
Earlier quoted context omitted.
Google has 3D models of cities nowadays. 20years ago we only had 2D maps. Why do you think this trend will not continue?
Google Street View is full of artifacts, it's not even close to being accurate. The same goes for satellite imagery of rugged mountains. I'm not even mentioning the vegetation, snow cover, river levels, etc.
For power plant critical structures we want 0.5mm, updated every 6months. For forest management we want 5m, updated every 2 years.
But this increase in spatial and temporal accuracy will keep on going. At some point in the future a small swarm of insect-sized drones will be able to capture a whole forest in a day for a super low cost. And a few people walking with basic smartphones for will be enough to map a whole city.
In 1980 you would have said that google maps 3D and google street view would never exist...