Marc Andreessen on the atomization of AI • Hacker News Board

Re: Marc Andreessen on the atomization of AI

#31

Post by outsideline » Wed, Sep 14, 2016, 1:59 PM UTC

dharma1 wrote: post #17

outsideline wrote: post #7

Earlier quoted context omitted.

You're right on many points. My observation is, when it comes to deeper Ai that trends towards AGI, the private sector seems lost in terms of the big ideas and applications. The VCs are also lost and have no idea on how to appreciate fundamentally groundbreaking R&D development ventures. To be honest, most of the funding and deeply interesting work for ground breaking Ai research lies in the public and defense sector…

I agree, but it's the same problem with all basic research. There are no guarantees of results and it can take a long time. So much of it gets done with public resources. In fact much of the deep learning advances we (including Google) enjoy today were done with Canadian (University of Montreal) and European (IDSIA) taxpayers money over the last couple of decades

More should be done to highlight this. The persistent fanboy like culture, among those even in tech, which centers on only a handful of popular corporate names who serve as system integrators after the fact is dishonest and disingenuous in my opinion.

Given that the public funding of basic research most always results in the ushering in of new technological paradigms that make corporations tens of billions if not hundreds of billions of dollars, more should be done to make sure they pay back into the system that ensures their constant success.

Re: Marc Andreessen on the atomization of AI

#32

Post by outsideline » Wed, Sep 14, 2016, 2:11 PM UTC

daveguy wrote: post #28

outsideline wrote: post #12

Earlier quoted context omitted.

Yes, do you know of the number of groups who are actively developing AGI? who have made steady progress over the years? I see no big names invested in them. Why would someone try to create your own group having no background in development of AGI? Seems a bit off the mark no? Most AGI focused ventures are not even in the U.S. A prominent one is in China. Another is in Europe. The ones in the U.S are privately funded…

>do you know of the number of groups who are actively developing AGI? who have made steady progress over the years? I do not. It seems like researchers who call themselves AGI researchers have made no more significant progress than researchers developing specific analysis techniques. Do you have a list of these groups? I would be very interested to read about their approach and progress. On the human brain initiative…

I'm not attempting to be a smart arse. However, I really think you should google it :). It is too common a trend for one to latch on to popular names as if they're the only ones doing anything important in a space. There are many cases whereby they're doing the least important work.

I mentioned George Hotz to highlight a capable individual with vision, passion, and capability who 'thinks different'. With many individuals, the capability is there. However, often times the vision, passion, or capability to think different isn't. Funny then that there becomes this hiring norm which attempts to find the 'most capable' individual and ignores the other more important characteristics.

Time will tell. I'm personally looking elsewhere beyond the names everyone mentions when it comes to development in this space.

Re: Marc Andreessen on the atomization of AI

#33

Post by outsideline » Wed, Sep 14, 2016, 2:22 PM UTC

thr0waway1239 wrote: post #13

I was thinking about this notion of using simulated worlds for doing training. An open source simulated world would be fantastic - to the extent that it is actually realistic. For example, maybe collect archived gameplay from some MMORPG and use it to build the simulated world. As new players play in the game, the gameplay data is collected and available for the community to use. Obviously this is not well thought ou…

There are several groups that have already done this and there are several tools similar to OpenAI gym. Why is only one company mentioned when it comes to this? You do understand that a large chunk of OpenAI gym are open source software packages like physics engines, box2d?

Here's what you need to make an Ai Gym :

> Physics engine (pick your open source package)

> Simulator (pick your open source package)

> Write a thin wrapper around it

It can be done and has been done by many individuals in less than a week or two of dedicated work.

Re: Marc Andreessen on the atomization of AI

#34

Post by jandrewrogers » Wed, Sep 14, 2016, 2:55 PM UTC

Xcelerate wrote: post #29

WilliamDhalgren wrote: post #27

Earlier quoted context omitted.

Truly doing much with little data may simply be impossible; its not a magical black box after all, but just as ideal a learner as possible - but if small data cannot give it confidence to confirm the existence of some subtile pattern, on what grounds is even an ideal learner supposed to believe in it? For example, no smarts would ever confirm the Higgs in the tevatron in its 10 year run even though the machine produc…

> Truly doing much with little data may simply be impossible; its not a magical black box after all, but just as ideal a learner as possible This is interesting, and something that needs further explored IMO. I've been doing research on extracting mutual information from noisy, shifted copies of a ground truth signal, and it turns out there is a crossover point where recovering the ground truth essentially becomes im…

The concept of an upper bound on prediction for a given data set does exist, I've seen it discussed explicitly in some mid-1990s universal sequence prediction literature. I haven't seen the term (or concept) used in modern predictive algorithm literature but I believe the term of art is "prediction error complexity" which is conceptually similar to concepts like Kolmogorov complexity in algorithmic information theory.

Re: Marc Andreessen on the atomization of AI

#35

Post by jcannell » Wed, Sep 14, 2016, 3:37 PM UTC

This notion that AI/ML requires tons of data that only big tech companies have is at least partly a myth.

Big private datasets are important only in narrow domains - if you want to train an ANN to do ad prediction or something in medical imaging, sure.

But longer term and for the most valuable applications, the required data is plentiful and free. The data advanced robots (ex. self-driving cars) and AGI need to learn is all around us and it is free. Progress here is limited by compute/algorithms, not data.

Re: Marc Andreessen on the atomization of AI

#36

Post by kough » Wed, Sep 14, 2016, 3:38 PM UTC

vonnik wrote: post #5

Xcelerate wrote: post #2

The common saying is "more data usually beats a better algorithm". However, this is only for short term progress. Long term progress in the field of AI clearly requires better algorithms, and doing more with less data is exactly the kind of problem that a startup in the field could solve with a clever idea. That said, these startups will have to be extremely research heavy — how does one draw the most talented resear…

Vanilla deep learning is so far ahead of what most companies use for analytics, that very little research is needed to offer a product with significant advantages over what's in practice now. In fact, I'd say most ML/DL/AI startups don't need to do research at all. They just need to spread more widely what exists now. I disagree with Andreessen that Google's Tensorflow propaganda on Udacity is "opening the kimono" or…

Yeah, Tensorflow seems cool, but then you benchmark and realize that (with CUDA/CUDNN installed) it's 50% slower than Caffe.

Re: Marc Andreessen on the atomization of AI

#37

Post by [deleted] » Wed, Sep 14, 2016, 3:58 PM UTC

outsideline wrote: post #7

Xcelerate wrote: post #2

The common saying is "more data usually beats a better algorithm". However, this is only for short term progress. Long term progress in the field of AI clearly requires better algorithms, and doing more with less data is exactly the kind of problem that a startup in the field could solve with a clever idea. That said, these startups will have to be extremely research heavy — how does one draw the most talented resear…

You're right on many points. My observation is, when it comes to deeper Ai that trends towards AGI, the private sector seems lost in terms of the big ideas and applications. The VCs are also lost and have no idea on how to appreciate fundamentally groundbreaking R&D development ventures. To be honest, most of the funding and deeply interesting work for ground breaking Ai research lies in the public and defense sector…

[deleted]

Re: Marc Andreessen on the atomization of AI

#38

Post by daveguy » Wed, Sep 14, 2016, 4:13 PM UTC

outsideline wrote: post #32

daveguy wrote: post #28

Earlier quoted context omitted.

>do you know of the number of groups who are actively developing AGI? who have made steady progress over the years? I do not. It seems like researchers who call themselves AGI researchers have made no more significant progress than researchers developing specific analysis techniques. Do you have a list of these groups? I would be very interested to read about their approach and progress. On the human brain initiative…

I'm not attempting to be a smart arse. However, I really think you should google it :). It is too common a trend for one to latch on to popular names as if they're the only ones doing anything important in a space. There are many cases whereby they're doing the least important work. I mentioned George Hotz to highlight a capable individual with vision, passion, and capability who 'thinks different'. With many individ…

I'm not attempting to be a smart ass, but you should back up your speculation with facts and references. Is the best research a secret?

Your post implied that you have some special insight into research groups that are doing AGI research and are making steady progress. I have done plenty of Google searches and not run across any notable AGI progress.

The one reference you provided was for a deep learning implementation, which is what you seemed to be dismissing.

Re: Marc Andreessen on the atomization of AI

#39

Post by jcannell » Wed, Sep 14, 2016, 4:23 PM UTC

outsideline wrote: post #7

Xcelerate wrote: post #2

The common saying is "more data usually beats a better algorithm". However, this is only for short term progress. Long term progress in the field of AI clearly requires better algorithms, and doing more with less data is exactly the kind of problem that a startup in the field could solve with a clever idea. That said, these startups will have to be extremely research heavy — how does one draw the most talented resear…

You're right on many points. My observation is, when it comes to deeper Ai that trends towards AGI, the private sector seems lost in terms of the big ideas and applications. The VCs are also lost and have no idea on how to appreciate fundamentally groundbreaking R&D development ventures. To be honest, most of the funding and deeply interesting work for ground breaking Ai research lies in the public and defense sector…

>You're right on many points. My observation is, when it comes to deeper Ai that trends towards AGI, the private sector seems lost in terms of the big ideas and applications. The VCs are also lost and have no idea on how to appreciate fundamentally groundbreaking R&D development ventures.

DeepMind was VC funded before Google acquired them. A number of AI/ML startups would like to pursue AGI in the long-term, but why not make revenue along the way? Still, there is at least one startup - Vicarious - focusing purely on long-term research towards AGI that VCs poured plenty of money into, and there are probably others.

Re: Marc Andreessen on the atomization of AI

#40

Post by thr0waway1239 » Wed, Sep 14, 2016, 4:51 PM UTC

outsideline wrote: post #33

thr0waway1239 wrote: post #13

I was thinking about this notion of using simulated worlds for doing training. An open source simulated world would be fantastic - to the extent that it is actually realistic. For example, maybe collect archived gameplay from some MMORPG and use it to build the simulated world. As new players play in the game, the gameplay data is collected and available for the community to use. Obviously this is not well thought ou…

There are several groups that have already done this and there are several tools similar to OpenAI gym. Why is only one company mentioned when it comes to this? You do understand that a large chunk of OpenAI gym are open source software packages like physics engines, box2d? Here's what you need to make an Ai Gym : > Physics engine (pick your open source package) > Simulator (pick your open source package) > Write a t…

Since I honestly don't know enough about this topic, it would be nice if you can mention the other companies. I just mention OpenAI because they are a prominent organization which happen to have both open and AI in their name.

Maybe I am also not being very clear, but I am actually visualizing a game which is popular enough to attract a lot of gameplay, but has gameplay elements which are actually the kinds of things which the AI problems are trying to solve. E.g. imagine capturing gameplay from a popular multiplayer drone game to improve drone navigation.

Drones apparently don't fly very well autonomously.

http://spectrum.ieee.org/automaton/robotics/drones/skydio-ca...

Now, imagine if it were possible to get open data where any researcher or group could work on this problem using a gameplay dataset. Perhaps someone with an unexpected take on the problem could make some important contribution?