Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Marc Andreessen on the atomization of AI

techcrunch.com

Post reply on HN
Print view View original

41–47 of 47 posts

Re: Marc Andreessen on the atomization of AI

#41

Post by mdonahoe »

kough wrote: post #36
vonnik wrote: post #5

Earlier quoted context omitted.

Vanilla deep learning is so far ahead of what most companies use for analytics, that very little research is needed to offer a product with significant advantages over what's in practice now. In fact, I'd say most ML/DL/AI startups don't need to do research at all. They just need to spread more widely what exists now. I disagree with Andreessen that Google's Tensorflow propaganda on Udacity is "opening the kimono" or…

Yeah, Tensorflow seems cool, but then you benchmark and realize that (with CUDA/CUDNN installed) it's 50% slower than Caffe.

Was google too busy working with their custom asics to notice the speed difference on gpus?

Re: Marc Andreessen on the atomization of AI

#42

Post by outsideline »

daveguy wrote: post #38
outsideline wrote: post #32

Earlier quoted context omitted.

I'm not attempting to be a smart arse. However, I really think you should google it :). It is too common a trend for one to latch on to popular names as if they're the only ones doing anything important in a space. There are many cases whereby they're doing the least important work. I mentioned George Hotz to highlight a capable individual with vision, passion, and capability who 'thinks different'. With many individ…

I'm not attempting to be a smart ass, but you should back up your speculation with facts and references. Is the best research a secret? Your post implied that you have some special insight into research groups that are doing AGI research and are making steady progress. I have done plenty of Google searches and not run across any notable AGI progress. The one reference you provided was for a deep learning implementati…

> I'm not attempting to be a smart ass, but you should back up your speculation with facts and references. Is the best research a secret?

Yes. Given the attitudes of people in the space and beyond and lack of funding why shouldn't the more valuable and fundamental research be kept secret? They're taking all of the risk to pursue something that everyone is saying is impossible. Why would they publish details? If more funding and support would come through maybe they would. Otherwise, they're seemingly developing a fundamental AGI that has the ability to operate under its own free-will and mechanisms. It is being said that it's impossible... Once it's completed no one will be able to deny it at that point. Maybe the money will come then... Maybe the openness will too.

> Your post implied that you have some special insight into research groups that are doing AGI research and are making steady progress. I have done plenty of Google searches and not run across any notable AGI progress.

Yes and I will not disclose where my insights derive from. I'm sure people have their ways of discovering it nonetheless. Some groups and individuals are developing conscious self-willed software solutions. I'm referencing them. Is that not what 'life' which intelligence springs from fundamentally is? As will be true for AGI... Not programs that mimic human behavior or optimization algorithms.. Software solutions that are truly aware.

Re: Marc Andreessen on the atomization of AI

#43

Post by outsideline »

jcannell wrote: post #39
outsideline wrote: post #7

Earlier quoted context omitted.

You're right on many points. My observation is, when it comes to deeper Ai that trends towards AGI, the private sector seems lost in terms of the big ideas and applications. The VCs are also lost and have no idea on how to appreciate fundamentally groundbreaking R&D development ventures. To be honest, most of the funding and deeply interesting work for ground breaking Ai research lies in the public and defense sector…

>You're right on many points. My observation is, when it comes to deeper Ai that trends towards AGI, the private sector seems lost in terms of the big ideas and applications. The VCs are also lost and have no idea on how to appreciate fundamentally groundbreaking R&D development ventures. DeepMind was VC funded before Google acquired them. A number of AI/ML startups would like to pursue AGI in the long-term, but why…

Deep Mind was funded by Elon Musk as well as others in its earlier stages. Elon Musk is one of the individuals who speaks out as if he is wildly fearful of where Ai is going. Fundamentally, an individual has no control over what an AGI development group does unless they're invested in them. There are many notable groups beyond the handful that get the spotlight. They are for instance : working on a Conscious Artificial Intelligence. That is not a 'future goal' of theirs. That is the one and only goal of their efforts. Who do you think will get their first?

>A number of AI/ML startups would like to pursue AGI in the long-term, but why not make revenue along the way? That's exactly the dilemma that many capable researchers faced. Some chose to make money along the way. It shapes the way you then go about solving AGI down the road. It saps up your resources, imagination, and potential and it redirects it to shorter term thinking. Being in the space, there should be an understanding of Short-term vs. long-term thinking based on a reward system that targets short-term success.

> Still, there is at least one startup - Vicarious - focusing purely on long-term research towards AGI that VCs poured plenty of money into, and there are probably others. I'm familiar with Vicarious. You are correct and they notably publish far less details than others. They are funded by top names and are probably the least mentioned when people mention Ai. Why, given the impact that AGI can have, aren't more groups funded? A solution is seemingly right around the corner... Do people not see this?

Re: Marc Andreessen on the atomization of AI

#44

Post by eli_gottlieb »

Xcelerate wrote: post #2

The common saying is "more data usually beats a better algorithm". However, this is only for short term progress. Long term progress in the field of AI clearly requires better algorithms, and doing more with less data is exactly the kind of problem that a startup in the field could solve with a clever idea. That said, these startups will have to be extremely research heavy — how does one draw the most talented resear…

>how does one draw the most talented researchers to work for a small group of people with minimal pay over working for a large company with a high salary and all the computational resources you might need?

You mean how does one draw people to graduate school?

Re: Marc Andreessen on the atomization of AI

#45

Post by eli_gottlieb »

outsideline wrote: post #42
daveguy wrote: post #38

Earlier quoted context omitted.

I'm not attempting to be a smart ass, but you should back up your speculation with facts and references. Is the best research a secret? Your post implied that you have some special insight into research groups that are doing AGI research and are making steady progress. I have done plenty of Google searches and not run across any notable AGI progress. The one reference you provided was for a deep learning implementati…

> I'm not attempting to be a smart ass, but you should back up your speculation with facts and references. Is the best research a secret? Yes. Given the attitudes of people in the space and beyond and lack of funding why shouldn't the more valuable and fundamental research be kept secret? They're taking all of the risk to pursue something that everyone is saying is impossible. Why would they publish details? If more…

>Given the attitudes of people in the space and beyond and lack of funding why shouldn't the more valuable and fundamental research be kept secret?

If it's secret, how do you know about it? If you don't have specific knowledge, how can you claim there is "fundamental AGI" research done in secret, as opposed to it just not being done, as all appearances indicate?

>It is being said that it's impossible...

Nobody who understands recent neuroscience claims any such thing is impossible. We just claim it hasn't been done in the lab yet.

Re: Marc Andreessen on the atomization of AI

#46

Post by eli_gottlieb »

Xcelerate wrote: post #29
WilliamDhalgren wrote: post #27

Earlier quoted context omitted.

Truly doing much with little data may simply be impossible; its not a magical black box after all, but just as ideal a learner as possible - but if small data cannot give it confidence to confirm the existence of some subtile pattern, on what grounds is even an ideal learner supposed to believe in it? For example, no smarts would ever confirm the Higgs in the tevatron in its 10 year run even though the machine produc…

> Truly doing much with little data may simply be impossible; its not a magical black box after all, but just as ideal a learner as possible This is interesting, and something that needs further explored IMO. I've been doing research on extracting mutual information from noisy, shifted copies of a ground truth signal, and it turns out there is a crossover point where recovering the ground truth essentially becomes im…

>I've been doing research on extracting mutual information from noisy, shifted copies of a ground truth signal, and it turns out there is a crossover point where recovering the ground truth essentially becomes impossible (in an information theoretic sense). What's interesting is that this crossover point is sharp and it looks a lot like a phase transition that one might see in statistical physics.

That's awesome! Can you link me to your research? I've been writing a similar paper on the information theory behind probabilistic programming and (possibly) deep learning.

>We need more research that provides limits on what we are capable of predicting from a given dataset — an upper bound, in other words. This would let us know if it's worth it to spend time trying to get more predictivity out of a dataset, or if the data just simply doesn't contain sufficient enough information.

Definitely! Not only is the Shannon entropy of large or high-dimensional datasets computationally obscene to estimate nonparametrically, what we actually care about is the conditional entropy given an untrained model, and the degree to which we can reduce it by training the model.

Re: Marc Andreessen on the atomization of AI

#47

Post by eli_gottlieb »

YeGoblynQueenne wrote: post #20
Xcelerate wrote: post #2

The common saying is "more data usually beats a better algorithm". However, this is only for short term progress. Long term progress in the field of AI clearly requires better algorithms, and doing more with less data is exactly the kind of problem that a startup in the field could solve with a clever idea. That said, these startups will have to be extremely research heavy — how does one draw the most talented resear…

>> doing more with less data is exactly the kind of problem that a startup in the field could solve with a clever idea. It will take much more than a "clever idea" to overcome the reliance on data of the current AI state of the art- that is to say, machine learning. If you think about it, machine learning algorithms are essentially clever search procedures for some optimum in a heap of data (that's optimisation algor…

>What we need is much more than a clever idea: it's a paradigm shift. There's probably some sort of way to make a system that can learn from very few examples and at a very short time, like we do ourselves. Machine learning is certainly not that way. We need to find another way.

That's more like probabilistic methods, which also fight overfitting by regularizing (with a prior distribution) and allocating probability density in a smear around the training data rather than concentrating it as hard as it can (as optimization methods usually do, so to speak). And then hierarchical Bayes is what's used in many of the neat one-shot learning demos, and in theoretical neuroscience these days.

Post reply on HN