Live data from Hacker News

Marc Andreessen on the atomization of AI

techcrunch.com

11–20 of 47 posts

Re: Marc Andreessen on the atomization of AI

#11
post #8

Earlier quoted context omitted.

You're right on many points. My observation is, when it comes to deeper Ai that trends towards AGI, the private sector seems lost in terms of the big ideas and applications. The VCs are also lost and have no idea on how to appreciate fundamentally groundbreaking R&D development ventures. To be honest, most of the funding and deeply interesting work for ground breaking Ai research lies in the public and defense sector…

> Ultimately I find this a bit ridiculous given how vocal some individuals in the tech field are about the 'dangers of Ai'. They harp and harp about what could happen if more capable Ai gets developed but you don't see their money anywhere near the groups doing deeper development in Ai. ...because if you think AGI is dangerous, you should fund groups trying to develop it? I am puzzled by the presentation of this prop…

Yep. "The best way to predict the future is to create it"

Don't stretch your mind to far on this one. If someone or various groups develop AGI and you have no stake in them, you will have no say in how it gets used. For once, people need to be honest with themselves about this.

You're not going to be able to dictate from a third party consortium of PhDs how someone should run their company or develop their software.

Re: Marc Andreessen on the atomization of AI

#12
post #9

Earlier quoted context omitted.

You're right on many points. My observation is, when it comes to deeper Ai that trends towards AGI, the private sector seems lost in terms of the big ideas and applications. The VCs are also lost and have no idea on how to appreciate fundamentally groundbreaking R&D development ventures. To be honest, most of the funding and deeply interesting work for ground breaking Ai research lies in the public and defense sector…

Have you heard of the OpenAI initiative?

Yes, do you know of the number of groups who are actively developing AGI? who have made steady progress over the years? I see no big names invested in them. Why would someone try to create your own group having no background in development of AGI? Seems a bit off the mark no?

Most AGI focused ventures are not even in the U.S. A prominent one is in China. Another is in Europe. The ones in the U.S are privately funded beyond the big names. Furthermore, where is the U.S's equivalent of : https://www.humanbrainproject.eu/ ?

Where are the seasoned software engineers in the ranks at these highly funded silicon valley Ai companies? I typically see a roster of 'big names' and PhDs... Whereas, I look at the companies specifically pursuing AGI and I see a whole range of individuals.. PHD neuroscience, Senior game developer, Robotics Engineer, Senior firmware developer, Guy from down the hall who can code circles around silicon valley's most decorated engineer, etc etc.

Look at the Geohot story, that's the kind of individual and company who ushers in a new paradigm ....

If the solution requires you to 'think different', how do you expect to achieve it by padding your company with a bunch of individuals who are all centered on the same techniques from academia?

Re: Marc Andreessen on the atomization of AI

#13
I was thinking about this notion of using simulated worlds for doing training. An open source simulated world would be fantastic - to the extent that it is actually realistic. For example, maybe collect archived gameplay from some MMORPG and use it to build the simulated world. As new players play in the game, the gameplay data is collected and available for the community to use. Obviously this is not well thought out, but it would be an excellent candidate for a group like OpenAI, assuming they aren't already building something like that.

I wonder if it will be a chicken and egg problem though? Maybe you cannot even create decent gameplay without already having a trove of data, but bad gameplay will not attract more data from potential players?

Re: Marc Andreessen on the atomization of AI

#14
post #3

I think your second question is the biggest thing. If you're developing some groundbreaking technology or doing heavy research you face a bunch of gnarly questions. How to convince investors when the climate favors revenue and customers over technological promise? How to attract users or customers to some subset of what you're doing while you build the larger thing? How to explain to everyone what you're doing if, as…

Not everyone has 'vision' and its hard convincing those who don't have it to see the potential in your idea before its fully formed. In cases of fundamental technology like Ai, there should be a national effort to fund individuals and people who want to passionately pursue it. Europe has been on top of the ball when it comes to this lately.

Thankfully we live in an age in which the most important research to developing fundamental things like Ai is 'time'. If you have an internet connection and a reasonably decent computer, you have enough resources to create AGi.

If you passionately care about seeing your idea through, save up money, move to a cheaper location, wire your computer to the internet, and get to work.

Once you've proven your vision, who can deny you then? Slam it on the boardroom table, disrupt present day business models, and be the next tech giant.

Re: Marc Andreessen on the atomization of AI

#15

The common saying is "more data usually beats a better algorithm". However, this is only for short term progress. Long term progress in the field of AI clearly requires better algorithms, and doing more with less data is exactly the kind of problem that a startup in the field could solve with a clever idea. That said, these startups will have to be extremely research heavy — how does one draw the most talented resear…

You're right on many points. My observation is, when it comes to deeper Ai that trends towards AGI, the private sector seems lost in terms of the big ideas and applications. The VCs are also lost and have no idea on how to appreciate fundamentally groundbreaking R&D development ventures. To be honest, most of the funding and deeply interesting work for ground breaking Ai research lies in the public and defense sector…

>> outlining a forward thinking application of Ai.

As someone miles away from the field, what are those forward thinking applications (ie things that are within the reach of MI today but are basically ignored by private sector)

Re: Marc Andreessen on the atomization of AI

#16

The common saying is "more data usually beats a better algorithm". However, this is only for short term progress. Long term progress in the field of AI clearly requires better algorithms, and doing more with less data is exactly the kind of problem that a startup in the field could solve with a clever idea. That said, these startups will have to be extremely research heavy — how does one draw the most talented resear…

> how does one draw the most talented researchers to work for a small group of people with minimal pay over working for a large company with a high salary and all the computational resources you might need?

Offer them a PhD

Re: Marc Andreessen on the atomization of AI

#17

The common saying is "more data usually beats a better algorithm". However, this is only for short term progress. Long term progress in the field of AI clearly requires better algorithms, and doing more with less data is exactly the kind of problem that a startup in the field could solve with a clever idea. That said, these startups will have to be extremely research heavy — how does one draw the most talented resear…

You're right on many points. My observation is, when it comes to deeper Ai that trends towards AGI, the private sector seems lost in terms of the big ideas and applications. The VCs are also lost and have no idea on how to appreciate fundamentally groundbreaking R&D development ventures. To be honest, most of the funding and deeply interesting work for ground breaking Ai research lies in the public and defense sector…

I agree, but it's the same problem with all basic research. There are no guarantees of results and it can take a long time. So much of it gets done with public resources.

In fact much of the deep learning advances we (including Google) enjoy today were done with Canadian (University of Montreal) and European (IDSIA) taxpayers money over the last couple of decades

Re: Marc Andreessen on the atomization of AI

#18

The common saying is "more data usually beats a better algorithm". However, this is only for short term progress. Long term progress in the field of AI clearly requires better algorithms, and doing more with less data is exactly the kind of problem that a startup in the field could solve with a clever idea. That said, these startups will have to be extremely research heavy — how does one draw the most talented resear…

> how does one draw the most talented researchers to work for a small group of people with minimal pay over working for a large company with a high salary and all the computational resources you might need? And what kind of product could theoretical research produce in the short term that would keep a startup afloat long enough to come up with ground breaking ideas?

I think the short answer is, you don't. Between Google, OpenAI, Facebook, Baidu, Microsoft, IBM, and every other startup that is putting "deep learning" in their pitch deck to see what happens, there's enough money being pumped into the area. 90% of the results of that money being spent will be made available to everyone in the world for free. IE, if you're investing for competitive advantage, you will lose. If you're not already on one of those teams or studying the subject, I don't see any opportunity for at least a year, maybe 2.

To be clear, I'm talking about opportunities around starting a new company. If your business is consulting or you're working as an engineer for an existing company, there are many applications for deep learning, but none that stand on their own.

In 2 years, content generation will be a big thing. Like pandora with purely algorithmically generated music, or games with assets that create themselves in a way far superior to what we see in the current generation of "procedural generation". The big cos will have very good versions of Siri, but no startup is going to be able to touch that near term.

Re: Marc Andreessen on the atomization of AI

#19

I was thinking about this notion of using simulated worlds for doing training. An open source simulated world would be fantastic - to the extent that it is actually realistic. For example, maybe collect archived gameplay from some MMORPG and use it to build the simulated world. As new players play in the game, the gameplay data is collected and available for the community to use. Obviously this is not well thought ou…

https://gym.openai.com

Just needs integration with popular game engines like UE and unity3d, at the moment it mostly works with 8bit game emulators

Re: Marc Andreessen on the atomization of AI

#20

The common saying is "more data usually beats a better algorithm". However, this is only for short term progress. Long term progress in the field of AI clearly requires better algorithms, and doing more with less data is exactly the kind of problem that a startup in the field could solve with a clever idea. That said, these startups will have to be extremely research heavy — how does one draw the most talented resear…

>> doing more with less data is exactly the kind of problem that a startup in the field could solve with a clever idea.

It will take much more than a "clever idea" to overcome the reliance on data of the current AI state of the art- that is to say, machine learning.

If you think about it, machine learning algorithms are essentially clever search procedures for some optimum in a heap of data (that's optimisation algorithms, so about 90% of the field). You can always get smarter in the way you search, but unless you search far enough what you find won't generalise very well to unseen data. And if "unseen data" is the real world, you need immense amounts of data to get anywhere- hence, the reliance on big data.

What we need is much more than a clever idea: it's a paradigm shift. There's probably some sort of way to make a system that can learn from very few examples and at a very short time, like we do ourselves. Machine learning is certainly not that way. We need to find another way.

I'm not sure startups with clever ideas have a very good chance to just stumble on such an other way while looking for a way to do "Facebook with deep learning".

Post reply on HN