Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Andreessen-Horowitz craps on “AI” startups from a great height

scottlocklin.wordpress.com

Post reply on HN
Print view View original

21–30 of 256 posts

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#21

Post by icheishvili »

harias wrote: post #5

>That’s right; that’s why a lone wolf like me, or a small team can do as good or better a job than some firm with 100x the head count and 100m in VC backing. goes on to say >I agree, but the hockey stick required for VC backing, and the army of Ph.D.s required to make it work doesn’t really mix well with those limited domains, which have a limited market. Choose one? Also assumes running your own data center to be ea…

I don't think these are necessarily contradictory. With pytorch-transformers, you can use a full-blown BERT model like the best in the world. And yet, to make this novel and defensible, you would need to build on top of it and innovate significantly, which would require significant capital to achieve.

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#22

Post by mereel »

alephnan wrote: post #18
m0zg wrote: post #8

"Huge compute bills" usually come from training, or to be more precise, hyperparameter search that's required before you find a model that works well. You could also fail to find such a model, but that's another discussion. So yeah, you could spend one or two FTE salaries' (or one deep learning PhD's) worth of cash on finding such models for your startup if you insist on helping Jeff Bezos to wipe his tears with cris…

> Just don't call it a "datacenter" or NVIDIA will have a stroke. Context please :) ?

Just a guess but maybe it's some licensing issue? https://www.nvidia.com/en-us/drivers/geforce-license/

No Datacenter Deployment. The SOFTWARE is not licensed for datacenter deployment, except that blockchain processing in a datacenter is permitted.

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#23

Post by ThePadawan »

alephnan wrote: post #18
m0zg wrote: post #8

"Huge compute bills" usually come from training, or to be more precise, hyperparameter search that's required before you find a model that works well. You could also fail to find such a model, but that's another discussion. So yeah, you could spend one or two FTE salaries' (or one deep learning PhD's) worth of cash on finding such models for your startup if you insist on helping Jeff Bezos to wipe his tears with cris…

> Just don't call it a "datacenter" or NVIDIA will have a stroke. Context please :) ?

Datacenter GPUs are mostly identical to the much cheaper consumer versions. The only thing preventing you from running a datacenter with consumer hardware is the licensing agreement you accept.

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#25

Post by OkGoDoIt »

alephnan wrote: post #18
m0zg wrote: post #8

"Huge compute bills" usually come from training, or to be more precise, hyperparameter search that's required before you find a model that works well. You could also fail to find such a model, but that's another discussion. So yeah, you could spend one or two FTE salaries' (or one deep learning PhD's) worth of cash on finding such models for your startup if you insist on helping Jeff Bezos to wipe his tears with cris…

> Just don't call it a "datacenter" or NVIDIA will have a stroke. Context please :) ?

NVIDIA forces you to buy significantly more expensive cards that perform marginally better if you are using them for datacenter use. They try to enforce not letting businesses use consumer grade gaming cards. I assume this is so cloud providers don't buy up all the supply of graphics cards and make it hard for gamers to get decent cards, like what happened during the bitcoin craze.

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#26

Post by ativzzz »

I agree with the author's opinion about

> I’ll go out on a limb and assert that most of the up front data pipelining and organizational changes which allow for it are probably more valuable than the actual machine learning piece.

Especially at non-tech companies with outdated internal technology. I've consulted at one of these and the biggest wins from the project (I left before the whole thing finished unfortunately) were overall improvements to the internal data pipeline, such as standardization and consolidation of similar or identical data from different business units.

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#27

Post by [deleted] »

alephnan wrote: post #18
m0zg wrote: post #8

"Huge compute bills" usually come from training, or to be more precise, hyperparameter search that's required before you find a model that works well. You could also fail to find such a model, but that's another discussion. So yeah, you could spend one or two FTE salaries' (or one deep learning PhD's) worth of cash on finding such models for your startup if you insist on helping Jeff Bezos to wipe his tears with cris…

> Just don't call it a "datacenter" or NVIDIA will have a stroke. Context please :) ?

[deleted]

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#28

Post by [deleted] »

alephnan wrote: post #18
m0zg wrote: post #8

"Huge compute bills" usually come from training, or to be more precise, hyperparameter search that's required before you find a model that works well. You could also fail to find such a model, but that's another discussion. So yeah, you could spend one or two FTE salaries' (or one deep learning PhD's) worth of cash on finding such models for your startup if you insist on helping Jeff Bezos to wipe his tears with cris…

> Just don't call it a "datacenter" or NVIDIA will have a stroke. Context please :) ?

[deleted]

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#29

Post by bob1029 »

m0zg wrote: post #8

"Huge compute bills" usually come from training, or to be more precise, hyperparameter search that's required before you find a model that works well. You could also fail to find such a model, but that's another discussion. So yeah, you could spend one or two FTE salaries' (or one deep learning PhD's) worth of cash on finding such models for your startup if you insist on helping Jeff Bezos to wipe his tears with cris…

I find it fun how the cost of the cloud is forcing people to consider what absolutely must run in the cloud (presumably for stability and compliance reasons) and what can be brought back on-prem.

We don't train ML models, but we are in a similar boat regarding cloud compute costs. Building our solutions for our clients is a compute-heavy task which is getting expensive in the cloud. We are considering options such as building commodity threadripper rigs, throwing them in various developers' (home) offices, installing a VPN client on each and then attaching as build agents to our AWS-hosted jenkins instance. In this configuration we could drop down to a t3a.micro for Jenkins and still see much faster builds. The reduction in iteration time over a month would easily pay for the new hardware. An obvious next step up from this is to do proper colocation, but I am of a mindset that if I have to start racking servers I am bringing 100% of our infrastructure out of the cloud.

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#30

Post by jjeaff »

harias wrote: post #5

>That’s right; that’s why a lone wolf like me, or a small team can do as good or better a job than some firm with 100x the head count and 100m in VC backing. goes on to say >I agree, but the hockey stick required for VC backing, and the army of Ph.D.s required to make it work doesn’t really mix well with those limited domains, which have a limited market. Choose one? Also assumes running your own data center to be ea…

>rare 10 minute peaks

But is that really the use case here? I haven't worked in ML. But I'm not seeing where you are going to need to handle a 10 minute spike that requires a whole datacenter.

A month's worth of a quad gpu instance on AWS could pay for a server with similar capacity in a few months of usage.

And hardware is pretty resilient these days. Especially if you co-locate it in a datacenter that handles all the internet and power up time for you. And when something does go wrong, they offer "magic hands" service to go swap out hardware for you. Colocation is surprisingly cheap. As is leasing 'managed' equipment.

Post reply on HN