Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Andreessen-Horowitz craps on “AI” startups from a great height

scottlocklin.wordpress.com

Post reply on HN
Print view View original

41–50 of 256 posts

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#41

Post by KaiserPro »

ignoramous wrote: post #36
m0zg wrote: post #8

"Huge compute bills" usually come from training, or to be more precise, hyperparameter search that's required before you find a model that works well. You could also fail to find such a model, but that's another discussion. So yeah, you could spend one or two FTE salaries' (or one deep learning PhD's) worth of cash on finding such models for your startup if you insist on helping Jeff Bezos to wipe his tears with cris…

As someone hoping to build a world-wide footprint, say 25 to 50 DCs, of servers to deploy to with unmetered bandwidth, what are some alternatives to the usual suspects? I have come across fly.io, Vultr, Scaleway, Stackpath, Hetzner, and OVH but either they are expensive (in that they charge for bandwidth and uptime) or do not have a wide enough foot-print. I guess colos are the way to go, but how does one work with c…

> As someone hoping to build a world-wide footprint

Does adding an extra 100ms to the response time cost you that much business wise?

As for colos, it depends on scale. If you have 30k servers world wide, it pays to have someone manage the contracts for you. If not it pays to go for the painful arseholes like vodaphone, or whoever bought Cable & wireless's stuff.

as for security, it gets very difficult. You need to make sure that each machine is actually running _what_ you told it, and know if someone has inserted a hypervisor shim between you and your bare metal.

none of that is off the shelf.

Which is why people pay the big boys, so that they can prove chain of custody and have very big locks on the cages.

K8s gives you scheduling and a datastore. For a large globally distributed system its going to scale like treacle.

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#42

Post by mam2 »

mereel wrote: post #22
alephnan wrote: post #18

Earlier quoted context omitted.

> Just don't call it a "datacenter" or NVIDIA will have a stroke. Context please :) ?

Just a guess but maybe it's some licensing issue? https://www.nvidia.com/en-us/drivers/geforce-license/ No Datacenter Deployment. The SOFTWARE is not licensed for datacenter deployment, except that blockchain processing in a datacenter is permitted.

well they are the one writing the rules, so i'd side with OP

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#43

Post by KaiserPro »

ThePadawan wrote: post #23
alephnan wrote: post #18

Earlier quoted context omitted.

> Just don't call it a "datacenter" or NVIDIA will have a stroke. Context please :) ?

Datacenter GPUs are mostly identical to the much cheaper consumer versions. The only thing preventing you from running a datacenter with consumer hardware is the licensing agreement you accept.

And the cooling, amount of ram and the doubles performance.

the chip might be the same, but the rest of it isn't

Granted, its not worth the $3k price bump, but thats a different issue.

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#44

Post by KaiserPro »

bryanrasmussen wrote: post #40

Generally the use of the phrase from a great height implies the height is one of morality, intellect, or valor (each of these decreasing in usage), I'm not exactly sure what the great height Andreessen-Horowitz craps from is composed of - maybe money? I think they may just be crapping on them from a reasonable vantage point.

The height is not really about morals. Its more about the blast radius of the shit.

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#45

Post by scottlocklin »

allovernow wrote: post #35

All of this might be true currently, but that's because this current first generation "AI" (technically should just be called ML) is mostly bullshit. To clarify, I don't mean anyone is lying or selling snake oil - what I mean by bullshit is that the vast majority of these services are cooked up by software developers without any background in mathematics, selling adtechy services in domains like product recommendatio…

>The next generation of Machine Learning is just emerging, and looks nothing like this. Funds are being raised, patents are being filed, and everything is in early stage development, so you probably haven't heard much yet ...

Citations needed. Large claims: presumably you can name one example of this, and hopefully it's not a company you work at.

I've seen projects on literally all the things you mention: materials science, medical stuff, geology/prospecting -none of them worked well enough to build a stand alone business around them. I do know the oil companies are using DL ideas with some small successes, but this only makes sense for them, as they've been working on inverse problems for decades. None of them buy canned software/services: it's all done in house. Probably always will be, same as their other imaging efforts.

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#46

Post by andreilys »

raiyu wrote: post #6

The number of places where machine learning can be used effectively from both a cost perspective and a return perspective are small. They are usually tremendously large datasets at gigantic companies, and they probably have to build in house expertise because it's hard to package this up into a product and resell it for various industries, datasets, etc. Certainly something like autonomous driving needs machine learn…

“The number of places where machine learning can be used effectively from both a cost perspective and a return perspective are small.”

Thankfully transfer learning and super convergence invalidates this claim.

Using pre-trained models + specific training techniques significantly reduces the amount of data you need, your training time and the cost to create near state of the art models.

Both Kaggle and google colab offer free GPU.

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#47

Post by buckminster »

bob1029 wrote: post #29
m0zg wrote: post #8

"Huge compute bills" usually come from training, or to be more precise, hyperparameter search that's required before you find a model that works well. You could also fail to find such a model, but that's another discussion. So yeah, you could spend one or two FTE salaries' (or one deep learning PhD's) worth of cash on finding such models for your startup if you insist on helping Jeff Bezos to wipe his tears with cris…

I find it fun how the cost of the cloud is forcing people to consider what absolutely must run in the cloud (presumably for stability and compliance reasons) and what can be brought back on-prem. We don't train ML models, but we are in a similar boat regarding cloud compute costs. Building our solutions for our clients is a compute-heavy task which is getting expensive in the cloud. We are considering options such as…

I once had a borrowed Sun blade server in my home office. The fan in it sounded like an industrial vacuum cleaner. It got moved to a different room and was powered on as little as possible.

Your plan makes sense but be mindful of the acoustics or your devs may grow to hate you.

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#48

Post by KaiserPro »

_bxg1 wrote: post #39

> Training a single AI model can cost hundreds of thousands of dollars (or more) in compute resources Why don't they buy their own hardware for this part? The training process doesn't need to be auto-scalable or failure-resistant or distributed across the world. The value proposition of cloud hosting doesn't seem to make sense here. Surely at this price the answer isn't just "it's more convenient"?

because you are trading speed for cash.

Say you have $8M in funding, and you need to train a model to do x

You can either:

a) gain access to a system that scale ondemand and allows instant, actionable results.

b) hire a infrastructure person, someone to write a K8s deployment system. Another person to come in a throw that all away. Another person to negotiate and buy the hardware, and another to install it.

Option b is can be the cheapest in the long term, but it carries the most risk of failing before you've even trained a single model. It also costs time, and if speed to market is your thing, then you're shit out of luck.

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#49

Post by noelsusman »

ativzzz wrote: post #26

I agree with the author's opinion about > I’ll go out on a limb and assert that most of the up front data pipelining and organizational changes which allow for it are probably more valuable than the actual machine learning piece. Especially at non-tech companies with outdated internal technology. I've consulted at one of these and the biggest wins from the project (I left before the whole thing finished unfortunately…

I do data science at a non-tech company with outdated internal technology and I've seen this over and over again. Honestly though, it's worth every penny because often the only way to get the resources to truly solve data pipeline issues is to get an executive to buy some crap from a vendor and force everyone to make it work.

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#50

Post by pridkett »

m0zg wrote: post #8

"Huge compute bills" usually come from training, or to be more precise, hyperparameter search that's required before you find a model that works well. You could also fail to find such a model, but that's another discussion. So yeah, you could spend one or two FTE salaries' (or one deep learning PhD's) worth of cash on finding such models for your startup if you insist on helping Jeff Bezos to wipe his tears with cris…

There’s also the issue that data scientists often want to go running to hyperparameter optimization and neural architecture search. In most cases improving your data pipelines and ensuring the data are clean and efficient will pay off much more quickly.
Post reply on HN