Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Andreessen-Horowitz craps on “AI” startups from a great height

scottlocklin.wordpress.com

Post reply on HN
Print view View original

71–80 of 256 posts

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#71

Post by Barrin92 »

Q6T46nT668w6i3m wrote: post #14
raiyu wrote: post #6

The number of places where machine learning can be used effectively from both a cost perspective and a return perspective are small. They are usually tremendously large datasets at gigantic companies, and they probably have to build in house expertise because it's hard to package this up into a product and resell it for various industries, datasets, etc. Certainly something like autonomous driving needs machine learn…

How would you explain the rise (and success) of machine learning in science? A lab that uses some learning-based method will likely be limited to just one or two people (responsible for data acquisition, feature engineering, evaluation, etc.) and extremely finite data.

I'm not sure there is a rise. 'Science' is a huge domain. Machine learning if I had to guess maybe plays a role in Also it's doubtful to even categorize machine-learning as science. The goal of science is to generate insight and knowledge, ML solves particular engineering problems or searches problem spaces, it doesn't build fundamental scientific models.

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#72

Post by PeterisP »

_bxg1 wrote: post #57
KaiserPro wrote: post #48

Earlier quoted context omitted.

because you are trading speed for cash. Say you have $8M in funding, and you need to train a model to do x You can either: a) gain access to a system that scale ondemand and allows instant, actionable results. b) hire a infrastructure person, someone to write a K8s deployment system. Another person to come in a throw that all away. Another person to negotiate and buy the hardware, and another to install it. Option b…

Why in the world do you need a Kubernetes deployment system to run a single, manual, one-time (or a handful of times), high-compute job?

Because that high-compute job needs to be distributed on many, many machines, and if you're using cheap preemptible instances you have to handle machines dropping off and joining in while you're running that single job.

It's definitely not something that you can launch manually - perhaps Kubernetes is not the best solution, but you definitely need some automation.

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#73

Post by derefr »

mereel wrote: post #22
alephnan wrote: post #18

Earlier quoted context omitted.

> Just don't call it a "datacenter" or NVIDIA will have a stroke. Context please :) ?

Just a guess but maybe it's some licensing issue? https://www.nvidia.com/en-us/drivers/geforce-license/ No Datacenter Deployment. The SOFTWARE is not licensed for datacenter deployment, except that blockchain processing in a datacenter is permitted.

> except that blockchain processing in a datacenter is permitted

Well, Nvidia, y'see, my new blockchain does AI training as its Proof-of-Work step...

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#74

Post by m0zg »

fxtentacle wrote: post #58
m0zg wrote: post #8

"Huge compute bills" usually come from training, or to be more precise, hyperparameter search that's required before you find a model that works well. You could also fail to find such a model, but that's another discussion. So yeah, you could spend one or two FTE salaries' (or one deep learning PhD's) worth of cash on finding such models for your startup if you insist on helping Jeff Bezos to wipe his tears with cris…

No, also inference is quite expensive. You'll have 100% usage on a $10,000 GPU for 3s per customer image for a decently sized optical flow network. That's 3 hours of compute time for 1 minute of 60fps video. Now let's say your customer wants to analyze 2 hours = 120 minutes of video and doesn't want to wait more than those 3 hours, then suddenly you need 120 servers with one $10k GPU each to service this one customer…

Some kinds of inference are expensive, yes, not going to dispute that. But 99.95% of it is actually surprisingly inexpensive. Hell, a lot of useful workloads can be deployed on a cell phone nowadays, and that fraction will increase over time, further reducing inference costs or eliminating them outright (or rather moving them to the consumer).

For the vast majority of people the main expense is creating the combination of a dataset and model that works for their practical problem, with the dataset being the harder (and sometimes more expensive) problem of the two.

The dataset is also their "moat", even though most of them don't realize it, and don't put enough care into that part of the pipeline.

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#75

Post by motohagiography »

correlator wrote: post #15

No need to look at AZ for this. If you're building "AI" I wish you a speedy road to being acquired by a company that can put it to use. You've become a high priced recruiting firm. If you're solving a real problem and use ML in service of solving that problem, then you've got a great moat....happy trusting customers. It's not complicated

Sssh! Valuations are a function of projected market size and opacity of the problem. Clarity like this collapses the uncertainty and destroys value. If you pour enough capital into rooms full of PhD's something's gotta hit.

My way of saying, you're very, very right.

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#76

Post by joshuaellinger »

I just spent $50K on coloc hardware. I'm taking a $10K/mo Azure spend down to a $1K/mo hosting cost.

But the real kicker is that I get x5 the cores, x20 RAM, x10 storage, and a couple of GPUs. I'm running last-generation Infiniband (56gb/sec) and modern U.2 SSDs (say 500MB/sec per device).

I figure it is going to take me about $10K in labor to move and then $1K/mo to maintain and pay for services that are bundled in the cloud. And because I have all this dedicated hardware, I don't have to mess around with docker/k8s/etc.

It's not really a big data problem but it shows the ROI on owning your own hardware. If you need 100 servers for one day per month, the cloud is amazing. But I do a bunch of resampling, simple models, and interactive BI type stuff, so co-loc wins easily.

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#77

Post by zitterbewegung »

m0zg wrote: post #8

"Huge compute bills" usually come from training, or to be more precise, hyperparameter search that's required before you find a model that works well. You could also fail to find such a model, but that's another discussion. So yeah, you could spend one or two FTE salaries' (or one deep learning PhD's) worth of cash on finding such models for your startup if you insist on helping Jeff Bezos to wipe his tears with cris…

I was training ML models on AWS / Google Colab. After racking up a few hundred dollars on AWS I bought a Titan RTX (I also play video games so it does that very well also.

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#78

Post by fxtentacle »

I predict a great future for startups that sell pickaxes, err, tools for AI.

AI is like the new gold rush. And just like back then, it's not the gold diggers that will get rich.

"Most people in AI forget that the hardest part of building a new AI solution or product is not the AI or algorithms — it’s the data collection and labeling."

https://medium.com/startup-grind/fueling-the-ai-gold-rush-7a...

(from 2017)

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#79

Post by artsyca »

m0zg wrote: post #8

"Huge compute bills" usually come from training, or to be more precise, hyperparameter search that's required before you find a model that works well. You could also fail to find such a model, but that's another discussion. So yeah, you could spend one or two FTE salaries' (or one deep learning PhD's) worth of cash on finding such models for your startup if you insist on helping Jeff Bezos to wipe his tears with cris…

Slow clap

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#80

Post by derefr »

m0zg wrote: post #31
fizixer wrote: post #20

Earlier quoted context omitted.

- Or AMD could change their policy of 'never miss an opportunity to miss an opportunity' and offer high-performance OpenCL GPGPU offerings. Then nVidia could have all the stroke they wanted. - Or Tensorflow/Pytorch could've crapped on OpenCL a little less by releasing a fully functional OpenCL version everytime they released a fully functional Cuda version, instead of worshipping Cuda year in and year out. - Or Googl…

As someone who has actually worked on this stuff soup to nuts, it's not as easy as people imagine, because you can't just support some subset of available ops and call it a day. If you want to make OpenCL pie from scratch, you must first make the universe, and support every single stupid thing (among thousands) and even mimic some of the bugs so that models work "the same". This is hard and time consuming, and this f…

> support every single stupid thing (among thousands) and even mimic some of the bugs so that models work "the same".

Do you need to do the stupid things performantly, though? Because that sounds like a case for skipping microcode shims, and going straight to instructions that trap into a software implementation. Or just running the whole compute-kernel in a driver-side software emulator that then compiles real sub-kernels for each stretch of non-stupid GPGPU instructions, uploads those, and then calls into them from the software emulation at the right moments. Like a profile-guided JIT, but one that can't actually JIT everything, only some things.

Post reply on HN