Andreessen-Horowitz craps on “AI” startups from a great height • Hacker News Board

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#181

Post by angry_octet » Tue, Feb 25, 2020, 2:43 AM UTC

There are many problems which are simply impossible to do with traditional optimisation or human analysis, that ML can do really well at. But I get the sense that this is not the type of problem that these "AI" startups referred to are addressing. Instead its like 'here is a problem I can charge for, with some ML magic it will be easy'. This is classic snake oil.

Being able to sift/classify/analyse data with ML really can be a 'moat', an extreme competitive advantage. But using "AI" doesn't automatically get you there.

Separately, AWS is an expensive luxury, which is worth it if for some reason you can't manage your own computers.

I really annoys me when analysts like this guy mangle together things which are obvious and then comes up with an unsupported conclusion, like "second AI winter is coming man".

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#182

Post by liuliu » Tue, Feb 25, 2020, 2:51 AM UTC

m0zg wrote: post #151

liuliu wrote: post #132

Earlier quoted context omitted.

I've been playing with custom-built 2080 Ti workstation for a while: https://www.youtube.com/watch?v=OF3JYEIsjH8 Several issues: 1. electricity bill is still an issue, I've been paying anywhere between $500 to $1000 per month for this workstation (always have something to train). 2. something with a decent memory size (Titan RTX and RTX 8000) cost way too much; 3. once you reached a point of 4-2080Ti-is-not-fast-enou…

>> $500 to $1000 per month How much is your electricity? I currently run 12 GPUs in my garage pretty much non-stop. 4 GPUs per machine, 3 machines. Each machine is about 1.2KW on average (I can tell because each machine is connected through its own rack UPS), or 13.2 cents per hour, or $95/mo. Which, IMO, is not bad at all. That's less than $300 per month for 12 GPUs.

Sorry, it is 2-month billing cycle. We have around 30 cents per kwh I think.

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#183

Post by MrK93 » Tue, Feb 25, 2020, 2:57 AM UTC

shoo wrote: post #167

inthewoods wrote: post #154

Having briefly worked for an AI company, I agree with the conclusion that AI companies are more like services businesses than software companies. I would add only one other thing: to me going forward there likely won't be "AI companies" - AI exists to power applications. And in my experience, unless the output is truly differentiated, customers aren't willing to spend more for something "powered by AI" - they just ex…

For an example of a genuine software company vaguely in this ecosystem, consider companies that build the tools that some AI/ML/optimisation systems use as building blocks. Eg optimisation algorithms. If you need to solve gnarly industrial scale mixed integer combinatorial optimisation problems in the guts of your ML / optimisation engine, the commercial MIP solvers (gurobi , CPLEX ) or non-MIP based alternative comb…

I'm all for linear optimization and other optimization techniques. It's refreshing to see other people talk about Gurobi, CPLEX, etc... Having done research in the field of scheduling and now getting contacted by companies, it's demoralizing to see that everybody usually speaks about machine learning while many problems can be solved in a more precise way with other techniques.

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#184

Post by jeremysalwen » Tue, Feb 25, 2020, 2:57 AM UTC

yogrish wrote: post #118

Now a days DL models are becoming commodities very fast. By the time you train NN to solve a particular problem, a new efficient model is out somewhere and is available public. So you need to go through the process entirely or else you risk losing business. Unless your NN is so unique like you are handcrafting your own in which case you take lot of time to arrive at a best model and you need more PhDs.

Props to the ML community for being so open.

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#185

Post by liuliu » Tue, Feb 25, 2020, 3:09 AM UTC

eyegor wrote: post #146

liuliu wrote: post #132

Earlier quoted context omitted.

I've been playing with custom-built 2080 Ti workstation for a while: https://www.youtube.com/watch?v=OF3JYEIsjH8 Several issues: 1. electricity bill is still an issue, I've been paying anywhere between $500 to $1000 per month for this workstation (always have something to train). 2. something with a decent memory size (Titan RTX and RTX 8000) cost way too much; 3. once you reached a point of 4-2080Ti-is-not-fast-enou…

10gbe will depend on the workload. In general, I'd assume it's fine because it takes a parallel raid setup to saturate. Upgrading to 100gbe is pretty unreasonable cost wise unless you buy network gear from a back alley van dealer. Although once you reach 4 2080ti, you ought to consider switching to a titanium grade psu and rewiring if you're in a 100-120v country. If you're feeling cheap, just steal the phases from t…

I've encountered some latency issues with allreduce on transformer models due to vocabulary sizes when communication have to cross PCIe lanes. Increasing batch size helps a lot, but low-latency & high-throughput is universally more helpful to lift these minor concerns (I don't really want to care about my batch size to improve allreduce performance). Hence worried not only throughput, but also latency on consumer-grade 10ghe equipments.

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#186

Post by divbzero » Tue, Feb 25, 2020, 3:24 AM UTC

izendejas wrote: post #165

shoo wrote: post #159

> most people haven’t figured out that ML oriented processes almost never scale like a simpler application would. You will be confronted with the same problem as using SAP; there is a ton of work done up front; all of it custom. I’ll go out on a limb and assert that most of the up front data pipelining and organizational changes which allow for [ML to be used operationally by an org] are probably more valuable than t…

That's because, ML and operations-research problems can be simplified to set of optimization problems and the underlying math and statistics are all very similar if not identical in some cases. And the input matters, a lot. So the differentiating factor isn't the models, it's the data and companies like Google figured it out a long time ago. In short, find interesting problems, then the solutions -- not the other way…

This is spot on. Hence the open sourcing of ML code while keeping an iron grip on data.

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#187

Post by nl » Tue, Feb 25, 2020, 3:34 AM UTC

perl4ever wrote: post #149

derefr wrote: post #127

Earlier quoted context omitted.

> You can't build hand-build a feature detector as accurate as (say) a ResNet50 by hand. Yes, you can . If , that is, you can actually understand what the produced model is doing. And, of course, no human can do that, because no human understands the algorithm being employed by the produced model, because it's a really freaking complex algorithm whose optimal formulation really is just a graph of matrix transformatio…

"There would be no gain from using ML in RDBMS query planning. It's regular data; the AI's trained model would just be a recapitulation of the query-planning algorithm we already have." Most of what you wrote seems fine, until I got to this. A query optimizer seems like something that tends to be very opaque, very complex, and in my experience blows up without a good explanation frequently in typical situations. It's…

Most (?) RDBMS query planners rely on updating statistics on the data in the table to decide what type of joins to perform.

I can imagine cases (distributed databases, different speed storage) where it would make sense to test the queries and learn which optimisations make sense. It'd be self tuning and able to adapt to changing hardware.

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#188

Post by sp527 » Tue, Feb 25, 2020, 3:57 AM UTC

harias wrote: post #5

>That’s right; that’s why a lone wolf like me, or a small team can do as good or better a job than some firm with 100x the head count and 100m in VC backing. goes on to say >I agree, but the hockey stick required for VC backing, and the army of Ph.D.s required to make it work doesn’t really mix well with those limited domains, which have a limited market. Choose one? Also assumes running your own data center to be ea…

Training ML models usually doesn’t have the same uptime requirements as production systems. If your training goes down for a bit, it probably won’t make much difference to the underlying business, in most cases.

That’s why the author found it glaringly obvious that it should be brought in-house. It’s often both the most costly and most “in-housable” compute work involved in these companies.

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#189

Post by joshuaellinger » Tue, Feb 25, 2020, 4:10 AM UTC

andrew311 wrote: post #174

joshuaellinger wrote: post #76

I just spent $50K on coloc hardware. I'm taking a $10K/mo Azure spend down to a $1K/mo hosting cost. But the real kicker is that I get x5 the cores, x20 RAM, x10 storage, and a couple of GPUs. I'm running last-generation Infiniband (56gb/sec) and modern U.2 SSDs (say 500MB/sec per device). I figure it is going to take me about $10K in labor to move and then $1K/mo to maintain and pay for services that are bundled in…

What colo company did you use?

In Austin, DataFoundry is by far the best. It was overkill for me and went with something off the beaten path but they have an amazing facility.

I wound up at a facility run by a fiber vendor because they'd sell me a fixed 250mbps pipe for the same price that a data center would sell me 20mbps pipe that bursts to 1gbps. It only works for me because of the nature of my business -- most people would be better off somewhere else.

Choosing a co-loc facility is complicated. My recommendation is to tour and get quotes from 3-5 vendors in your area before choosing anyone. Ideally, take someone who has done it before.

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#190

Post by joshuaellinger » Tue, Feb 25, 2020, 4:19 AM UTC

dmak wrote: post #134

joshuaellinger wrote: post #76

I just spent $50K on coloc hardware. I'm taking a $10K/mo Azure spend down to a $1K/mo hosting cost. But the real kicker is that I get x5 the cores, x20 RAM, x10 storage, and a couple of GPUs. I'm running last-generation Infiniband (56gb/sec) and modern U.2 SSDs (say 500MB/sec per device). I figure it is going to take me about $10K in labor to move and then $1K/mo to maintain and pay for services that are bundled in…

How did you estimate your hardware needs?

Gee... if only there was a service where you could spin up machines on demand. (joke)

I kinda worked backwards from the cost. I ran the business for a year on Azure but each 'sample' of the resample took about 2 mins so it precluded any near real-time analysis. I ported the kernel to a GPU locally using python/numba and it ran in about 10 seconds and that was enough to seal-the-deal.

From there, I spec-ed out a GPU server and then machines that matched each role in my environment. I decided I was willing to spend $50K and just started loading up the machines.