Andreessen-Horowitz craps on “AI” startups from a great height • Hacker News Board

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#151

Post by m0zg » Tue, Feb 25, 2020, 1:15 AM UTC

liuliu wrote: post #132

m0zg wrote: post #8

"Huge compute bills" usually come from training, or to be more precise, hyperparameter search that's required before you find a model that works well. You could also fail to find such a model, but that's another discussion. So yeah, you could spend one or two FTE salaries' (or one deep learning PhD's) worth of cash on finding such models for your startup if you insist on helping Jeff Bezos to wipe his tears with cris…

I've been playing with custom-built 2080 Ti workstation for a while: https://www.youtube.com/watch?v=OF3JYEIsjH8 Several issues: 1. electricity bill is still an issue, I've been paying anywhere between $500 to $1000 per month for this workstation (always have something to train). 2. something with a decent memory size (Titan RTX and RTX 8000) cost way too much; 3. once you reached a point of 4-2080Ti-is-not-fast-enou…

>> $500 to $1000 per month

How much is your electricity? I currently run 12 GPUs in my garage pretty much non-stop. 4 GPUs per machine, 3 machines. Each machine is about 1.2KW on average (I can tell because each machine is connected through its own rack UPS), or 13.2 cents per hour, or $95/mo. Which, IMO, is not bad at all. That's less than $300 per month for 12 GPUs.

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#152

Post by phreeza » Tue, Feb 25, 2020, 1:16 AM UTC

rossdavidh wrote: post #60

So, way back in the last millenium, I did my Master's thesis (way smaller deal than a Ph.D. thesis) on neural networks. Since then, I have looked in on it every few years. I think they're cool, I like using them, and writing multi-level backpropagation neural networks used to be one of the first things I'd do in a new language, just to get a feel for how it worked (until pytorch came along and I decided for the first…

> There used to be a saying: "If you don't know how to do it, you don't know how to do it with a computer."

This is a tautology in the narrow sense, but in the broader sense I think there surely exist things that humans don't "know" how to do without a computer, but know how to do with a computer. And the space of solveable problems is expanding, though AI is only a narrow slice of that.

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#153

Post by perl4ever » Tue, Feb 25, 2020, 1:16 AM UTC

all_blue_chucks wrote: post #124

jedberg wrote: post #105

Earlier quoted context omitted.

Saying that you're going to "use AI" is more akin to saying "we're going to have a web application" back in 1998. Back then a lot of startups didn't have websites, because they were making other products (hardware, boxed software, etc). If they had a website it was just a marketing page. So saying that you were going to make a "web application" did in fact differentiate you, in that it showed your approach was very d…

"Web application" came later. In the nineties it was called a "cgi web page" by your webmaster.

In the nineties, there was a huge difference between 1995 and 1998. It wasn't all that apparent to some of us until later, but things moved really fast in that timeframe. The years leading up to 2000 were almost like the imagining of approaching an event horizon or asymptote.

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#154

Post by inthewoods » Tue, Feb 25, 2020, 1:16 AM UTC

Having briefly worked for an AI company, I agree with the conclusion that AI companies are more like services businesses than software companies. I would add only one other thing: to me going forward there likely won't be "AI companies" - AI exists to power applications. And in my experience, unless the output is truly differentiated, customers aren't willing to spend more for something "powered by AI" - they just expect that software has evolved to provide the kind of insights that AI sometimes deliver.

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#155

Post by eyegor » Tue, Feb 25, 2020, 1:21 AM UTC

joshuaellinger wrote: post #76

I just spent $50K on coloc hardware. I'm taking a $10K/mo Azure spend down to a $1K/mo hosting cost. But the real kicker is that I get x5 the cores, x20 RAM, x10 storage, and a couple of GPUs. I'm running last-generation Infiniband (56gb/sec) and modern U.2 SSDs (say 500MB/sec per device). I figure it is going to take me about $10K in labor to move and then $1K/mo to maintain and pay for services that are bundled in…

Yes, it's quite obvious when you actually have compute needs. At my current employer, we spent about 100k to build a small single purpose hpc. One year later, I calculated the azure costs (help bargain for more servers) would have been around 1.5m. This is almost 24/7 use though, and add another ~150k in electricity.

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#156

Post by randomdude402 » Tue, Feb 25, 2020, 1:22 AM UTC

all_blue_chucks wrote: post #124

jedberg wrote: post #105

Earlier quoted context omitted.

Saying that you're going to "use AI" is more akin to saying "we're going to have a web application" back in 1998. Back then a lot of startups didn't have websites, because they were making other products (hardware, boxed software, etc). If they had a website it was just a marketing page. So saying that you were going to make a "web application" did in fact differentiate you, in that it showed your approach was very d…

"Web application" came later. In the nineties it was called a "cgi web page" by your webmaster.

Or from time to time, a webmistress.

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#157

Post by jotakami » Tue, Feb 25, 2020, 1:23 AM UTC

> Better user interfaces are sorely underappreciated.

This is why I’m much more excited by AR and VR than AI. Human brains are fucking amazing at certain kinds of data processing and inference and pretty mediocre at others. We should be focusing more on creating interfaces and data visualizations that unlock that superpower for wider applications.

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#158

Post by edw » Tue, Feb 25, 2020, 1:28 AM UTC

perl4ever wrote: post #153

all_blue_chucks wrote: post #124

Earlier quoted context omitted.

"Web application" came later. In the nineties it was called a "cgi web page" by your webmaster.

In the nineties, there was a huge difference between 1995 and 1998. It wasn't all that apparent to some of us until later, but things moved really fast in that timeframe. The years leading up to 2000 were almost like the imagining of approaching an event horizon or asymptote.

What you’re describing is so hard to convey to people. In 1994 the we were building raised-floor data centers with halon for suppressors and marveling at our 2GB behemoth UNIX boxes. And writing our own web application framework using CGI. In ‘99 we were renting a suite at a colo and putting our own hardware there, running ColdFusion web apps. In ‘04 we were renting half a rack at the same colo and trying not to write three tier Java servlet based apps with 1,000 line web.xml files. And then AWS happened.

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#159

Post by shoo » Tue, Feb 25, 2020, 1:29 AM UTC

> most people haven’t figured out that ML oriented processes almost never scale like a simpler application would. You will be confronted with the same problem as using SAP; there is a ton of work done up front; all of it custom. I’ll go out on a limb and assert that most of the up front data pipelining and organizational changes which allow for [ML to be used operationally by an org] are probably more valuable than the actual machine learning piece.

Strong agreement from me: I've never worked on deploying ML models, but have worked on deploying operations-research type automated decision systems that have somewhat similar data requirements. Most of the work is client org specific in terms of setting up the human & machine processes to define a data pipeline to provide input and consume output of the clever little black box. A lot of this is super idiosyncratic & non repeatable between different client deployments.

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#160

Post by latch » Tue, Feb 25, 2020, 1:31 AM UTC

wpietri wrote: post #130

joshuaellinger wrote: post #76

I just spent $50K on coloc hardware. I'm taking a $10K/mo Azure spend down to a $1K/mo hosting cost. But the real kicker is that I get x5 the cores, x20 RAM, x10 storage, and a couple of GPUs. I'm running last-generation Infiniband (56gb/sec) and modern U.2 SSDs (say 500MB/sec per device). I figure it is going to take me about $10K in labor to move and then $1K/mo to maintain and pay for services that are bundled in…

I'm sure your right for your case. But I'd add one caveat for those less experienced: if you own the hardware, you need to be prepared to go to the colo when something breaks. The various clouds are a much nicer experience when hardware fails. At the very least people should have enough spare capacity that a hardware failure means going sometime in the next couple of weeks, rather than getting up at 3 am and fixing t…

Or take the middle road and just get rent the hardware (aka, dedicated hosting). You pay more than colo but still way less than cloud, get the same level of hardware support as a cloud provider but the same performance as colo.