Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Andreessen-Horowitz craps on “AI” startups from a great height

scottlocklin.wordpress.com

Post reply on HN
Print view View original

121–130 of 256 posts

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#121

Post by Aeolun »

bob1029 wrote: post #104
blt wrote: post #67

Earlier quoted context omitted.

If I worked from home and my employer asked me to install a server in my home, I would tell them to go fuck themselves. It's noisy, it takes up space, and presumably I'm on call to fix it if it breaks. You should pay them an extra 24x(PSU wattage)x(peak $/Wh in area) per day for the electricity too. I'm alarmed that someone in your company felt this idea was appropriate enough to propose.

We would certainly compensate employees. I didn't feel it appropriate to disclose every last detail regarding the arrangement in a thread which is only tangentially-related to the OP. This was my idea. I am a developer in my company. We are a flat structure. We have a lot of respect for each other. I am on a standup with the CEO every day. We all believe in our product and would happily participate in whatever activi…

I guess the ‘go fuck yourselves’ is heavily dependent on how able the company is to do this sort of stuff without involving you.

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#122

Post by wpietri »

roel_v wrote: post #114
m0zg wrote: post #53

Earlier quoted context omitted.

BTW, the only reason why consumer vacuum cleaners are so loud is because consumers associate loudness with suction power. "Backpack" style commercial vacuum cleaners have more suction, and are barely audible in comparison.

I don't know about backpack style vacuums (the few I've seen seem plenty loud), but the idea that manufacturers deliberately create loud vacuums to make them seem more powerful is ridiculous. There is an entire market segment of vacuums that are marketed as 'quiet', so there is plenty of demand for quiet vacuums. There is simply a trade off between suction / loudness / price, and various models make different trade o…

That some people want quiet vacuums is not proof that some people don't want loud vacuums. Cars and motorcycles are good examples of markets where you see both deliberately quiet and deliberately loud examples.

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#123

Post by dnautics »

fizixer wrote: post #20
m0zg wrote: post #8

"Huge compute bills" usually come from training, or to be more precise, hyperparameter search that's required before you find a model that works well. You could also fail to find such a model, but that's another discussion. So yeah, you could spend one or two FTE salaries' (or one deep learning PhD's) worth of cash on finding such models for your startup if you insist on helping Jeff Bezos to wipe his tears with cris…

- Or AMD could change their policy of 'never miss an opportunity to miss an opportunity' and offer high-performance OpenCL GPGPU offerings. Then nVidia could have all the stroke they wanted. - Or Tensorflow/Pytorch could've crapped on OpenCL a little less by releasing a fully functional OpenCL version everytime they released a fully functional Cuda version, instead of worshipping Cuda year in and year out. - Or Googl…

That article is mostly right, but there's one part that got skimped on that will mess you up big time with about an 20% chance if you run for long enough.

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#124

Post by all_blue_chucks »

jedberg wrote: post #105
rossdavidh wrote: post #60

So, way back in the last millenium, I did my Master's thesis (way smaller deal than a Ph.D. thesis) on neural networks. Since then, I have looked in on it every few years. I think they're cool, I like using them, and writing multi-level backpropagation neural networks used to be one of the first things I'd do in a new language, just to get a feel for how it worked (until pytorch came along and I decided for the first…

Saying that you're going to "use AI" is more akin to saying "we're going to have a web application" back in 1998. Back then a lot of startups didn't have websites, because they were making other products (hardware, boxed software, etc). If they had a website it was just a marketing page. So saying that you were going to make a "web application" did in fact differentiate you, in that it showed your approach was very d…

"Web application" came later. In the nineties it was called a "cgi web page" by your webmaster.

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#125

Post by wpietri »

webel0 wrote: post #115
paulddraper wrote: post #94

Earlier quoted context omitted.

> Also, if your startup is venture funded, AWS will give you $100K in credit AFAIK that is limited to <$20k and it expires.

We got $100k but boy oh boy once you’re on it it’s hard to get off. Now we’re close to 50-50 gcloud/aws

Hah! "First hit is free, man!" Makes sense, though. Anybody offering a "free" $100k of anything can afford a lot of MBA time to make sure that the CLTV is well over costs.

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#126

Post by calebkaiser »

m0zg wrote: post #8

"Huge compute bills" usually come from training, or to be more precise, hyperparameter search that's required before you find a model that works well. You could also fail to find such a model, but that's another discussion. So yeah, you could spend one or two FTE salaries' (or one deep learning PhD's) worth of cash on finding such models for your startup if you insist on helping Jeff Bezos to wipe his tears with cris…

Inference is also becoming a bigger contributor to compute bills, especially as models get bigger. With big models like GPT-2, its not unheard of for teams to scale up to hundreds of GPU instances to handle a surprisingly small number of concurrent users. Things can get expensive pretty quick.

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#127

Post by derefr »

nl wrote: post #113
derefr wrote: post #69

Earlier quoted context omitted.

Or, to put that another way: if you knew what algorithm the AI would be using to discriminate the signal from the noise in your data, why would you need the AI? Just write that algorithm.

That's not the same thing at all. You can't build hand-build a feature detector as accurate as (say) a ResNet50 by hand. Before 2014 people tried to do this with feature detectors like SIFT and HOG. These were patented and made the inventors significant money. If it was still possible to do it then someone would be and making profit from it. Hyperparameter search is just optimising the training parameters (things lik…

> You can't build hand-build a feature detector as accurate as (say) a ResNet50 by hand.

Yes, you can. If, that is, you can actually understand what the produced model is doing. And, of course, no human can do that, because no human understands the algorithm being employed by the produced model, because it's a really freaking complex algorithm whose optimal formulation really is just a graph of matrix transformations, rather than an imperative procedure that can be described by words like "variable."

This is an important idea to absorb, for the specific case where the AI converges on an optimal algorithm that's actually very simple—because the data has a regular, simple shape—rather than on one that's too complicated for our mortal minds. If you already knew that simple algorithm, then the work you did training an AI just to end up back at that same simple algorithm is wasted effort. An AI can't do better at e.g. being an AND gate, than an actual AND gate can. An AI can't do what wc(1) does better than wc(1) can.

If the data is regular—that is, if a model of its structure can be held fully in a human brain—then jumping immediately to Machine Learning, before trying to just solve the problem with an algorithm, is silly. The only time you should start with ML, is when it's clear that your problem can't be cleanly mapped into the domain of human procedural thinking.

The AI programmers of the 1960s were not wrong to start with Expert Systems (i.e. attempting to write general algorithms) for deduction, and only begrudgingly turn to fuzzy logic later on. Many deduction tasks are algorithmic. If you don't require the context of "common sense", but only need operate on data types you understand, you can get very far indeed with purely-algorithmic deduction, as e.g. modern RDBMS query planners do. There would be no gain from using ML in RDBMS query planning. It's regular data; the AI's trained model would just be a recapitulation of the query-planning algorithm we already have.

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#128

Post by starfallg »

bob1029 wrote: post #104
blt wrote: post #67

Earlier quoted context omitted.

If I worked from home and my employer asked me to install a server in my home, I would tell them to go fuck themselves. It's noisy, it takes up space, and presumably I'm on call to fix it if it breaks. You should pay them an extra 24x(PSU wattage)x(peak $/Wh in area) per day for the electricity too. I'm alarmed that someone in your company felt this idea was appropriate enough to propose.

We would certainly compensate employees. I didn't feel it appropriate to disclose every last detail regarding the arrangement in a thread which is only tangentially-related to the OP. This was my idea. I am a developer in my company. We are a flat structure. We have a lot of respect for each other. I am on a standup with the CEO every day. We all believe in our product and would happily participate in whatever activi…

Try the value data center providers like he.net.

A 42u rack and 1 Gbps connection is $400 per month.

Put cheap supermicro EPYC servers in rather than threadrippers (or build your own). High capacity RDIMMs are cheaper than UDIMMs.

This will give you a much more maintainable solution than workstations at employee's home via overlay VPN.

There is a maintenance cost for infrastructure that people tend to forget these days.

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#129

Post by woeirua »

allovernow wrote: post #56
scottlocklin wrote: post #45

Earlier quoted context omitted.

>The next generation of Machine Learning is just emerging, and looks nothing like this. Funds are being raised, patents are being filed, and everything is in early stage development, so you probably haven't heard much yet ... Citations needed. Large claims: presumably you can name one example of this, and hopefully it's not a company you work at. I've seen projects on literally all the things you mention: materials s…

>Citations needed. Large claims: presumably you can name one example of this, and hopefully it's not a company you work at. Unfortunately this is all emerging just now and yes, I do work at such a company, but I'm old enough to not be naively excited by some hot fad. There's something profound just starting to happen but everyone is keeping the tech rather secret because it isn't developed/differentiated enough yet t…

Fossil fuels are done. Even if they get a reprieve for another four years, they'll be done after that. As someone who currently works in the petroleum industry, I wouldn't recommend anyone start a company in it now. It's far too volatile, and the political winds are shifting strongly against it.

I don't believe anyone who says that we're getting close to AGI. There are interesting techniques that might get us closer, but they're so far behind in terms of compute power and scaling that it's hard to see how they could be practical in the near future. Anything that's based on back prop is not going to get there. The singularity is a loooong way away.

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#130

Post by wpietri »

joshuaellinger wrote: post #76

I just spent $50K on coloc hardware. I'm taking a $10K/mo Azure spend down to a $1K/mo hosting cost. But the real kicker is that I get x5 the cores, x20 RAM, x10 storage, and a couple of GPUs. I'm running last-generation Infiniband (56gb/sec) and modern U.2 SSDs (say 500MB/sec per device). I figure it is going to take me about $10K in labor to move and then $1K/mo to maintain and pay for services that are bundled in…

I'm sure your right for your case. But I'd add one caveat for those less experienced: if you own the hardware, you need to be prepared to go to the colo when something breaks. The various clouds are a much nicer experience when hardware fails. At the very least people should have enough spare capacity that a hardware failure means going sometime in the next couple of weeks, rather than getting up at 3 am and fixing things under pressure.
Post reply on HN