I just spent $50K on coloc hardware. I'm taking a $10K/mo Azure spend down to a $1K/mo hosting cost. But the real kicker is that I get x5 the cores, x20 RAM, x10 storage, and a couple of GPUs. I'm running last-generation Infiniband (56gb/sec) and modern U.2 SSDs (say 500MB/sec per device). I figure it is going to take me about $10K in labor to move and then $1K/mo to maintain and pay for services that are bundled in…
the point of Cloud is that it solves the problem of variable demand. I used to run on-prem back in the 2000's, and we were constantly dealing with demand fluctuation crises. Spinning up new physical servers to deal with new demand, or being massively over-specced when demand dropped, was a real pain. I'm starting a new thing this week, and using the Cloud for it because I have no idea what our demand will be. I can s…
Andreessen-Horowitz craps on “AI” startups from a great height
221–230 of 256 posts
Re: Andreessen-Horowitz craps on “AI” startups from a great height
#222I just spent $50K on coloc hardware. I'm taking a $10K/mo Azure spend down to a $1K/mo hosting cost. But the real kicker is that I get x5 the cores, x20 RAM, x10 storage, and a couple of GPUs. I'm running last-generation Infiniband (56gb/sec) and modern U.2 SSDs (say 500MB/sec per device). I figure it is going to take me about $10K in labor to move and then $1K/mo to maintain and pay for services that are bundled in…
the point of Cloud is that it solves the problem of variable demand. I used to run on-prem back in the 2000's, and we were constantly dealing with demand fluctuation crises. Spinning up new physical servers to deal with new demand, or being massively over-specced when demand dropped, was a real pain. I'm starting a new thing this week, and using the Cloud for it because I have no idea what our demand will be. I can s…
If your peak demand is 100x your baseline and only happens for ~1h each day, cloud is almost certainly a good choice. If it happens for ~12h a day or it's only 5x your baseline, the cost of the cloud is such that you're likely to save with dedicated hardware, even though much of your hardware sits around doing nothing part of the time.
> never have to worry about ordering new servers a month in advance so I have enough capacity when (or if) I need it.
There is a middle-ground that's very much worth considering: renting dedicated servers. It's not quite as cost-effective as colocation and owning your hardware when you have at least a cabinet worth of stuff but it does offload the management of the hardware and provisioning to somebody else. They can also usually be provisioned in a matter of minutes.
In some cases (e.g. Packet.net) these machines can even be treated essentially like cloud instances, with hourly pricing.
There's also yet another middle ground: using dedicated to handle the known and predictable baseline traffic and using the cloud to handle the unexpected bursts.
Re: Andreessen-Horowitz craps on “AI” startups from a great height
#223Finally a correct use of "AI".
Re: Andreessen-Horowitz craps on “AI” startups from a great height
#224Cost wise though it's clearly being not knowledgeable about how it works or at least think all AI startups have huge training set. For many companies owning your hardware for training is a very easy step to rationalise cost.
It feels like an article written about all AI companies but actually (very) true only for some AI companies.
Re: Andreessen-Horowitz craps on “AI” startups from a great height
#225Earlier quoted context omitted.
the point of Cloud is that it solves the problem of variable demand. I used to run on-prem back in the 2000's, and we were constantly dealing with demand fluctuation crises. Spinning up new physical servers to deal with new demand, or being massively over-specced when demand dropped, was a real pain. I'm starting a new thing this week, and using the Cloud for it because I have no idea what our demand will be. I can s…
I half-agree. The Cloud specifically solves the problem of _highly_ variable demand. If your peak demand is 100x your baseline and only happens for ~1h each day, cloud is almost certainly a good choice. If it happens for ~12h a day or it's only 5x your baseline, the cost of the cloud is such that you're likely to save with dedicated hardware, even though much of your hardware sits around doing nothing part of the tim…
Re: Andreessen-Horowitz craps on “AI” startups from a great height
#226I just spent $50K on coloc hardware. I'm taking a $10K/mo Azure spend down to a $1K/mo hosting cost. But the real kicker is that I get x5 the cores, x20 RAM, x10 storage, and a couple of GPUs. I'm running last-generation Infiniband (56gb/sec) and modern U.2 SSDs (say 500MB/sec per device). I figure it is going to take me about $10K in labor to move and then $1K/mo to maintain and pay for services that are bundled in…
Some of the same caveats apply with respect to software updates, configuration control, security, availability, business continuity, disaster recovery, and what happens if the local admin is hit by a bus.
Re: Andreessen-Horowitz craps on “AI” startups from a great height
#227The number of places where machine learning can be used effectively from both a cost perspective and a return perspective are small. They are usually tremendously large datasets at gigantic companies, and they probably have to build in house expertise because it's hard to package this up into a product and resell it for various industries, datasets, etc. Certainly something like autonomous driving needs machine learn…
How would you explain the rise (and success) of machine learning in science? A lab that uses some learning-based method will likely be limited to just one or two people (responsible for data acquisition, feature engineering, evaluation, etc.) and extremely finite data.
I briefly looked at using neural nets to analyse data from an experiment - analysing the efficacy of toilet bowl designs.
The entry level hardware was £250k in 1981 - it was much cheaper to take photo's and have a research assistant count squares.
Now you could use fairly cheap commodity hardware to do it.
It would have been an amazing cutting edge project if we could have got some government funding we did have an in-house knowledge engineer.
Re: Andreessen-Horowitz craps on “AI” startups from a great height
#228I just spent $50K on coloc hardware. I'm taking a $10K/mo Azure spend down to a $1K/mo hosting cost. But the real kicker is that I get x5 the cores, x20 RAM, x10 storage, and a couple of GPUs. I'm running last-generation Infiniband (56gb/sec) and modern U.2 SSDs (say 500MB/sec per device). I figure it is going to take me about $10K in labor to move and then $1K/mo to maintain and pay for services that are bundled in…
Yea we’re seeing this all over the place at Lambda ( https://lambdalabs.com ). Most people running consistent GPU training or inference jobs are building on-prem clusters or even groups of workstations. It just doesn’t make financial sense to use the big the cloud service providers for those with consistent workloads. I always hear stories where folks have saved hundreds of thousands in infrastructure costs with owni…
As an aside, thank you for your one-line installer script for tf/keras. Earlier, my team used to spend days figuring out the CUDA/tf/keras/CUDNN etc dependency charts, and you've brought that down to ~0.
Re: Andreessen-Horowitz craps on “AI” startups from a great height
#229So, way back in the last millenium, I did my Master's thesis (way smaller deal than a Ph.D. thesis) on neural networks. Since then, I have looked in on it every few years. I think they're cool, I like using them, and writing multi-level backpropagation neural networks used to be one of the first things I'd do in a new language, just to get a feel for how it worked (until pytorch came along and I decided for the first…
> There used to be a saying: "If you don't know how to do it, you don't know how to do it with a computer." This is a tautology in the narrow sense, but in the broader sense I think there surely exist things that humans don't "know" how to do without a computer, but know how to do with a computer. And the space of solveable problems is expanding, though AI is only a narrow slice of that.
Re: Andreessen-Horowitz craps on “AI” startups from a great height
#230This is so right. Using a term "artificial intelligence" for machine learning is like using "artificial horses" to describe cars. It is even worse, since we cannot even define what "natural intelligence" actually is. Stop talking about "artificial intelligence".