Live data from Hacker News

Andreessen-Horowitz craps on “AI” startups from a great height

scottlocklin.wordpress.com

31–40 of 256 posts

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#31
post #20
post #8

"Huge compute bills" usually come from training, or to be more precise, hyperparameter search that's required before you find a model that works well. You could also fail to find such a model, but that's another discussion. So yeah, you could spend one or two FTE salaries' (or one deep learning PhD's) worth of cash on finding such models for your startup if you insist on helping Jeff Bezos to wipe his tears with cris…

- Or AMD could change their policy of 'never miss an opportunity to miss an opportunity' and offer high-performance OpenCL GPGPU offerings. Then nVidia could have all the stroke they wanted. - Or Tensorflow/Pytorch could've crapped on OpenCL a little less by releasing a fully functional OpenCL version everytime they released a fully functional Cuda version, instead of worshipping Cuda year in and year out. - Or Googl…

As someone who has actually worked on this stuff soup to nuts, it's not as easy as people imagine, because you can't just support some subset of available ops and call it a day. If you want to make OpenCL pie from scratch, you must first make the universe, and support every single stupid thing (among thousands) and even mimic some of the bugs so that models work "the same".

This is hard and time consuming, and this field is hard enough as it is. What makes it even harder is that only NVIDIA has decent, mature tooling. There is some work on ROCM though, so AMD is not _totally_ dead in the water. I'd say they're about 90% dead in the water.

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#32
post #31
post #20

Earlier quoted context omitted.

- Or AMD could change their policy of 'never miss an opportunity to miss an opportunity' and offer high-performance OpenCL GPGPU offerings. Then nVidia could have all the stroke they wanted. - Or Tensorflow/Pytorch could've crapped on OpenCL a little less by releasing a fully functional OpenCL version everytime they released a fully functional Cuda version, instead of worshipping Cuda year in and year out. - Or Googl…

As someone who has actually worked on this stuff soup to nuts, it's not as easy as people imagine, because you can't just support some subset of available ops and call it a day. If you want to make OpenCL pie from scratch, you must first make the universe, and support every single stupid thing (among thousands) and even mimic some of the bugs so that models work "the same". This is hard and time consuming, and this f…

Are you in the Bay area? Would love to chat. Thinking of an idea where your expertise could be very handy. $my-hn-username at gmail.

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#33

Earlier quoted context omitted.

> Just don't call it a "datacenter" or NVIDIA will have a stroke. Context please :) ?

NVIDIA forces you to buy significantly more expensive cards that perform marginally better if you are using them for datacenter use. They try to enforce not letting businesses use consumer grade gaming cards. I assume this is so cloud providers don't buy up all the supply of graphics cards and make it hard for gamers to get decent cards, like what happened during the bitcoin craze.

No it's just pure price discrimination. They don't care about gamers they just know businesses will pay more if forced to while gamers can't.

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#34
post #29
post #8

"Huge compute bills" usually come from training, or to be more precise, hyperparameter search that's required before you find a model that works well. You could also fail to find such a model, but that's another discussion. So yeah, you could spend one or two FTE salaries' (or one deep learning PhD's) worth of cash on finding such models for your startup if you insist on helping Jeff Bezos to wipe his tears with cris…

I find it fun how the cost of the cloud is forcing people to consider what absolutely must run in the cloud (presumably for stability and compliance reasons) and what can be brought back on-prem. We don't train ML models, but we are in a similar boat regarding cloud compute costs. Building our solutions for our clients is a compute-heavy task which is getting expensive in the cloud. We are considering options such as…

This is not a new phenomenon. As early as in 2009 I worked for a company (ads, but not Google) which outgrew the typical "cloud" cost structure at the time, and moved everything to a more traditional datacenter, and saved substantial money even considering 3 more SREs they had to hire to absorb the increased support needs. AWS charges what the market will bear, and as such it was never designed to make sense for everyone. One needs to re-evaluate on the back of the napkin from time to time.

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#35
All of this might be true currently, but that's because this current first generation "AI" (technically should just be called ML) is mostly bullshit. To clarify, I don't mean anyone is lying or selling snake oil - what I mean by bullshit is that the vast majority of these services are cooked up by software developers without any background in mathematics, selling adtechy services in domains like product recommendation and sentiment analysis. They are single discipline applications accessable to devs without science backgrounds and do not rely on substantial expertise from other fields. That makes them narrow in technical scope and easy to rip off (hence no moat, lots of competition, and human reliance and lack of actual software).

The next generation of Machine Learning is just emerging, and looks nothing like this. Funds are being raised, patents are being filed, and everything is in early stage development, so you probably haven't heard much yet - but these ML startups are going after real problems in industry: cross disciplinary applications leveraging the power of heuristic learning to make cross disciplinary designs and decisions currently still limited to the human domain.

I'm talking about the kind of heuristics which currently exist only as human intuition expressed most compactly as concept graphs and, especially, mathematical relationships - e.g. component design with stress and materials constraints, geologic model building, treatment recommendation from a corpus of patient data, etc. ML solutions for problems like these cannot be developed without an intimate understanding of the problem domain. This is a generalist's game. I predict that the most successful ML engineers of the next decade will be those with hard STEM backgrounds, MS and PhD level, who have transitioned to ML. [Un]Fortunately for us, the current buzzwordy types of ML services give the rest of us a bad name, but looking at these upcoming applications the answers to the article tl;dr look different:

>Deep learning costs a lot in compute, for marginal payoffs

The payoffs here are far greater. Designs are in the pipeline which augment industry roles - accelerate design by replacing finite methods with vastly quicker ML for unprecedented iteration. Produce meaningful suggestions during the development of 3D designs. Fetch related technical documents in real time by scanning the progressive design as the engineer works, parsing and probabilistically suggesting alternative paths to research progression. Think Bonzi Buddy on steroids...this is a place for recurring software licenses, not SaaS.

>Machine learning startups generally have no moat or meaningful special sauce

For solving specific, technical problems, neural network design requires a certain degree of intuition with respect to the flow of information through the network, which both optimizes and limits the kind of patterns that a given net can learn. Thus designing NN for hard-industry applications is predicated upon an intimate understanding of domain knowledge, and these highly specialized neural nets become patentable secret sauces. That's half of the most - the other comes from competition for the software developers with first-hand experience in these fields, or a general enough math heavy background to capture the relationships that are being distilled into nets.

>Machine learning startups are mostly services businesses, not software businesses

Again only true because most current applications are NLP adtechy bullshit. Imagine coding in an IDE powered by an AI (multiple interacting neural nets) which guides the structure of your code at a high level and flags bugs as you write. This, at a more practical level, is the type of software that will eventually change every technical discipline, and you can sell licenses!

>Machine learning will be most productive inside large organizations that have data and process inefficiencies

This next generation goes far past simply optimizing production lines or counting missed pennies or extracting a couple extra percent of value from analytics data. This style of applied ML operates at a deeper level of design which will change everything.

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#36
post #8

"Huge compute bills" usually come from training, or to be more precise, hyperparameter search that's required before you find a model that works well. You could also fail to find such a model, but that's another discussion. So yeah, you could spend one or two FTE salaries' (or one deep learning PhD's) worth of cash on finding such models for your startup if you insist on helping Jeff Bezos to wipe his tears with cris…

As someone hoping to build a world-wide footprint, say 25 to 50 DCs, of servers to deploy to with unmetered bandwidth, what are some alternatives to the usual suspects?

I have come across fly.io, Vultr, Scaleway, Stackpath, Hetzner, and OVH but either they are expensive (in that they charge for bandwidth and uptime) or do not have a wide enough foot-print.

I guess colos are the way to go, but how does one work with colos, allocate servers, deploy to them, ensure security and uptime and so on from a single place, 'cause dealing with them individually might slow down the process? Is there a tooling that deals with multi-colos like the ones for multi-cloud like min.io, k8s, Triton etc;

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#37
An many times all these AI computations go into solving mundane problems like "What's the likelihood of this Ad to perform well".

AI is so shiny that makes people want to jump as fast as they can into that boat but a reasonable objective analysis shows that a huge and not insignificant amount of software problems can still be solved without relying on the "AI black box".

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#38
post #8

"Huge compute bills" usually come from training, or to be more precise, hyperparameter search that's required before you find a model that works well. You could also fail to find such a model, but that's another discussion. So yeah, you could spend one or two FTE salaries' (or one deep learning PhD's) worth of cash on finding such models for your startup if you insist on helping Jeff Bezos to wipe his tears with cris…

As someone hoping to build a world-wide footprint, say 25 to 50 DCs, of servers to deploy to with unmetered bandwidth, what are some alternatives to the usual suspects? I have come across fly.io, Vultr, Scaleway, Stackpath, Hetzner, and OVH but either they are expensive (in that they charge for bandwidth and uptime) or do not have a wide enough foot-print. I guess colos are the way to go, but how does one work with c…

(Hi, I'm from fly.io)

It depends what you need in your datacenters! If you just want servers, and don't care about doing something like anycast, you can find a bunch of local dedicated server providers in a bunch of cities and go to town. But you can't get them all from one provider, really, not with any kind of reasonable budget.

You _could_ buy colo from a place like Equinix in a bunch of cities, and then either use their transit or buy from other transit providers.

But also, unmetered bandwidth isn't a very sustainable service, so I'm curious what you're after? You're usually either going to have to pay for usage, or pay large monthly fixed prices to get reasonable transit connections in each datacenter.

In our case, we're constrained by Anycast. To expand past the 17 usual cities you end up needing to do your own network engineering which we'd rather not do yet.

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#39
> Training a single AI model can cost hundreds of thousands of dollars (or more) in compute resources

Why don't they buy their own hardware for this part? The training process doesn't need to be auto-scalable or failure-resistant or distributed across the world. The value proposition of cloud hosting doesn't seem to make sense here. Surely at this price the answer isn't just "it's more convenient"?

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#40
Generally the use of the phrase from a great height implies the height is one of morality, intellect, or valor (each of these decreasing in usage), I'm not exactly sure what the great height Andreessen-Horowitz craps from is composed of - maybe money?

I think they may just be crapping on them from a reasonable vantage point.

Post reply on HN