Live data from Hacker News

Andreessen-Horowitz craps on “AI” startups from a great height

scottlocklin.wordpress.com

131–140 of 256 posts

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#131
post #63

“ Embrace services. There are huge opportunities to meet the market where it stands. That may mean offering a full-stack translation service rather than translation software or running a taxi service rather than selling self-driving cars. Building hybrid businesses is harder than pure software, but this approach can provide deep insight into customer needs and yield fast-growing, market-defining companies. Services c…

The thesis of the article is that this is going to be called consultancy.

Single-focus disruptors bad. Generic consultancy good - with ML secret sauce, possibly helped by hired specialist human insight.

Companies that can make this work will kill it. Companies that can't will be killed.

It's going to be IBM, Oracle, SAP, etc all over again. Within 10 years there will be a dominant monopolistic player in the ML space. It will be selling corporate ML-as-a-service, doing all of that hard data wrangling and model building etc and setting it up for clients as a packaged service using its own economies of scale and "top sales talent" (it says here).

That's where the big big big big money will be. Not in individual specialist "We ML'd your pizza order/pet food/music choices/bicycle route to work" startups.

Amazon, Google, MS, and maybe the twitching remnants of IBM will be fighting it out in this space. But it's possible they'll get their lunch money stolen by a hungry startup, perhaps in collaboration with someone like McKinsey, or an investment bank, or a quant house with ambitions.

5-10 years after that customisable industrial-grade ML will start trickling down to the personal level. But it will probably have been superseded by primitive AGI by then, which makes prediction difficult - especially about that future.

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#132
post #8

"Huge compute bills" usually come from training, or to be more precise, hyperparameter search that's required before you find a model that works well. You could also fail to find such a model, but that's another discussion. So yeah, you could spend one or two FTE salaries' (or one deep learning PhD's) worth of cash on finding such models for your startup if you insist on helping Jeff Bezos to wipe his tears with cris…

I've been playing with custom-built 2080 Ti workstation for a while: https://www.youtube.com/watch?v=OF3JYEIsjH8

Several issues: 1. electricity bill is still an issue, I've been paying anywhere between $500 to $1000 per month for this workstation (always have something to train). 2. something with a decent memory size (Titan RTX and RTX 8000) cost way too much; 3. once you reached a point of 4-2080Ti-is-not-fast-enough, power management and connectivity setup would be a nightmare.

Would love to know other people's opinions on the on-prem setup, especially whether a consumer-grade 10Ghe is enough for connectivity-wise.

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#133

So, way back in the last millenium, I did my Master's thesis (way smaller deal than a Ph.D. thesis) on neural networks. Since then, I have looked in on it every few years. I think they're cool, I like using them, and writing multi-level backpropagation neural networks used to be one of the first things I'd do in a new language, just to get a feel for how it worked (until pytorch came along and I decided for the first…

> If you don't know how to do it, you don't know how to do it with a computer.

This is so true. We spent decades educating non-technical people that understanding a problem well is a prerequisite to programming it. Take something easy to understand like driving a car, doing it in a computer is now harder.

AI is undoing all that. People reach a vague problem they can't describe and assume computers will magically fix it.

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#134

I just spent $50K on coloc hardware. I'm taking a $10K/mo Azure spend down to a $1K/mo hosting cost. But the real kicker is that I get x5 the cores, x20 RAM, x10 storage, and a couple of GPUs. I'm running last-generation Infiniband (56gb/sec) and modern U.2 SSDs (say 500MB/sec per device). I figure it is going to take me about $10K in labor to move and then $1K/mo to maintain and pay for services that are bundled in…

How did you estimate your hardware needs?

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#135
post #63

“ Embrace services. There are huge opportunities to meet the market where it stands. That may mean offering a full-stack translation service rather than translation software or running a taxi service rather than selling self-driving cars. Building hybrid businesses is harder than pure software, but this approach can provide deep insight into customer needs and yield fast-growing, market-defining companies. Services c…

The thesis of the article is that this is going to be called consultancy. Single-focus disruptors bad. Generic consultancy good - with ML secret sauce, possibly helped by hired specialist human insight. Companies that can make this work will kill it. Companies that can't will be killed. It's going to be IBM, Oracle, SAP, etc all over again. Within 10 years there will be a dominant monopolistic player in the ML space.…

The big consulting firms have been building in-house ML libraries for common business problems for 3+ years. They don't need to acquire the data startups because as the article points out, these models are commoditized pretty quickly (especially when you have access to the transactional data of many large multinational companies). There is no secret sauce to ML that makes you any more likely to succeed with it than Accenture -- and they have a much deeper pipeline than you do. ML is a mature capability at all of the enterprise-tier consultancies, and they bundle it with their $100M system deployments. The mid-market consultancies are working on it. There is very little money to squeeze out of this market.

We're also a long way off from AGI. Nobody really even has a roadmap to what an AGI would look like. Heck, DNN/ML techniques have been widely-known since the early 90s; they just became practical with access to cloud-scale hardware, so the current situation has been 25+ years in the making.

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#136

Earlier quoted context omitted.

Saying that you're going to "use AI" is more akin to saying "we're going to have a web application" back in 1998. Back then a lot of startups didn't have websites, because they were making other products (hardware, boxed software, etc). If they had a website it was just a marketing page. So saying that you were going to make a "web application" did in fact differentiate you, in that it showed your approach was very d…

"Web application" came later. In the nineties it was called a "cgi web page" by your webmaster.

I know, but I'm writing to a modern audience. :)

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#137

I just spent $50K on coloc hardware. I'm taking a $10K/mo Azure spend down to a $1K/mo hosting cost. But the real kicker is that I get x5 the cores, x20 RAM, x10 storage, and a couple of GPUs. I'm running last-generation Infiniband (56gb/sec) and modern U.2 SSDs (say 500MB/sec per device). I figure it is going to take me about $10K in labor to move and then $1K/mo to maintain and pay for services that are bundled in…

I did similar at my current and last job. Rather than spend $24k/month, I spent $50k, bought a shitton of hardware, built a virtualization cluster at Corp, and upgraded our connections. Accounting thought i was a wizard.

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#138
post #134

I just spent $50K on coloc hardware. I'm taking a $10K/mo Azure spend down to a $1K/mo hosting cost. But the real kicker is that I get x5 the cores, x20 RAM, x10 storage, and a couple of GPUs. I'm running last-generation Infiniband (56gb/sec) and modern U.2 SSDs (say 500MB/sec per device). I figure it is going to take me about $10K in labor to move and then $1K/mo to maintain and pay for services that are bundled in…

How did you estimate your hardware needs?

I plan to do this in the near future once my GCP credits are used up (18 months of credits left).

My plan is to temporarily shift to dedicated hardware through a service like Hetzner to evaluate what kind of hardware I need. I can simply redirect a fraction of the traffic and extrapolate. Since this is elastic there will be no upfront costs, but I can play around with different sizes. Once I'm happy with my estimate, buy real hardware and move the rest over.

At least that's the plan. I don't think you can do much more than an educated guess and I think this will be as close as I can get.

Not AI related btw.

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#139
post #33

Earlier quoted context omitted.

NVIDIA forces you to buy significantly more expensive cards that perform marginally better if you are using them for datacenter use. They try to enforce not letting businesses use consumer grade gaming cards. I assume this is so cloud providers don't buy up all the supply of graphics cards and make it hard for gamers to get decent cards, like what happened during the bitcoin craze.

No it's just pure price discrimination. They don't care about gamers they just know businesses will pay more if forced to while gamers can't.

I wouldn't say they don't care about gamers, considering that gaming makes up about half of their revenue: https://www.anandtech.com/show/15513/nvidia-releases-q4-fy20...

Re: Andreessen-Horowitz craps on “AI” startups from a great height

#140
post #127
post #113

Earlier quoted context omitted.

That's not the same thing at all. You can't build hand-build a feature detector as accurate as (say) a ResNet50 by hand. Before 2014 people tried to do this with feature detectors like SIFT and HOG. These were patented and made the inventors significant money. If it was still possible to do it then someone would be and making profit from it. Hyperparameter search is just optimising the training parameters (things lik…

> You can't build hand-build a feature detector as accurate as (say) a ResNet50 by hand. Yes, you can . If , that is, you can actually understand what the produced model is doing. And, of course, no human can do that, because no human understands the algorithm being employed by the produced model, because it's a really freaking complex algorithm whose optimal formulation really is just a graph of matrix transformatio…

>> But manually improving the data pipeline requires an understanding of the problem

> Or, to put that another way: if you knew what algorithm the AI would be using to discriminate the signal from the noise in your data, why would you need the AI? Just write that algorithm.

My point is that this isn't the same thing at all.

Say your problem is plant detection from mobile phone photos. I can understand everything about plants, and I can manually build a highly optimised data processing pipeline.

But I can't build a feature extractor that outperforms ResNet50. That's the key algorithm.

> If the data is regular—that is, if a model of its structure can be held fully in a human brain—then jumping immediately to Machine Learning, before trying to just solve the problem with an algorithm, is silly.

True, but no one has made that argument. This is specifically about using hyperparameter optimisation vs improving your data pipeline.

Post reply on HN