Live data from Hacker News

Do AI companies work?

benn.substack.com

141–150 of 457 posts

Re: Do AI companies work?

#141
post #32

Earlier quoted context omitted.

What on earth would cardiologists use a Large Language Model for, except drafting fluff for journals? Safely and effectively, that is. Dangerous and inappropriate is obviously a much wider set of possibilities.

Like some kind of linter? The cardiologist checks the ECG, compare with the LLM results and checks the difference. If it can reduce error rate by like 10%, that's already really good. My current stance on LLM is that it's good for stuff which is painful to generate, but easy to check (for you). It's easier/faster to read an email than to write it. If you're a domain expert, you can check the output, and so on. The da…

> The cardiologist checks the ECG, compare with the LLM results and checks the difference.

Perhaps you're confusing the acronym LLM (Large Language Model) with ML (Machine Learning)?

Analyzing electrocardiogram waveform data using a text-predictor LLM doesn't make sense: No matter how much someone invests in tweaking it to give semi-plausible results part of the time, it's fundamentally the wrong tool/algorithm for the job.

Re: Do AI companies work?

#142
post #25

Earlier quoted context omitted.

I am not among those smartest, so take my opinion with a mountain of salt. But I'm just not convinced that this is going in the direction of AGI. The recent advances are truly jaw dropping. It absolutely merits being investigated to the hilt. There is a very good chance that it will end up being a net profit. But intuitively they don't feel to me like they're getting more human. If anything I feel like the recent rou…

Did you criticize the turing test as being meaningless before it was easily passed by LLMs? if not i don't see how you can avoid updating on "this is getting more human" or at least "this is getting closer to intelligence" to avoid the human-bias

I never gave much thought to the Turing test one way or the other. It never struck me as especially informative.

I've always been more interested in the non-verbal aspects of human intelligence. I believe that "true AGI", whatever that is, is likely to be able to mimic sentient but non-verbal species. I'd like to see an AGI do what a dog or cat does.

LLMs are quite astonishing at mimicking something humans specifically do, the most "rational" parts of our brain. But they seem to jump past the basic, non-rational parts of the brain. And I don't think we'll see it as "true AGI" until it does that -- whatever that is.

I'm reminded of the early AI researchers who taught AI to play chess because it's what smart people do, but it couldn't do any of the things dumb people do. I think the biggest question right now is whether our present techniques are a misleading local maximum, or if we're on the right slope and just need to keep climbing.

Re: Do AI companies work?

#143

Earlier quoted context omitted.

That's like saying "how do you monetize the internet?" There are so many ways, it makes the question seem nonsensical. Ways to monetize AI so far: Metered APIs (OpenAI and others) Subscription products built on it (Copilot, ChatGPT, etc.) Using it as a feature to give products a competitive edge (Apple Intelligence, Tesla FSD) Selling the hardware (Nvidia)

20 years ago people asked that exact question. E-Commerce emerged. People knew the physical process of buying things would move online. Took some time. Sure, more things emerged but monetizing the Internet still remains about selling you something. What similar parallel can we think of for AI?

Assuming AI progress continues, AI could replace both Microsoft's biggest product, OS, and Google's biggest product, search and ads. And there is a huge tail end of things autonomous driving/flying, drug discovery, robotics, programming, healthcare etc.

Re: Do AI companies work?

#144

I think we are in the middle of a steep S-curve of technology innovation. It is far from plateauing and there are still a bunch of major innovations that are likely to shift things even further. Interesting time and these companies are riding a wild wave. It is likely some will actually win big, but most will die - similar to previous technology revolutions. The ones that win will win not just on technology, but on t…

The car wasn't a horse that was better, but a car has not changed drastically since they went mainstream. They've gotten better, more efficient, loaded with tech, but are still roughly 4 seats, 4 doors, 4 wheels, driven by petroleum. I know that this is a massive oversimplification, but I think we have seen the "shape" of LLMs\Gen AI\AI products already and it's all incremental improvements from here on out with more…

The big missing thing between both the metaphor in the OP's link and yours is that I just can't fathom any of these companies being able to raise a paying subscriber base that can actually cover the outrageous costs of this tech. It feels like a pipe dream.

Putting aside that I fundamentally don't think AGI is in the tech tree of LLM, if you will, that there's no route from the latter to the former: even if there is, even if it takes, I dunno, ten years: I just don't think ChatGPT is a compelling enough product to fund about $70 billion in research costs. And sure, they aren't having to yet thanks to generous input from various commercial and private interests but like... if this is going to be a stable product at some point, analogous to something like AWS, doesn't it have to... actually make some money?

Like sure, I use ChatGPT now. I use the free version on their website and I have some fun with AI dungeon and occasionally use generative fill in Photoshop. I paid for AI dungeon (for awhile, until I realized their free models actually work better for how I like to play) but am now on the free version. I don't pay for ChatGPT's advanced models, because nothing I've seen in the trial makes it more compelling an offering than the free version. Adobe Firefly came to me free as an addon to my creative cloud subscription, but like, if Adobe increased the price, I'm not going to pay for it. I use it because they effectively gave it to me for free with my existing purchase. And I've played with Copilot a bit too, but honestly found it more annoying than useful and I'm certainly not paying for that either.

And I realize I am not everyone and obviously there are people out there paying for it (I know a few in fact!) but is there enough of those people ready to swipe cards for... fancy autocomplete? Text generation? Like... this stuff is neat. And that's about where I put it for myself: "it's neat." OpenAI supposedly has 3.9 million subscribers right now, and if those people had to foot that 7 billion annual spend to continue development, that's about $150 a month. This product has to get a LOT, LOT better before I personally am ready to drop a tenth of that, let alone that much.

And I realize this is all back-of-napkin math here but still: the expenses of these AI companies seem so completely out of step with anything approaching an actual paying user base, so hilariously outstripping even the investment they're getting from other established tech companies, that it makes me wonder how this is ever, ever going to make so much as a dime for all these investors.

In contrast, I never had a similar question about cars, or AWS. The pitch of AWS makes perfect sense: you get a server to use on the internet for whatever purpose, and you don't have to build the thing, you don't need to handle HVAC or space, you don't need a last-mile internet connection to maintain, and if you need more compute or storage or whatever, you move a slider instead of having to pop a case open and install a new hard drive. That's absolutely a win and people will pay for it. Who's paying for AI and why?

Re: Do AI companies work?

#145
post #140

Earlier quoted context omitted.

The car wasn't a horse that was better, but a car has not changed drastically since they went mainstream. They've gotten better, more efficient, loaded with tech, but are still roughly 4 seats, 4 doors, 4 wheels, driven by petroleum. I know that this is a massive oversimplification, but I think we have seen the "shape" of LLMs\Gen AI\AI products already and it's all incremental improvements from here on out with more…

Feels like someone might have said this in 1981 about personal computers. "We've pretty much seen their shape. The IBM PC isn't fundamentally very different from the Apple II. Probably it's just all incremental improvements from here on out."

I would agree with your counter if it weren't for the realities of power usage, hardware constraints, evident diminishing returns on training larger models, and as always the fact that AI is still looking for the problem it solves, aside from mass employment.

Computers solved a tangible problem in every area of life, AI is being forced everywhere and is arguably failing to make a big gain in areas that it should excel.

Re: Do AI companies work?

#146

This is like when VCs were funding all kinds of ride share, bike share, food delivery, cannabis delivery, and burning money so everyone gets subsidized stuff while the market figures out wtf is going on. I love it. More goodies for us

Where I live the ridesharing/delivering startups didn't bring goodies, they just made everything worse. They destroyed the Taxi industry, I used to be able to just walk out to the taxi rank and get in the first taxi, but not anymore. Now I have to organize it on an app or with a phone call to a robot, then wait for the car to arrive, and finally I have to find the car among all the others that other people called. Fo…

For where I live (Asia), I disagree with both of these examples.

Getting a taxi was awful before ride-sharing apps. You'd have to walk to a taxi stop, or wait on the side of the road and hope you could hail one. Once the ride-sharing apps came in, suddenly getting a ride became a lot simpler. Our taxi companies are still alive, though they have their own apps now -- something that wouldn't have happened without competition -- and they also work together with the ride-hailing companies as a provider. You could still hail taxis or get them from stops too, though that isn't recommended given that they might try to run the meter by taking a longer route.

For food delivery, before the apps, most places didn't deliver food. Nowadays, more places deliver. Even if a place already had their own delivery drivers, they didn't get rid of them. We get a choice, to use the app or to use the restaurant's own delivery. Usually the app is better for smaller meals since it has a lower minimum order amount, but the restaurant provides faster delivery for bigger orders.

Re: Do AI companies work?

#147
I lead an applied AI research team where I work - which is a mid-sized public enterprise products company. I've been saying this in my professional circles quite often.

We talk about scaling laws, superintelligence, AGI etc. But there is another threshold - the ability for humans to leverage super-intelligence. It's just incredibly hard to innovate on products that fully leverage superintelligence.

At some point, AI needs to connect with the real world to deliver economically valuable output. The ratelimiting step is there. Not smarter models.

In my mind, already with GPT-4, we're not generating ideas fast enough on how best to leverage it.

Getting AI to do work involves getting AI to understand what needs to be done from highly bandwidth constrained humans using mouse / keyboard / voice to communicate.

Anyone using a chatbot already has felt the frustration of "it doesn't get what I want". And also "I have to explain so much that I might as well just do it myself"

We're seeing much less of "it's making mistakes" these days.

If we have open-source models that match up to GPT-4 on AWS / Azure etc, not much point to go with players like OpenAI / Anthropic who may have even smarter models. We can't even use the dumber models fully.

Re: Do AI companies work?

#148
post #140

Earlier quoted context omitted.

The car wasn't a horse that was better, but a car has not changed drastically since they went mainstream. They've gotten better, more efficient, loaded with tech, but are still roughly 4 seats, 4 doors, 4 wheels, driven by petroleum. I know that this is a massive oversimplification, but I think we have seen the "shape" of LLMs\Gen AI\AI products already and it's all incremental improvements from here on out with more…

Feels like someone might have said this in 1981 about personal computers. "We've pretty much seen their shape. The IBM PC isn't fundamentally very different from the Apple II. Probably it's just all incremental improvements from here on out."

What do you think hasn't been?

I think the big game changer in the PC space was graphics cards, but since their introduction, it has all been incremental improvement -- at first, pretty fast, then... slower. Much like CPU improvements, although those started earlier.

I can't think of a point where the next generation of PCs was astoundingly different from the prior one... just better. It used to be that they were reliably faster or more capable almost every year, now the rate of improvements is almost negligible. (Yes, graphics are getting better, but not very fast if you aren't near the high end.)

Re: Do AI companies work?

#149

The fundamental question is how to monetize AI? I see 2 paths: - Consumers - the Google way: search and advertise to consumers - Businesses - the AWS way: attrack businesses to use your API and lock them in The first is fickle. Will OpenAI become the door to the Internet? You'll need people to stop using Google Search and rely on ChatGPT for that to happen. Will become a commodity. Short term you can charge a subscri…

I don't see how you charge enough for the second path to make the economics work.

Re: Do AI companies work?

#150

Earlier quoted context omitted.

> I think we are in the middle of a steep S-curve of technology innovation We are? What innovation? What do we need innovation for ? What present societal problems can tech innovation possibly address? Surely none of the big ones, right? So then is it fit to call technological change - 'innovation'? I'd agree that LLMs improve upon having to read Wikipedia for topics I'm interested in but would investing billions in…

I'm not sure the Wikipedia example is a strong one as that site has it's own serious problems with "abusive monopolies" in its moderator cliques and biases (as with any social platform). At least with the current big AI players there is the potential for differentiation through competition. Unless there is some similar initiative with the Wikipedias, the problem of single supplier dominance is a difficult one to see…

I can solve Wikipedia's woes quite easily - Wikipedia should limit itself to math, science, engineering, medicine, physics, chemistry, geography and other disciplines that are not at all in dispute.

Politics, history, religion and other topics of conversation that are matters of opinion, taste and state sponsored propaganda need to be off limits.

Its mission ought to be to provide a PhD level education in all technical fields, not engage in shortening historical events and/or opinions/preferences/beliefs down to a few pages and disputing which pages need to be left in or out. Let fools engage in that task on their own time.

Post reply on HN