Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Do AI companies work?

benn.substack.com

Post reply on HN
Print view View original

91–100 of 457 posts

Re: Do AI companies work?

#91

Post by bbqfog »

highfrequency wrote: post #89

Enjoyed the article and thought many of the points were good. Here's a counterargument. > In other words, the billions that AWS spent on building data centers is a lasting defense. The billions that OpenAI spent on building prior versions of GPT is not, because better versions of it are already available for free on Github. The money that OpenAI spends on renting GPUs to build the next model is not what builds the mo…

The infrastructure hardware and software is a commodity. Any real moat comes from access to data. I think we've seen data close up quite a bit since people realized that you can train LLMs with it, so I don't know that OpenAI's data access is better than when they trained GPT 4. In fact, it's probably worse unless they've cut independent deals with massive data providers.

Re: Do AI companies work?

#93

Post by tempfile »

akomtu wrote: post #71
llm_trw wrote: post #31

I've found that everything that works stops being called AI. Logic programming? AI until SQL came out. Now it's not AI. OCR, computer algebra systems, voice recognition, checkers, machine translation, go, natural language search. All solved, all not AI any more yet all were AI before they got solved by AI researchers. There's even a name for it: https://en.m.wikipedia.org/wiki/AI_effect?utm_source=perplex...

People innately believe that intelligence isn't an algorithm. When a complex problem presents itself for the first time, people think "oh, this must be so complex that no algorithm can solve it, only AI," and when an algorithmic solution is found, people realise that the problem isn't that complex. Indeed, if AI was an algorithm, imagine what would it feel like to be like one: at every step of your thinking process y…

The other option you don't mention is "algorithms can solve it, but they do something different to what humans do". That's what happened with Go and Chess, for example.

I agree with you that people don't consider intelligence as fundamentally algorithmic. But I think the appeal of algorithmic intelligence comes from the fact that a lot of intelligent behaviours (strategic thinking, decomposing a problem into subproblems, planning) are (or at least feel) algorithmic.

Re: Do AI companies work?

#94

Post by actsasbuffoon »

jsnell wrote: post #60
JumpCrisscross wrote: post #52

Earlier quoted context omitted.

> it seems irrational to start putting in even the fewer billions of dollars now, knowing the above two points This doesn’t follow. One, there are cash flows you can extract in the interim—standing in place is potentially lucrative. And two, we don’t know if the curve continues until infinity or asymptotes. If it asymptotes, being the first at the asymptote means owning a profitable market. If it doesn’t, you’re goin…

Why would being the first at the (horizontal) asymptote mean owning a profitable market? In six months, somebody else is at the asymptote as well. If different models are mostly interchangable quality-wise, then the decision will mostly be made on price. For a market to be both winner-take-all as well as lucrative, you need some kind of a feedback cycle from network effets, economies of scale, or maybe even really br…

Excellent point, but now consider the availability of open source models as well. They’re not as good as the frontier models, but at some point they get close enough, right?

I run Llama 3.x locally on my 5+ year old GPU. It’s pretty decent. It’s not as smart as GPT-4 or Claude, but it’s good enough for a lot of what I do, and most importantly there’s no cap on the number of prompts I can send in a given timeframe.

For many consumers, it’s fine to be a little bit behind the cutting edge. My smartphone is 2 models behind now. It’s fast enough, the camera is good enough, and the battery is holding up just fine. I don’t feel any need to fork over $1,000 for a new one right now.

I think the same phenomenon will become more common for AI enthusiasts. It’s hard to beat free.

And what happens to the economics of building frontier models then? I suspect a lot of air is going to be released from the AI bubble in the next couple of years.

Re: Do AI companies work?

#95

Post by benrutter »

analog31 wrote: post #40
llm_trw wrote: post #31

I've found that everything that works stops being called AI. Logic programming? AI until SQL came out. Now it's not AI. OCR, computer algebra systems, voice recognition, checkers, machine translation, go, natural language search. All solved, all not AI any more yet all were AI before they got solved by AI researchers. There's even a name for it: https://en.m.wikipedia.org/wiki/AI_effect?utm_source=perplex...

On an amusing note, I've read something similar: Everything that works stops being called philosophy. Science and math being the two familiar examples.

Just in case anyone's curious, this is from Bertrand Russell's "the history of philosophy".

> As soon as definite knowledge concerning any subject becomes possible, this subject ceases to be called philosophy, and becomes a separate science.

I'm not actually sure I agree with it, especially in light of less provable schools of science like string theory or some branches of economics, but it's a great idea.

Re: Do AI companies work?

#96

Post by gbalduzzi »

mdavidn wrote: post #87
akomtu wrote: post #71

Earlier quoted context omitted.

People innately believe that intelligence isn't an algorithm. When a complex problem presents itself for the first time, people think "oh, this must be so complex that no algorithm can solve it, only AI," and when an algorithmic solution is found, people realise that the problem isn't that complex. Indeed, if AI was an algorithm, imagine what would it feel like to be like one: at every step of your thinking process y…

Is that not the human experience? I have no “agency” over the next thought to pop into my head. I “feel” like I can choose where to focus attention, but that too is a predictable outcome arising from the integration of my embryology, memories, and recently reinforced behaviors. “I” am merely an observer of my own mental state. But that is an uncomfortable idea for most people.

While technically true, the difference is that we don't know in details how that process works and we can't predict it.

There is not much of a difference in considering it deterministic or not, because we can't determine it.

Re: Do AI companies work?

#97

Post by ninetyninenine »

bhouston wrote: post #90

I think we are in the middle of a steep S-curve of technology innovation. It is far from plateauing and there are still a bunch of major innovations that are likely to shift things even further. Interesting time and these companies are riding a wild wave. It is likely some will actually win big, but most will die - similar to previous technology revolutions. The ones that win will win not just on technology, but on t…

Likely we are because the actualized end result already exists: the human brain.

The fact that human intelligence exists means that the idea of human level intelligence is not a pipe dream.

The question is whether or not the basic underlying technology of the LLM can achieve that level of intelligence.

Re: Do AI companies work?

#98

Post by baq »

flappyeagle wrote: post #2

This is like when VCs were funding all kinds of ride share, bike share, food delivery, cannabis delivery, and burning money so everyone gets subsidized stuff while the market figures out wtf is going on. I love it. More goodies for us

First VC to exit a nuclear reactor profitably is going to be something.

Re: Do AI companies work?

#99

Post by ninetyninenine »

llm_trw wrote: post #31

I've found that everything that works stops being called AI. Logic programming? AI until SQL came out. Now it's not AI. OCR, computer algebra systems, voice recognition, checkers, machine translation, go, natural language search. All solved, all not AI any more yet all were AI before they got solved by AI researchers. There's even a name for it: https://en.m.wikipedia.org/wiki/AI_effect?utm_source=perplex...

Yeah but honestly we all know LLMs are different then say some chess ai.

You can thank social media for dumbing down a human technological milestone in artificial intelligence. I bet if there was social media around when we landed on the moon you’d get a lot of self important people rolling their eyes at the whole thing too.

Re: Do AI companies work?

#100

Post by snapcaster »

jfengel wrote: post #25
AI_beffr wrote: post #6

the hype will never die. all the smartest people in industry and government believe that there is a very high probability that this technology is near the edge of starting the AGI landslide. you dont need AGI to start the AGI landslide, you just need AI tools that are smart enough to automate the process of discovering and building the first AGI models. every conceivable heuristic indicates that we are near the edge.…

I am not among those smartest, so take my opinion with a mountain of salt. But I'm just not convinced that this is going in the direction of AGI. The recent advances are truly jaw dropping. It absolutely merits being investigated to the hilt. There is a very good chance that it will end up being a net profit. But intuitively they don't feel to me like they're getting more human. If anything I feel like the recent rou…

Did you criticize the turing test as being meaningless before it was easily passed by LLMs? if not i don't see how you can avoid updating on "this is getting more human" or at least "this is getting closer to intelligence" to avoid the human-bias
Post reply on HN