Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google “We have no moat, and neither does OpenAI”

semianalysis.com

Post reply on HN
Print view View original

531–540 of 1001 posts

Re: Google “We have no moat, and neither does OpenAI”

#531

Post by rewgs »

dahwolf wrote: post #389

The current paradigm is that AI is a destination. A product you go to and interact with. That's not at all how the masses are going to interact with AI in the near future. It's going to be seamlessly integrated into every-day software. In Office/Google docs, at the operating system level (Android), in your graphics editor (Adobe), on major web platforms: search, image search, Youtube, the like. Since Google and other…

This is why I think Apple's direction of building a neural engine into the M1 architecture is low-key brilliant. It's just there and part of their API; as AI capabilities increase and the developer landscape solidifies, they can incrementally expand and improve its capabilities.

As always, Apple's focus is hardware-first, and I think it will once again pay off here.

Re: Google “We have no moat, and neither does OpenAI”

#532

Post by addandsubtract »

se4u wrote: post #506
eulers_secret wrote: post #331

Earlier quoted context omitted.

If you use the web interface (oobabooga), then training a LoRa is as easy as clicking the "training" tab, keeping all the defaults, and giving it a flat text file of your data. The defaults are sane enough to not begin undermining any instruction tuning too much. Takes 3-5 hours on a 3080 for 7B, 4bit model (and ~1KWh). So far I've trained 3: 2 on the entire text of ASOIAF (converted from e-books) and 1 on the Harry…

I am surprised that people aren't using google colab pro/pro+ in this context. You basically get access to multiple A100 for $10/month and with some simple javascript tricks, you can get a session to last for 24hrs at least. Pro+ is more expensive at $50/mo but it allows for more simplified background execution. if you are only just getting started and don't expect to be training for multiple months, then colab or ot…

Google has recently limited the pro plan to 100 compute units per month. Using an A100 on colab burns 13 units an hour. So you could be out of units within 8 hours. Not really the $10/month deal you're looking for.

Re: Google “We have no moat, and neither does OpenAI”

#533

Post by thereisnospork »

dahwolf wrote: post #389

The current paradigm is that AI is a destination. A product you go to and interact with. That's not at all how the masses are going to interact with AI in the near future. It's going to be seamlessly integrated into every-day software. In Office/Google docs, at the operating system level (Android), in your graphics editor (Adobe), on major web platforms: search, image search, Youtube, the like. Since Google and other…

I agree with your assertion that AI will seamlessly integrate into existing software and services but my expectation is that it will be unequivocally superior as a 3rd party integration[0]. People will get to know 'their AI' and vice versa. Why would I want Bard to recommend me a funny YouTube clip when my neutral assistant[1] has a far better understanding of my sense of humor? Bard can only ever learn from the context of interaction with google services -- something independent can pull from a larger variety of sources supersetting a locked system.

Nevermind more specialized tools that don't have the resources to develop their own competent AI - google might pull it off but adobe won't, and millions Saas and small programs won't even try. As another example, how could an Adobe AI ever have a better interpretation of 'Paint a pretty sunset in the style of Picasso' than a model which can access my photos, wallpapers, location, vacations, etc?

[0]Much how smart phones seamlessly integrate with automobiles via CarPlay and not GM-play. Once AI can use a mouse, if a person can integrate with a service an AI can do so on their behalf.

[1]Mind it's entirely possible it will be Apple or MSFT providing said 'neutral' AI.

Re: Google “We have no moat, and neither does OpenAI”

#535

Post by spiffytech »

Taek wrote: post #490
narrator wrote: post #396

Earlier quoted context omitted.

I think the problem with AI being everywhere and ubiquitous is that AI is the first technology in a very long time that requires non-trivial compute power. That compute power costs money. This is why you only get a limited number of messages every few hours from GPT4. It simply costs too much to be a ubiquitous technology. For example, the biggest LLama model only runs on an A100 that costs about $15,000 on ebay. The…

The biggest llama model has near 100% fidelity (its like 99.3%) at 4 bit quantization, which allows it to fit on any 40GB or 48GB GPU, which you can get for $3500. Or at about a 10x speed reduction you can run it on 128 GB of RAM for only around $250. The story is not anywhere near as bleak as you paint.

Something I haven't figured out: should I think about these memory requirements as comparable to the baseline memory an app uses, or like per-request overhead? If I needed to process 10 prompts at once, do I need 10x those memory figures?

Re: Google “We have no moat, and neither does OpenAI”

#536

Post by ekanes »

dahwolf wrote: post #389

The current paradigm is that AI is a destination. A product you go to and interact with. That's not at all how the masses are going to interact with AI in the near future. It's going to be seamlessly integrated into every-day software. In Office/Google docs, at the operating system level (Android), in your graphics editor (Adobe), on major web platforms: search, image search, Youtube, the like. Since Google and other…

Everything you say is true, and Google has cards left to play, but this is absolutely an existential threat to Google. How could it be otherwise?

For the first time in a very long time, people are open to a new search/answers engine. The game they won must now be replayed, and because it was so dominant, Google has nowhere to go but downwards.

Re: Google “We have no moat, and neither does OpenAI”

#537

Post by yellowapple »

jldugger wrote: post #501
TeMPOraL wrote: post #409

Earlier quoted context omitted.

Honestly, I can't see Google failing here. Like other tech giants, they're sitting on a ridiculously large war chest. Worst case, they can wait for the space to settle a bit and spend a few billion to buy the market leader. If AI really is an existential threat to their business prospects, spending their reserves on this is a no-brainer.

> Honestly, I can't see Google failing here. Like other tech giants, they're sitting on a ridiculously large war chest. Worst case, they can wait for the space to settle a bit and spend a few billion to buy the market leader. It seems incredibly likely that the FTC will block that. New leadership seems to be of the opinion that consumer harm is the wrong standard. Buying the competition with profits from a search mon…

> The article claims meta won but it seems like nvidia is the winner: everyone uses their chipsets for training, fine tuning and inference. And the more entrants and niche applications show up the more demand there is for their product.

Like the saying goes: during a gold rush, sell shovels.

Re: Google “We have no moat, and neither does OpenAI”

#538

Post by cush »

narrator wrote: post #396
dahwolf wrote: post #389

The current paradigm is that AI is a destination. A product you go to and interact with. That's not at all how the masses are going to interact with AI in the near future. It's going to be seamlessly integrated into every-day software. In Office/Google docs, at the operating system level (Android), in your graphics editor (Adobe), on major web platforms: search, image search, Youtube, the like. Since Google and other…

I think the problem with AI being everywhere and ubiquitous is that AI is the first technology in a very long time that requires non-trivial compute power. That compute power costs money. This is why you only get a limited number of messages every few hours from GPT4. It simply costs too much to be a ubiquitous technology. For example, the biggest LLama model only runs on an A100 that costs about $15,000 on ebay. The…

Yeah, today.

Re: Google “We have no moat, and neither does OpenAI”

#539

Post by newswasboring »

dahwolf wrote: post #389

The current paradigm is that AI is a destination. A product you go to and interact with. That's not at all how the masses are going to interact with AI in the near future. It's going to be seamlessly integrated into every-day software. In Office/Google docs, at the operating system level (Android), in your graphics editor (Adobe), on major web platforms: search, image search, Youtube, the like. Since Google and other…

> This so-called "competition" from open source is going to be free labor. Any winning idea ported into Google's products on short notice. Thanks open source!

How else, exactly, is open source supposed to work? Nobody wants to make their code GPL but everybody complains when companies use their code. I get that open source projects will like companies to contribute back, but shouldn't that go for everyone using this code? Like, I don't get what the proposed way of working is here.

Re: Google “We have no moat, and neither does OpenAI”

#540

Post by omeze »

bbor wrote: post #507
seydor wrote: post #156

Not only they have no moat, Open source models are uncensored and this is huge. Censorship is not just political , it cripples the product to basically an infantile stage and precludes so many applications. For once, it is a liability But this article doesn't state the very obvious: When will google (the inventor of Transformer, and "rightful" godfather of modern LLMs) , release a full open source, tinkerable model b…

My very naive opinion is that the best way to predict the big-picture actions of Google is a simple question: WWIitND - What Would IBM in the Nineties Do? In more direct terms, their sole, laser focus seems to be on maintaining short-term shareholder value, and I really don't trust the typical hedge fund manager to approve of any risky OSS moves for a project/tech that they're surely paying a LOT of attention to. Giv…

Yes, this reminds me of the story of transistors at IBM, when they had to pick between MOSFETs vs BJTs. MOSFETs were theoretically more scalable and what Intel eventually commercialized to great success. IBM had a lot of the best electrical engineers at the time, but chose to focus on BJTs because they supported their core product, mainframes. MOSFETs could theoretically scale better but without a clear line of sight to a product line or enhancement, they chose not to aggressively pursue MOSFET r&d. It makes sense, even in retrospect, because IBM didnt want to be a chip manufacturer.

Google doesn’t want to be an LLM manufacturer. But the benefits of having the industry center on your technical underpinnings are huge, s IBM found out eventually and as Google will, one way or the other. Meta understands this, at least

Post reply on HN