Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google “We have no moat, and neither does OpenAI”

semianalysis.com

Post reply on HN
Print view View original

691–700 of 1001 posts

Re: Google “We have no moat, and neither does OpenAI”

#691

Post by codethief »

1vuio0pswjnm7 wrote: post #522
dahwolf wrote: post #389

The current paradigm is that AI is a destination. A product you go to and interact with. That's not at all how the masses are going to interact with AI in the near future. It's going to be seamlessly integrated into every-day software. In Office/Google docs, at the operating system level (Android), in your graphics editor (Adobe), on major web platforms: search, image search, Youtube, the like. Since Google and other…

"Any winning idea ported into Google's products on short notice." Imagine for a moment, in a different universe, in a different galaxy, another planet is ostensibly a mirror image of Earth, evolving along the same trajectory. However on this hypothetical planet, anything is possible . This has resulted in some interesting differences. The No Google License Neither Google, its subsidiaries, business partners nor its a…

Let's call it the Underdog License. It must not be used by any of the top N tech companies in terms of market share and/or market capitalization.

Re: Google “We have no moat, and neither does OpenAI”

#692

Post by wing-_-nuts »

codethief wrote: post #689
wing-_-nuts wrote: post #663

Earlier quoted context omitted.

There are two things that make a good LLM. The amount of data available for training, and the amount of compute available. Google's bard sucks in comparison to Open AI, and even compared to Bing. It's pretty clear that GPT4 has some secret sauce that's giving them a competitive edge. I also don't think that Open Source LLMs are that big of a threat, for exactly this reason. They will always be behind on the amount of…

Did you read the article? It refutes almost every claim you're making and, I must say, rather convincingly so.

I'll admit, I skimmed it. I went back and re-read it, and the timeline of events was especially shocking. I still think the big models hold an edge, simply because it will most likely be better at handling edge cases, but wow, my days of underestimating the oss llms are certainly coming to a middle

Re: Google “We have no moat, and neither does OpenAI”

#693

Post by agitator »

kweingar wrote: post #659
safety1st wrote: post #591

Earlier quoted context omitted.

There are no guarantees about who will or won't own the future, just the observation that disruptive technology makes everyone's fate more volatile. Big tech companies like Google have a lot of in-built advantages, but they're notoriously bad at executing on pivots which fundamentally alter or commoditize their core business. If that wasn't true we'd all be using Microsoft phones (or heck, IBM PCs AND phones). In Goo…

> In Google's case they are still really focused on search whereas LLMs arguably move the focus to answers. I would love to see what proportion of searches are questions that would benefit from natural language answers. The huge majority of my searches would not be improved by LLMs and in fact would probably be made worse. “Thai food near me”, “IRS phone number”, “golang cmp documentation”

And isn’t that the problem?

Why do I need to translate my question into an optimal set of keywords that will give me what I want while minimizing unwanted results? Google search was a great stepping stone and connects you with the web, but it’s broken in many ways when it comes to what value we are really trying to extract.

A machine that can hone in on what I’m getting at in an intuitive sense while having all of human data available to generate a response is so much more powerful.

Re: Google “We have no moat, and neither does OpenAI”

#694

Post by jabradoodle »

mattferderer wrote: post #451
irrational wrote: post #446

Earlier quoted context omitted.

It being everywhere worries me a lot. It outputs a lot of false information and the typical person doesn’t have the time or inclination to vet the output. Maybe this is a problem that will be solved. I’m not optimistic on that front.

Same can be said about the results that pop up on your favorite search engine or asking other people questions. If anything advances in AI & search tech will do a better job at providing citations that agree & disagree with the results given. But this can be a turtles all the way down problem.

Low quality blogs etc stand out as low quality, LLMs can eloquently state truths with convincing sounding nonsense sprinkled through out. It's a different problem and many people already take low quality propaganda at face value.

Re: Google “We have no moat, and neither does OpenAI”

#695

Post by JohnFen »

kmmlng wrote: post #653
safety1st wrote: post #591

Earlier quoted context omitted.

There are no guarantees about who will or won't own the future, just the observation that disruptive technology makes everyone's fate more volatile. Big tech companies like Google have a lot of in-built advantages, but they're notoriously bad at executing on pivots which fundamentally alter or commoditize their core business. If that wasn't true we'd all be using Microsoft phones (or heck, IBM PCs AND phones). In Goo…

I feel like search still has its place. New information is being generated all the time. I want to be able to access it without having to retrain my LLM. It's also easier to validate that my search results are real. With LLMs, you never know if the answers are hallucinated or real. Where LLMs really shine is in understanding what I actually want. Where search still gives me many irrelevant answers, LLMs just get my q…

> I feel like search still has its place.

I sure hope so. Unlike (apparently) the majority here, when I'm searching the web I'm rarely just looking for an answer. I'm researching. What I want is a list of possible resources to investigate. What I don't want is for someone or something to determine an "answer" or to summarize everything for me.

I hope my use case isn't tossed aside.

Re: Google “We have no moat, and neither does OpenAI”

#696

Post by [deleted] »

bsaul wrote: post #602
safety1st wrote: post #591

Earlier quoted context omitted.

There are no guarantees about who will or won't own the future, just the observation that disruptive technology makes everyone's fate more volatile. Big tech companies like Google have a lot of in-built advantages, but they're notoriously bad at executing on pivots which fundamentally alter or commoditize their core business. If that wasn't true we'd all be using Microsoft phones (or heck, IBM PCs AND phones). In Goo…

in 1995 people understood very well what internet was going to become. The technology just wasn't there yet, but every kid and parents remember very well the sound of that modem and the phone lines beeing busy. That memo would have been prescient if made 5 years before.

[deleted]

Re: Google “We have no moat, and neither does OpenAI”

#697

Post by [deleted] »

Taek wrote: post #490
narrator wrote: post #396

Earlier quoted context omitted.

I think the problem with AI being everywhere and ubiquitous is that AI is the first technology in a very long time that requires non-trivial compute power. That compute power costs money. This is why you only get a limited number of messages every few hours from GPT4. It simply costs too much to be a ubiquitous technology. For example, the biggest LLama model only runs on an A100 that costs about $15,000 on ebay. The…

The biggest llama model has near 100% fidelity (its like 99.3%) at 4 bit quantization, which allows it to fit on any 40GB or 48GB GPU, which you can get for $3500. Or at about a 10x speed reduction you can run it on 128 GB of RAM for only around $250. The story is not anywhere near as bleak as you paint.

[deleted]

Re: Google “We have no moat, and neither does OpenAI”

#698

Post by scyzoryk_xyz »

dahwolf wrote: post #389

The current paradigm is that AI is a destination. A product you go to and interact with. That's not at all how the masses are going to interact with AI in the near future. It's going to be seamlessly integrated into every-day software. In Office/Google docs, at the operating system level (Android), in your graphics editor (Adobe), on major web platforms: search, image search, Youtube, the like. Since Google and other…

OpenAI is more of a lab than a company though, no?

Aren’t they, in some sense, kind of like that lab division that invented the computer mouse? Or for that matter, any other laboratory that made significant breakthroughs but left the commercialization to others?

It would make sense to me what you’re describing. Only, we will probably be laughing from the future the extent of our current imagination with this stuff is still limited to GUI’s, excels and docs.

Re: Google “We have no moat, and neither does OpenAI”

#699

Post by 1vuio0pswjnm7 »

1vuio0pswjnm7 wrote: post #522
dahwolf wrote: post #389

The current paradigm is that AI is a destination. A product you go to and interact with. That's not at all how the masses are going to interact with AI in the near future. It's going to be seamlessly integrated into every-day software. In Office/Google docs, at the operating system level (Android), in your graphics editor (Adobe), on major web platforms: search, image search, Youtube, the like. Since Google and other…

"Any winning idea ported into Google's products on short notice." Imagine for a moment, in a different universe, in a different galaxy, another planet is ostensibly a mirror image of Earth, evolving along the same trajectory. However on this hypothetical planet, anything is possible . This has resulted in some interesting differences. The No Google License Neither Google, its subsidiaries, business partners nor its a…

Neither Alphabet, Google nor their successors, subsidiaries, affiliates, business partners, academic collaborators or parent companies may use this software; all of the foregoing are specifically prohibited from any use of this software.

Re: Google “We have no moat, and neither does OpenAI”

#700

Post by jxmorris12 »

bionhoward wrote: post #630
keenon wrote: post #581

This is so indicative of Google culture missing the point. The idea of spending $10M training a single model is treated as a casual reality. But “tHaNk GoOdNeSs those generous open source people published their HiGh QuAlItY datasets of ten thousand examples each. Otherwise we’d have no way of creating datasets like that…” :| the sustainable competitive advantage has been and will continue to be HUGE PROPRIETARY DATAS…

One way to push back on the data argument is to consider the progress DeepMind made with self play. Perhaps Bard can self-dialogue and achieve superhuman results. I won’t be surprised. Plus the underlying architecture is dense. Sparse transformers are a major upgrade. That’s only one of many upgrades you can make. There is still a lot of headroom and IMHO GPT-4 already implements AGI if you give it the right context

Self-play works for eg Go because there’s a perfect simulator of the game – which gives at least one very clear training signal, winning. There’s no simulator for conversation, no winning, and no training signal. Self-dialog doesn’t make any sense
Post reply on HN