What does a leaked Google memo reveal about the future of AI? • Hacker News Board

Re: What does a leaked Google memo reveal about the future of AI?

#31

Post by sitkack » Sun, May 14, 2023, 5:47 PM UTC

akiselev wrote: post #29

sitkack wrote: post #25

Earlier quoted context omitted.

The doc is too short and not puffy to be a normal google doc. There is a high likelihood this was placed to proactively shake off any regulation or scruitiny while they get the deck chairs in order. They might not appear to have a moat, but they definitely have an advantage that they can keep for 5+ years. They also have a "copy of the internet" that no one else does.

What are the chances that their copy is more useful than CommonCrawl? It’s been the target of SEO for decades - are they even capable of separating the good content from the crap anymore? Google may already be terminal

Data pollution isn't the problem it is made out to be. You can even use a model trained on garbage data to relabel the original set and retrain.

Google of all companies is sitting on exaflops of TPU capacity, they can retrain models larger than anyone else, faster than anyone else.

Re: What does a leaked Google memo reveal about the future of AI?

#32

Post by UncleMeat » Sun, May 14, 2023, 6:15 PM UTC

startupsfail wrote: post #9

UncleMeat wrote: post #3

All the presentation of this has been weird. "Leaked Google memo" is true in a sense, but the doc was written by one person who isn't the primary leading force within relevant orgs at Google. The headline is more accurately described as "what does the opinion of one Googler reveal..."

Does it matter what is the origin of that opinion? As long as it is a good insight into the situation and accurately reflects the state of affairs, it shouldn’t matter, right? And as to not being the leader at Google, this hardly matters. Being politically adept and leadership skills don’t necessarily correlate with having better feel and insights into the future technology. And not being the leader reduces filters,…

Google has nearly 200,000 employees. Almost every possible opinion is held by somebody within the company.

Imagine this was written by Blake Lemoine, who believed that one of the earlier models was sentient. Would it be reasonable to present this as a company-wide belief?

Re: What does a leaked Google memo reveal about the future of AI?

#33

Post by UncleMeat » Sun, May 14, 2023, 6:19 PM UTC

sitkack wrote: post #25

gumby wrote: post #24

Was it leaked or “leaked”? Google has a good, two part strategy available. First is to neutralize the threat: support the open source efforts covertly and/or overtly, as those efforts undercut the revenue of OpenAI and other potential competitors for the base technology. This is what they did with Android: at a minimum all they had to do was ensure that Apple did not become a monopoly, which they accomplished (the se…

The doc is too short and not puffy to be a normal google doc. There is a high likelihood this was placed to proactively shake off any regulation or scruitiny while they get the deck chairs in order. They might not appear to have a moat, but they definitely have an advantage that they can keep for 5+ years. They also have a "copy of the internet" that no one else does.

> The doc is too short and not puffy to be a normal google doc.

Loads of internal position papers read exactly like this. I've written docs that write like this.

Re: What does a leaked Google memo reveal about the future of AI?

#34

Post by siliconc0w » Sun, May 14, 2023, 6:36 PM UTC

On the low end, google can sprinkle in well integrated cost-optimized 'pretty good' LLMs into all their products which already have billions of DAU. It's hard for OpenAI to compete here, their models have to be significantly better to encourage people to copy and paste into ChatGPT and yet still cost effective. The high end is more interesting in that I'm not sure Google has a response to Gpt4. Specialized Palm2 models are allegedly SOTA in certain verticals but they're not generally available so it's hard to say. It's possible Google could offer a beefy model to Google One subscribers or something but they seem to be focused on the low end for now at least in the B2C arena with specialized models for B2B. I honestly don't think OSS is a big factor business-wise but it's nice to see and certainly enables more participation in the ecosystem.

Re: What does a leaked Google memo reveal about the future of AI?

#35

Post by Animats » Sun, May 14, 2023, 7:16 PM UTC

Wasn't this note from someone at Google discussed on HN last week, before The Economist picked it up?

It is a huge issue for Google. Not because they're behind, but because chat engines don't seem to map well to an ad-supported model. Unless they talk like the Youtubers who start with "But first, let me tell you about our sponsor, FBX Crypto".

They do map well to a surveillance model. That's already getting regulator attention. Look what happened to TikTok, which is really good at focusing on a user's interests. Chat engines focus like that. They have to, to work well.

Re: What does a leaked Google memo reveal about the future of AI?

#36

Post by medoc » Sun, May 14, 2023, 7:26 PM UTC

AnonymousPlanet wrote: post #11

scrollaway wrote: post #6

Earlier quoted context omitted.

> Because 70 years of named work based on centuries of interest and efforts can remain unseen if one really tries. You’re really trying to be uncharitable eh? We’ve been talking about time travel for millennia. But I can guarantee you that if tomorrow we figure out a way for most people to actually reliably perform it and play with it, it will be defined as “bursting into the public consciousness”.

I must have overlooked all those conferences on time travel that are on the same level as, e.g., AAAI. But yes, public perception is a different thing. Most people are totally baffled when I tell them that the first spam email was sent in 1978, because for a lot of people the internet (and with it email) popped into existence somewhere around 2004.

Not to nitpick but this would rather be around 199x when AOL joined the Internet.

Re: What does a leaked Google memo reveal about the future of AI?

#37

Post by flutas » Sun, May 14, 2023, 8:10 PM UTC

Animats wrote: post #35

Wasn't this note from someone at Google discussed on HN last week, before The Economist picked it up? It is a huge issue for Google. Not because they're behind, but because chat engines don't seem to map well to an ad-supported model. Unless they talk like the Youtubers who start with "But first, let me tell you about our sponsor, FBX Crypto". They do map well to a surveillance model. That's already getting regulator…

Yep.

Re: What does a leaked Google memo reveal about the future of AI?

#38

Post by mrtranscendence » Sun, May 14, 2023, 8:15 PM UTC

UncleMeat wrote: post #5

FooBarBizBazz wrote: post #4

Earlier quoted context omitted.

"Leaks" can be press releases. "Oops, you weren't supposed to see that" can make the message seem more credible. Is this Google preemptively fending off accusations of monopoly in an important space? On the other hand, if Google is perceived as weak, then its share price will be negatively affected. So maybe not. Maybe it's just what some guy wrote. But then how did the leak happen? Somebody chose to publicize this p…

> "Leaks" can be press releases. It'd be weird not to integrate the comments first if this was being done by PR. > But then how did the leak happen? Somebody thought the doc was interesting and shared it with a news outlet. I've read 100+ docs that read like this. Some sort of position paper designed to spark discussion, much of which happens in the comments alongside the doc. All of them are just the opinion of a pe…

Well, how do you tell the difference between a truly leaked internal document that reflects the opinion of one person, and a "leaked" document that just so happens to put forth a position that's beneficial to Google given its current standing in the LLM space? I think some skepticism is healthy here, as long as we understand that no one knows for sure.

Re: What does a leaked Google memo reveal about the future of AI?

#39

Post by fauigerzigerk » Sun, May 14, 2023, 8:24 PM UTC

startupsfail wrote: post #9

UncleMeat wrote: post #3

All the presentation of this has been weird. "Leaked Google memo" is true in a sense, but the doc was written by one person who isn't the primary leading force within relevant orgs at Google. The headline is more accurately described as "what does the opinion of one Googler reveal..."

Does it matter what is the origin of that opinion? As long as it is a good insight into the situation and accurately reflects the state of affairs, it shouldn’t matter, right? And as to not being the leader at Google, this hardly matters. Being politically adept and leadership skills don’t necessarily correlate with having better feel and insights into the future technology. And not being the leader reduces filters,…

>Does it matter what is the origin of that opinion? As long as it is a good insight into the situation and accurately reflects the state of affairs, it shouldn’t matter, right?

It does matter a great deal whether a particular opinion is held by someone with the power to act on it or by someone who can only try to influence those who can act on it.

Re: What does a leaked Google memo reveal about the future of AI?

#40

Post by chaxor » Sun, May 14, 2023, 9:50 PM UTC

akiselev wrote: post #29

sitkack wrote: post #25

Earlier quoted context omitted.

The doc is too short and not puffy to be a normal google doc. There is a high likelihood this was placed to proactively shake off any regulation or scruitiny while they get the deck chairs in order. They might not appear to have a moat, but they definitely have an advantage that they can keep for 5+ years. They also have a "copy of the internet" that no one else does.

What are the chances that their copy is more useful than CommonCrawl? It’s been the target of SEO for decades - are they even capable of separating the good content from the crap anymore? Google may already be terminal

Their data may not be that much better; however Deepmind is still one of the best institutions that study AI fundamentals. OpenAI works hard and focused towards products, but the academic research is far better from Deepmind. So since Google hasn't been trying to put any AI into products really at all for the past 10 years, they're going to have to push a little bit to make the public facing interfaces and put polish on things, but the underlying models they have made are far better than OpenAI (just not the ones made publicly accessible to anyone).