Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What does a leaked Google memo reveal about the future of AI?

economist.com

Post reply on HN
Print view View original

41–50 of 55 posts

Re: What does a leaked Google memo reveal about the future of AI?

#41

Post by UncleMeat »

mrtranscendence wrote: post #38
UncleMeat wrote: post #5

Earlier quoted context omitted.

> "Leaks" can be press releases. It'd be weird not to integrate the comments first if this was being done by PR. > But then how did the leak happen? Somebody thought the doc was interesting and shared it with a news outlet. I've read 100+ docs that read like this. Some sort of position paper designed to spark discussion, much of which happens in the comments alongside the doc. All of them are just the opinion of a pe…

Well, how do you tell the difference between a truly leaked internal document that reflects the opinion of one person, and a "leaked" document that just so happens to put forth a position that's beneficial to Google given its current standing in the LLM space? I think some skepticism is healthy here, as long as we understand that no one knows for sure.

In this case, the trick is to look at the document internally and see the comment history.

Re: What does a leaked Google memo reveal about the future of AI?

#42

Post by chaxor »

siliconc0w wrote: post #34

On the low end, google can sprinkle in well integrated cost-optimized 'pretty good' LLMs into all their products which already have billions of DAU. It's hard for OpenAI to compete here, their models have to be significantly better to encourage people to copy and paste into ChatGPT and yet still cost effective. The high end is more interesting in that I'm not sure Google has a response to Gpt4. Specialized Palm2 mode…

This idea that ChatGPT is better than the models that Google has is false; but I understand the perception. Deepmind has far better models than OpenAI, and the models OpenAI have made are based on the research done at Deepmind. For example, ChatGPT was not the first model trained by RLHF - that was Sparrow by Deepmind. The difference is that Google has been very disinterested in making products out of these systems, and didn't spend any effort to make them publicly accessible.

Deepmind is still the leader in AI, and they have much better models - it's just going to take a while for people who don't study ML to realize this because Google hasn't released any of their good models for public use yet (and yeah, I'm not counting Bard, etc because the real model was probably put together by one guy in an afternoon at Google, and the rest of the time working was marketing).

Re: What does a leaked Google memo reveal about the future of AI?

#43

Post by visarga »

chaxor wrote: post #40
akiselev wrote: post #29

Earlier quoted context omitted.

What are the chances that their copy is more useful than CommonCrawl? It’s been the target of SEO for decades - are they even capable of separating the good content from the crap anymore? Google may already be terminal

Their data may not be that much better; however Deepmind is still one of the best institutions that study AI fundamentals. OpenAI works hard and focused towards products, but the academic research is far better from Deepmind. So since Google hasn't been trying to put any AI into products really at all for the past 10 years, they're going to have to push a little bit to make the public facing interfaces and put polish…

They have no model on par with GPT-4.

Re: What does a leaked Google memo reveal about the future of AI?

#44

Post by visarga »

sitkack wrote: post #31
akiselev wrote: post #29

Earlier quoted context omitted.

What are the chances that their copy is more useful than CommonCrawl? It’s been the target of SEO for decades - are they even capable of separating the good content from the crap anymore? Google may already be terminal

Data pollution isn't the problem it is made out to be. You can even use a model trained on garbage data to relabel the original set and retrain. Google of all companies is sitting on exaflops of TPU capacity, they can retrain models larger than anyone else, faster than anyone else.

I agree, but it would probably require to use LLMs to filter or augment the raw text. They can afford it.

Re: What does a leaked Google memo reveal about the future of AI?

#45

Post by visarga »

siliconc0w wrote: post #34

On the low end, google can sprinkle in well integrated cost-optimized 'pretty good' LLMs into all their products which already have billions of DAU. It's hard for OpenAI to compete here, their models have to be significantly better to encourage people to copy and paste into ChatGPT and yet still cost effective. The high end is more interesting in that I'm not sure Google has a response to Gpt4. Specialized Palm2 mode…

> I honestly don't think OSS is a big factor business-wise

Lots of companies are evaluating fine-tuning strategies for <60B LLMs. Such models can be efficient, private, cheaper and adapted to the use case. They may not be generalists like GPT-4, but you need them for scaling up usage on high volume calls on the same task if you don't want to have to pay a fortune to OpenAI, or for processing confidential information such as in medical, B2B and other cases. Not to mention the fear that OpenAI might snoop on their private data and use it in their corpus.

Re: What does a leaked Google memo reveal about the future of AI?

#46

Post by AnonymousPlanet »

medoc wrote: post #36
AnonymousPlanet wrote: post #11

Earlier quoted context omitted.

I must have overlooked all those conferences on time travel that are on the same level as, e.g., AAAI. But yes, public perception is a different thing. Most people are totally baffled when I tell them that the first spam email was sent in 1978, because for a lot of people the internet (and with it email) popped into existence somewhere around 2004.

Not to nitpick but this would rather be around 199x when AOL joined the Internet.

Oh, it's a different time for any group of people you ask. For some it was AOL, for others the media coverage that picked up around 1995/1996. And some were late to the game and only noticed when every one around them got DSL in the early 2000's. I kind of picked 2004 at random. It represents a (sadly rather sizeable) group of people in rural areas who neither got DSL nor internet over cable before that time and therefore just didn't pay attention.

Re: What does a leaked Google memo reveal about the future of AI?

#47

Post by startupsfail »

skybrian wrote: post #30
mattlondon wrote: post #26

Earlier quoted context omitted.

> but there is no special training in being unbiased Citation needed! ;-)

Well, I didn't get any training like that when I worked there, but it was a long time ago and I don't know what they're teaching Nooglers these days. I suppose you might be referring to diversity training, but it's not the same thing.

Understanding and preventing unwanted biases is a part of basic training in science, statistics and experimental design.

Re: What does a leaked Google memo reveal about the future of AI?

#48

Post by startupsfail »

UncleMeat wrote: post #32
startupsfail wrote: post #9

Earlier quoted context omitted.

Does it matter what is the origin of that opinion? As long as it is a good insight into the situation and accurately reflects the state of affairs, it shouldn’t matter, right? And as to not being the leader at Google, this hardly matters. Being politically adept and leadership skills don’t necessarily correlate with having better feel and insights into the future technology. And not being the leader reduces filters,…

Google has nearly 200,000 employees. Almost every possible opinion is held by somebody within the company. Imagine this was written by Blake Lemoine, who believed that one of the earlier models was sentient. Would it be reasonable to present this as a company-wide belief?

No, but a company-wide belief doesn’t matter. Some 200,000 lemmings can be mistaken, along with the chief lemming. Or all of them can be right. The discussion here is regarding a particular opinion of a particular employee.

And what matters is how informed and honest that particular employee is. And if ultimately that employee is right or wrong. If there is some objective truth in his or her statement.

(And for a reference, I don’t think that these 200,000 Googlers or Lemoine were right or wrong, it feels like Lemoine was asking/answering a wrong question, which needs to be asked differently to have a well defined answer).

Re: What does a leaked Google memo reveal about the future of AI?

#49

Post by chaxor »

visarga wrote: post #43
chaxor wrote: post #40

Earlier quoted context omitted.

Their data may not be that much better; however Deepmind is still one of the best institutions that study AI fundamentals. OpenAI works hard and focused towards products, but the academic research is far better from Deepmind. So since Google hasn't been trying to put any AI into products really at all for the past 10 years, they're going to have to push a little bit to make the public facing interfaces and put polish…

They have no model on par with GPT-4.

I would heavily disagree with that - the only reason that you think that is that GPT-4 is available to the public for cheap, whereas Deepminds models have not been released publicly.
Post reply on HN