Why did Google Brain exist? • Hacker News Board

Re: Why did Google Brain exist?

#31

Post by renewiltord » Wed, Apr 26, 2023, 6:21 PM UTC

marricks wrote: post #12

dekhn wrote: post #2

because once Jeff Dean had solved Google's maslow problems (scaling web search, making ads profitable, developing high performance machine learning systems) he wanted to return to doing academic-style research, but with the benefit of Google's technical and monetary resources, and not part of X, which never produces anything of long-term value. I know for sure he wanted to make an impact in medical AI and felt that b…

[flagged]

These safety people guarantee a useless product that never does unsafe things. ChatGPT proved that you can have a product do unsafe things and still be useful if you put a disclaimer on it. Overall, as a user, I couldn't give a damn if things are unsafe by the definition of this style of ethicist. They were a ZIRP and my life is better for their absence.

Re: Why did Google Brain exist?

#32

Post by marricks » Wed, Apr 26, 2023, 6:21 PM UTC

dekhn wrote: post #20

marricks wrote: post #12

Earlier quoted context omitted.

[flagged]

That's a very simplified version of the story, but I would say that Dean greatly reduced his stature when he defended Megan Kacholia for her abrupt termination of Timnit. Note that Timnit was verbally abusive to Jeff's reports, anybody who worked there could see what she was posting to internal group discussions, so her time at Google was limited, but most managers would say that she should had at least been put on a…

Google has since fired other folks on her team and was in crisis mode to protect Dean. Like, I’m not really going to give them the benefit of the doubt on this.

When people brought Dean up Timnit came up as something to consider, it’s interesting to see how all anyone has to say in these threads is reverence towards him. People should try to see the whole picture.

Re: Why did Google Brain exist?

#33

Post by joseph_grobbles » Wed, Apr 26, 2023, 6:24 PM UTC

vl wrote: post #4

>PyTorch/Nvidia GPUs easily overtaking TensorFlow/Google TPUs. TF lost to PyTorch, and this is Google’s fault - TF APIs are both insane and badly documented. But nothing comes close to performance of Google’s TPU exaflop mega-clusters. Nvidia is not even in the same ballpark.

[dead]

Re: Why did Google Brain exist?

#34

Post by DavidSJ » Wed, Apr 26, 2023, 6:24 PM UTC

> … the publication of key research like LSTMs in 2014 …

Minor nitpick, but LSTMs date to 1997 and were not invented by Google. [1]

[1] Hochreiter and Schmidhuber (1997). Long short-term memory. https://ieeexplore.ieee.org/abstract/document/6795963

Re: Why did Google Brain exist?

#35

Post by amoss » Wed, Apr 26, 2023, 6:30 PM UTC

> Today, thought leaders casually opine on how and where ML will be useful, and MBAs feel like this is an acceptable substitute for expert opinion.

Sounds like standard operating procedure.

Re: Why did Google Brain exist?

#36

Post by Silverback_VII » Wed, Apr 26, 2023, 6:36 PM UTC

marricks wrote: post #12

dekhn wrote: post #2

because once Jeff Dean had solved Google's maslow problems (scaling web search, making ads profitable, developing high performance machine learning systems) he wanted to return to doing academic-style research, but with the benefit of Google's technical and monetary resources, and not part of X, which never produces anything of long-term value. I know for sure he wanted to make an impact in medical AI and felt that b…

[flagged]

She appears to be a symbol for everything that went wrong at Google. These are the kind of problems that arise when life is too easy, just before the downfall. In other words, decadence. How else can one explain that Google's AI research was dethroned by OpenAI?

Re: Why did Google Brain exist?

#37

Post by CogitoCogito » Wed, Apr 26, 2023, 6:37 PM UTC

michaelrpeskin wrote: post #10

dr_dshiv wrote: post #7

Lots of great insight. Here’s one: “Given the long timelines of a PhD program, the vast majority of early ML researchers were self-taught crossovers from other fields. This created the conditions for excellent interdisciplinary work to happen. This transitional anomaly is unfortunately mistaken by most people to be an inherent property of machine learning to upturn existing fields. It is not. Today, the vast majority…

obligatory XKCD: https://xkcd.com/793/

https://xkcd.com/1831/

Re: Why did Google Brain exist?

#38

Post by tdullien » Wed, Apr 26, 2023, 6:37 PM UTC

vl wrote: post #4

>PyTorch/Nvidia GPUs easily overtaking TensorFlow/Google TPUs. TF lost to PyTorch, and this is Google’s fault - TF APIs are both insane and badly documented. But nothing comes close to performance of Google’s TPU exaflop mega-clusters. Nvidia is not even in the same ballpark.

TF in its first version was stellarly misdesigned. It was infuriatingly difficult to use, particularly if you were of the "I just want to write code and have it autodiffed + SGDed" school, I found it crazy to use Python to manually construct a computational graph...

Re: Why did Google Brain exist?

#39

Post by ghaff » Wed, Apr 26, 2023, 6:38 PM UTC

kevviiinn wrote: post #25

qumpis wrote: post #15

Earlier quoted context omitted.

I'm yet to see an ML PhD be required to learn chemistry to a similar extent that chemists would need to doing ML (especially at research level)

That's because application and research are quite different. If one does a PhD in ML they learn how to research ML. Someone with a PhD in chemistry learns how to research chemistry, they only need to apply ML to that research

Back when O'Reilly was still hosting events (sigh), at one of their AI conferences, someone from Google gave a talk about differences between research/academic AI and applied AI. I think she had a PhD in the field herself but basically she made the argument that someone who is just looking to more or less apply existing tools to business or other problems mostly doesn't need a lot of the math-heavy theory you'll get in a PhD program. You do need to understand limitations etc. of tools and techniques. But that's different from the kind of novel investigation that's needed to get a PhD.

Re: Why did Google Brain exist?

#40

Post by asdfman123 » Wed, Apr 26, 2023, 6:38 PM UTC

light_hue_1 wrote: post #3

> The next obvious reason for Google to invest in pure research is for the breakthrough discoveries it has yielded and can continue to yield. As a rudimentary brag sheet, Brain gave Google TensorFlow, TPUs, significantly improved Translate, JAX, and Transformers. Except that these advances have made other companies an existential threat for Google. 2 years ago it was hard to imagine what could topple Google. Now a lo…

Google has very good LLMs. It just let OpenAI beat them to the punch by releasing them earlier.

As an established business, Google felt it had a lot to lose by releasing "unsafe" AI into the world. But OpenAI doesn't have a money printing machine, and it's sink-or-swim for them.