Why did Google Brain exist? • Hacker News Board

Re: Why did Google Brain exist?

#11

Post by cma » Wed, Apr 26, 2023, 6:08 PM UTC

light_hue_1 wrote: post #3

> The next obvious reason for Google to invest in pure research is for the breakthrough discoveries it has yielded and can continue to yield. As a rudimentary brag sheet, Brain gave Google TensorFlow, TPUs, significantly improved Translate, JAX, and Transformers. Except that these advances have made other companies an existential threat for Google. 2 years ago it was hard to imagine what could topple Google. Now a lo…

> And yet at the same time, so much has leaked out, and so many people have left with that knowledge Google paid for, that Google might go the way of Yahoo in 10 years.

Google couldn't have hired the talent they did without allowing them to publish.

Re: Why did Google Brain exist?

#12

Post by marricks » Wed, Apr 26, 2023, 6:08 PM UTC

dekhn wrote: post #2

because once Jeff Dean had solved Google's maslow problems (scaling web search, making ads profitable, developing high performance machine learning systems) he wanted to return to doing academic-style research, but with the benefit of Google's technical and monetary resources, and not part of X, which never produces anything of long-term value. I know for sure he wanted to make an impact in medical AI and felt that b…

[flagged]

Re: Why did Google Brain exist?

#13

Post by belval » Wed, Apr 26, 2023, 6:08 PM UTC

vl wrote: post #4

>PyTorch/Nvidia GPUs easily overtaking TensorFlow/Google TPUs. TF lost to PyTorch, and this is Google’s fault - TF APIs are both insane and badly documented. But nothing comes close to performance of Google’s TPU exaflop mega-clusters. Nvidia is not even in the same ballpark.

There is a first mover handicap there though. TF1.0 included a bunch of things that were harder to understand like tf.Session(). PyTorch was inspired from the good parts and "we will eager-everything". Internally I'm sure there was a lot of debate in the TF team that culminated with TF2.0, but by that time the damage was done and people saw PyTorch as easier.

Re: Why did Google Brain exist?

#14

Post by dekhn » Wed, Apr 26, 2023, 6:09 PM UTC

light_hue_1 wrote: post #3

> The next obvious reason for Google to invest in pure research is for the breakthrough discoveries it has yielded and can continue to yield. As a rudimentary brag sheet, Brain gave Google TensorFlow, TPUs, significantly improved Translate, JAX, and Transformers. Except that these advances have made other companies an existential threat for Google. 2 years ago it was hard to imagine what could topple Google. Now a lo…

Google never talked much about it externally, but Google Research (the predecessor to Brain) had a single project which almost entirely funded the entire division- a growth-oriented machine learnign system called Sibyl. What was sibyl used for? Growing youtube and google play and other products by making the more addictive. Sibyl wasn't a very good system (I've never seen a product that had more technical debt) but it did basically "pay for" all of the research for a while.

Re: Why did Google Brain exist?

#15

Post by qumpis » Wed, Apr 26, 2023, 6:09 PM UTC

dr_dshiv wrote: post #7

Lots of great insight. Here’s one: “Given the long timelines of a PhD program, the vast majority of early ML researchers were self-taught crossovers from other fields. This created the conditions for excellent interdisciplinary work to happen. This transitional anomaly is unfortunately mistaken by most people to be an inherent property of machine learning to upturn existing fields. It is not. Today, the vast majority…

I'm yet to see an ML PhD be required to learn chemistry to a similar extent that chemists would need to doing ML (especially at research level)

Re: Why did Google Brain exist?

#16

Post by mupuff1234 » Wed, Apr 26, 2023, 6:10 PM UTC

marricks wrote: post #12

dekhn wrote: post #2

because once Jeff Dean had solved Google's maslow problems (scaling web search, making ads profitable, developing high performance machine learning systems) he wanted to return to doing academic-style research, but with the benefit of Google's technical and monetary resources, and not part of X, which never produces anything of long-term value. I know for sure he wanted to make an impact in medical AI and felt that b…

[flagged]

Or maybe it's just not perceived as controversial?

Her boss told her to do something, she refused and got sacked.

Re: Why did Google Brain exist?

#17

Post by abtinf » Wed, Apr 26, 2023, 6:11 PM UTC

light_hue_1 wrote: post #3

> The next obvious reason for Google to invest in pure research is for the breakthrough discoveries it has yielded and can continue to yield. As a rudimentary brag sheet, Brain gave Google TensorFlow, TPUs, significantly improved Translate, JAX, and Transformers. Except that these advances have made other companies an existential threat for Google. 2 years ago it was hard to imagine what could topple Google. Now a lo…

It’s like Xerox PARC all over again.

Re: Why did Google Brain exist?

#18

Post by qumpis » Wed, Apr 26, 2023, 6:11 PM UTC

marricks wrote: post #12

dekhn wrote: post #2

because once Jeff Dean had solved Google's maslow problems (scaling web search, making ads profitable, developing high performance machine learning systems) he wanted to return to doing academic-style research, but with the benefit of Google's technical and monetary resources, and not part of X, which never produces anything of long-term value. I know for sure he wanted to make an impact in medical AI and felt that b…

[flagged]

Wiki doesn't seem to give detail into the situation, nor the paper in question

Re: Why did Google Brain exist?

#19

Post by aix1 » Wed, Apr 26, 2023, 6:11 PM UTC

vl wrote: post #9

jdlyga wrote: post #6

Earlier quoted context omitted.

I can speak from experience on this. Getting started with TensorFlow was very complicated with sparse documentation, so we dropped the idea of using it.

I had to use TF when I worked at G, when I left I immediately started to use PyTorch and never looked back.

Even internally at Google/DeepMind, all the cool kids have long moved to JAX.

Re: Why did Google Brain exist?

#20

Post by dekhn » Wed, Apr 26, 2023, 6:11 PM UTC

marricks wrote: post #12

dekhn wrote: post #2

because once Jeff Dean had solved Google's maslow problems (scaling web search, making ads profitable, developing high performance machine learning systems) he wanted to return to doing academic-style research, but with the benefit of Google's technical and monetary resources, and not part of X, which never produces anything of long-term value. I know for sure he wanted to make an impact in medical AI and felt that b…

[flagged]

That's a very simplified version of the story, but I would say that Dean greatly reduced his stature when he defended Megan Kacholia for her abrupt termination of Timnit. Note that Timnit was verbally abusive to Jeff's reports, anybody who worked there could see what she was posting to internal group discussions, so her time at Google was limited, but most managers would say that she should had at least been put on a pip and given 6 months.

Dean has since cited the paper in a subsequent paper (which tears apart the Stochastic Parrots paper).