Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why did Google Brain exist?

moderndescartes.com

Post reply on HN
Print view View original

21–30 of 328 posts

Re: Why did Google Brain exist?

#21

Post by fknorangesite »

michaelrpeskin wrote: post #10
dr_dshiv wrote: post #7

Lots of great insight. Here’s one: “Given the long timelines of a PhD program, the vast majority of early ML researchers were self-taught crossovers from other fields. This created the conditions for excellent interdisciplinary work to happen. This transitional anomaly is unfortunately mistaken by most people to be an inherent property of machine learning to upturn existing fields. It is not. Today, the vast majority…

obligatory XKCD: https://xkcd.com/793/

Similarly: https://www.smbc-comics.com/comic/2012-03-21

Re: Why did Google Brain exist?

#22

Post by dekhn »

aix1 wrote: post #19
vl wrote: post #9

Earlier quoted context omitted.

I had to use TF when I worked at G, when I left I immediately started to use PyTorch and never looked back.

Even internally at Google/DeepMind, all the cool kids have long moved to JAX.

Once the ads model runs on Jax instead of TF, it's curtains for TF.

Re: Why did Google Brain exist?

#23

Post by jeffbee »

light_hue_1 wrote: post #3

> The next obvious reason for Google to invest in pure research is for the breakthrough discoveries it has yielded and can continue to yield. As a rudimentary brag sheet, Brain gave Google TensorFlow, TPUs, significantly improved Translate, JAX, and Transformers. Except that these advances have made other companies an existential threat for Google. 2 years ago it was hard to imagine what could topple Google. Now a lo…

> Basically nothing has spun out of it to benefit Google

Quite a ridiculous statement. Google has inserted ML all over their products. Maybe you just don't notice, to their credit. But for example the fact that YouTube can automatically generate subtitles for any written language from any spoken language is a direct outcome of Google ML research. There are lots of machine-inferred search ranking signals. Google Sheets will automatically fill in your formulas, that's in-house ML research, too.

Re: Why did Google Brain exist?

#25

Post by kevviiinn »

qumpis wrote: post #15
dr_dshiv wrote: post #7

Lots of great insight. Here’s one: “Given the long timelines of a PhD program, the vast majority of early ML researchers were self-taught crossovers from other fields. This created the conditions for excellent interdisciplinary work to happen. This transitional anomaly is unfortunately mistaken by most people to be an inherent property of machine learning to upturn existing fields. It is not. Today, the vast majority…

I'm yet to see an ML PhD be required to learn chemistry to a similar extent that chemists would need to doing ML (especially at research level)

That's because application and research are quite different. If one does a PhD in ML they learn how to research ML. Someone with a PhD in chemistry learns how to research chemistry, they only need to apply ML to that research

Re: Why did Google Brain exist?

#26

Post by javajosh »

michaelrpeskin wrote: post #10
dr_dshiv wrote: post #7

Lots of great insight. Here’s one: “Given the long timelines of a PhD program, the vast majority of early ML researchers were self-taught crossovers from other fields. This created the conditions for excellent interdisciplinary work to happen. This transitional anomaly is unfortunately mistaken by most people to be an inherent property of machine learning to upturn existing fields. It is not. Today, the vast majority…

obligatory XKCD: https://xkcd.com/793/

That's a great xkcd, but there are 2 upsides to this arrogant approach. First, arrogance is a nerd-snipe maximizer. Second, there is a small chance you're absolutely right, and you've just obviated a whole field from first principles. It doesn't happen often, but when it does happen and there is no clout like "emporer's new clothes" clout.

EDIT: The downside, of course, is that you appear arrogant, and people won't like you. This can hurt your reputation because it is apparently anti-social behavior on several levels. I think its fair to call it a little bit of an intellectual punk rock move that is probably better left to the young. It's an interesting emotional anchor to mapping a new field, though.

Re: Why did Google Brain exist?

#27

Post by bitL »

belval wrote: post #13
vl wrote: post #4

>PyTorch/Nvidia GPUs easily overtaking TensorFlow/Google TPUs. TF lost to PyTorch, and this is Google’s fault - TF APIs are both insane and badly documented. But nothing comes close to performance of Google’s TPU exaflop mega-clusters. Nvidia is not even in the same ballpark.

There is a first mover handicap there though. TF1.0 included a bunch of things that were harder to understand like tf.Session(). PyTorch was inspired from the good parts and "we will eager-everything". Internally I'm sure there was a lot of debate in the TF team that culminated with TF2.0, but by that time the damage was done and people saw PyTorch as easier.

I think the main problem was debugging tensors on the fly, impossible with TF/Keras, but completely natural to PyTorch. Most researchers needed to sequentially observe what is going on in tensors (histograms etc.) and even doing backprop for their newly constructed layers by hand and that was difficult with TF.

Re: Why did Google Brain exist?

#28

Post by gillesjacobs »

vl wrote: post #4

>PyTorch/Nvidia GPUs easily overtaking TensorFlow/Google TPUs. TF lost to PyTorch, and this is Google’s fault - TF APIs are both insane and badly documented. But nothing comes close to performance of Google’s TPU exaflop mega-clusters. Nvidia is not even in the same ballpark.

I have used both but ended up dropping TF for PyTorch after 2018. Mainly it was the larger PyTorch ecosystem in my field (NLP) and clear API design and documentation that did it for me.

However, TF was still a valid contender and it was not clearcut back in 2016-17 which framework was better.

Re: Why did Google Brain exist?

#29

Post by temac »

dekhn wrote: post #14
light_hue_1 wrote: post #3

> The next obvious reason for Google to invest in pure research is for the breakthrough discoveries it has yielded and can continue to yield. As a rudimentary brag sheet, Brain gave Google TensorFlow, TPUs, significantly improved Translate, JAX, and Transformers. Except that these advances have made other companies an existential threat for Google. 2 years ago it was hard to imagine what could topple Google. Now a lo…

Google never talked much about it externally, but Google Research (the predecessor to Brain) had a single project which almost entirely funded the entire division- a growth-oriented machine learnign system called Sibyl. What was sibyl used for? Growing youtube and google play and other products by making the more addictive. Sibyl wasn't a very good system (I've never seen a product that had more technical debt) but i…

Seems to be quite evil though.

Re: Why did Google Brain exist?

#30

Post by Traubenfuchs »

marricks wrote: post #12
dekhn wrote: post #2

because once Jeff Dean had solved Google's maslow problems (scaling web search, making ads profitable, developing high performance machine learning systems) he wanted to return to doing academic-style research, but with the benefit of Google's technical and monetary resources, and not part of X, which never produces anything of long-term value. I know for sure he wanted to make an impact in medical AI and felt that b…

[flagged]

A lot of people, especially on hacker news, feel disdain for researchers of ethics, bias and fairness, as they are perceived as both holding technology back and profiting from advances in it (that they can then analyse and criticize).
Post reply on HN