Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why did Google Brain exist?

moderndescartes.com

Post reply on HN
Print view View original

41–50 of 328 posts

Re: Why did Google Brain exist?

#41

Post by disgruntledphd2 »

belval wrote: post #13
vl wrote: post #4

>PyTorch/Nvidia GPUs easily overtaking TensorFlow/Google TPUs. TF lost to PyTorch, and this is Google’s fault - TF APIs are both insane and badly documented. But nothing comes close to performance of Google’s TPU exaflop mega-clusters. Nvidia is not even in the same ballpark.

There is a first mover handicap there though. TF1.0 included a bunch of things that were harder to understand like tf.Session(). PyTorch was inspired from the good parts and "we will eager-everything". Internally I'm sure there was a lot of debate in the TF team that culminated with TF2.0, but by that time the damage was done and people saw PyTorch as easier.

Nope, Pytorch was inspired by the Lua version of Torch which well predates Tensor flow. To be fair, basically every other DL framework made the same mistake though.

Also, tensorflow was a total nightmare to install while Pytorch was pretty straightforward, which definitely shouldn't be discounted.

Re: Why did Google Brain exist?

#42

Post by calrissian »

marricks wrote: post #12
dekhn wrote: post #2

because once Jeff Dean had solved Google's maslow problems (scaling web search, making ads profitable, developing high performance machine learning systems) he wanted to return to doing academic-style research, but with the benefit of Google's technical and monetary resources, and not part of X, which never produces anything of long-term value. I know for sure he wanted to make an impact in medical AI and felt that b…

[flagged]

> Also not surprised at the immediate down votes for questioning Googles new AI lead!

That's because you are wrong to pretend he did anything wrong by firing T.G. And also, because you added this weird lie/mudslinging/whatever on top of it:

> while she was on vacation

Re: Why did Google Brain exist?

#43

Post by Scea91 »

DavidSJ wrote: post #34

> … the publication of key research like LSTMs in 2014 … Minor nitpick, but LSTMs date to 1997 and were not invented by Google. [1] [1] Hochreiter and Schmidhuber (1997). Long short-term memory. https://ieeexplore.ieee.org/abstract/document/6795963

Seems more than a nitpick to me. I find the essay interesting but this line raised some distrust in me. How can someone have these deep insights into Google's ML strategy and the evolution of the field and simultaneously think LSTMs were invented by Google in 2014?

Re: Why did Google Brain exist?

#44

Post by opportune »

marricks wrote: post #32
dekhn wrote: post #20

Earlier quoted context omitted.

That's a very simplified version of the story, but I would say that Dean greatly reduced his stature when he defended Megan Kacholia for her abrupt termination of Timnit. Note that Timnit was verbally abusive to Jeff's reports, anybody who worked there could see what she was posting to internal group discussions, so her time at Google was limited, but most managers would say that she should had at least been put on a…

Google has since fired other folks on her team and was in crisis mode to protect Dean. Like, I’m not really going to give them the benefit of the doubt on this. When people brought Dean up Timnit came up as something to consider, it’s interesting to see how all anyone has to say in these threads is reverence towards him. People should try to see the whole picture.

Being somewhat involved in one bad thing doesn’t justify cancelling someone.

To my knowledge Dean was essentially doing post-hoc damage control for what one of the middle managers in his org did. Even if they did want Timnit gone (as others mention, you are getting only one side of the story in media) they did it in a bad way, for sure. At the same time I don’t think one botched firing diminishes decades of achievements from a legitimately kind person.

Re: Why did Google Brain exist?

#45

Post by hintymad »

marricks wrote: post #12
dekhn wrote: post #2

because once Jeff Dean had solved Google's maslow problems (scaling web search, making ads profitable, developing high performance machine learning systems) he wanted to return to doing academic-style research, but with the benefit of Google's technical and monetary resources, and not part of X, which never produces anything of long-term value. I know for sure he wanted to make an impact in medical AI and felt that b…

[flagged]

Controvery of what? Did you read Gebru's paper? For instance, her calculation of carbon footprint of training BERT assumes that companies will train BERT 24x7. Gebru is a disgrace to the community because she always, I mean literally always, attacks her critics by motives. You think bias is a data problem? You're a bigot (See her dispute with LeCun). You disagree with my assessment on an ML model? You are white male oppressor (her attacking a Google's SVP).

Gebru is not a researcher. She is a modern-age Trofim Lysenko, who politicizes everything and weaponizes political correctness.

And yeah, she deserves to be fired. Many times.

Re: Why did Google Brain exist?

#46

Post by asdfman123 »

My theory is that broadly, tech learned not to act like Microsoft in the 90s -- closed off, anti-competitive, unpopular -- but swung too far in the opposite direction.

Google has been basically giving away technology for free, which was easy because of all the easy money. It's good for reputation and attracting the best talent. That is, until a competitor starts to threaten to overtake you with the technology you gave them (ChatGPT based on LLM research, Edge based on Chromium, etc.).

Re: Why did Google Brain exist?

#47

Post by brilee »

Scea91 wrote: post #43
DavidSJ wrote: post #34

> … the publication of key research like LSTMs in 2014 … Minor nitpick, but LSTMs date to 1997 and were not invented by Google. [1] [1] Hochreiter and Schmidhuber (1997). Long short-term memory. https://ieeexplore.ieee.org/abstract/document/6795963

Seems more than a nitpick to me. I find the essay interesting but this line raised some distrust in me. How can someone have these deep insights into Google's ML strategy and the evolution of the field and simultaneously think LSTMs were invented by Google in 2014?

sorry, I had a mind fart. I was thinking of seq2seq https://research.google/pubs/pub43155/

Pushing the fix now...

Re: Why did Google Brain exist?

#48

Post by jeffbee »

marricks wrote: post #32
dekhn wrote: post #20

Earlier quoted context omitted.

That's a very simplified version of the story, but I would say that Dean greatly reduced his stature when he defended Megan Kacholia for her abrupt termination of Timnit. Note that Timnit was verbally abusive to Jeff's reports, anybody who worked there could see what she was posting to internal group discussions, so her time at Google was limited, but most managers would say that she should had at least been put on a…

Google has since fired other folks on her team and was in crisis mode to protect Dean. Like, I’m not really going to give them the benefit of the doubt on this. When people brought Dean up Timnit came up as something to consider, it’s interesting to see how all anyone has to say in these threads is reverence towards him. People should try to see the whole picture.

Timnit and the other ex-ML ethics crowd who got fired from Google seem like some of the most ignorant people around. I don't defend Dean reflexively, it just seems like he is on the right side of the issue. For example, here is Emma Strubell accusing Dean of creating a "toxic workplace culture" after he and David Patterson had to refute her paper in print.

https://twitter.com/strubell/status/1634164078691098625?lang...

The thing is if David Patterson and Jeff Dean think your numbers for the energy cost of machine learning might be wrong, then you are probably wrong. These ML meta-researchers are not practitioners and appear to have no idea what they are talking about. Keeping a person like Timnit or Strubell on staff seems like it costs more than its worth.

Re: Why did Google Brain exist?

#49

Post by hintymad »

Silverback_VII wrote: post #36
marricks wrote: post #12

Earlier quoted context omitted.

[flagged]

She appears to be a symbol for everything that went wrong at Google. These are the kind of problems that arise when life is too easy, just before the downfall. In other words, decadence. How else can one explain that Google's AI research was dethroned by OpenAI?

Yeah, Dean's fault is hiring such people in the first place. If you hire an activist, you get activism. And if you hire someone whose livelihood depends on finding more problems, well, they will scream more problems, one way or another. Otherwise, why would state U of Mich got one DEI officer per three staff members?

Re: Why did Google Brain exist?

#50

Post by kps »

amoss wrote: post #35

> Today, thought leaders casually opine on how and where ML will be useful, and MBAs feel like this is an acceptable substitute for expert opinion. Sounds like standard operating procedure.

Sounds like something LLMs would actually be good for. They're not getting us fusion power or cancer cures.
Post reply on HN