Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why did Google Brain exist?

moderndescartes.com

Post reply on HN
Print view View original

71–80 of 328 posts

Re: Why did Google Brain exist?

#71

Post by kevviiinn »

selimthegrim wrote: post #66
kevviiinn wrote: post #25

Earlier quoted context omitted.

That's because application and research are quite different. If one does a PhD in ML they learn how to research ML. Someone with a PhD in chemistry learns how to research chemistry, they only need to apply ML to that research

Well I think the issue is more of if you’re Genentech and you need ML people and can’t afford to pay them you’re probably better off retraining chemistry PhDs.

I think you missed my point. Genentech, AFAIK, was not doing research on machine learning as in the principles of how machine learning works and how to make it better. They do biotech research which uses applied machine learning. You don't need a PhD in ML to apply things that are already known

Re: Why did Google Brain exist?

#72

Post by logicchains »

Scea91 wrote: post #43
DavidSJ wrote: post #34

> … the publication of key research like LSTMs in 2014 … Minor nitpick, but LSTMs date to 1997 and were not invented by Google. [1] [1] Hochreiter and Schmidhuber (1997). Long short-term memory. https://ieeexplore.ieee.org/abstract/document/6795963

Seems more than a nitpick to me. I find the essay interesting but this line raised some distrust in me. How can someone have these deep insights into Google's ML strategy and the evolution of the field and simultaneously think LSTMs were invented by Google in 2014?

>How can someone have these deep insights into Google's ML strategy and the evolution of the field and simultaneously think LSTMs were invented by Google in 2014?

It may not have been accidental; there's a deliberate movement among some people in the ML community to deny Jürgen Schmidhuber credit for inventing LSTMs and GANs.

Re: Why did Google Brain exist?

#73

Post by choppaface »

khazhoux wrote: post #59

> I sat on it because I wasn’t sure of the optics of posting such an essay while employed by Google Brain. But then Google made my decision easier by laying me off in January. My severance check cleared... I'm really baffled by how people think it's OK to write public accounts of their previous ( and sometime current! ) employers' inner workings. This guy got paid a shitload of money to do work and to keep all intern…

I'm really baffled by how many people think a job is more than a job and there's some ownership over the employee's critical thinking capabilities during the job and after it ends.

While I agree the OP is "going for internet brownie points" (or probably a bit butthurt from being laid off from a certifiably top-5 cushiest job in the United States) the article doesn't include anything even remotely trade secret and is predominantly opinion. It's really totally fine to blog about how you feel about your employer. There are certainly risks, but a company has to pay extra if they actually don't want you to blog at all (or they have to be extremely litigious).

There's a strong precedent of employer-employee loyalty that has substantially been set due to pre-internet information disparities between the two. In the past year or so, there have been some pretty unprecedented layoffs (e.g. Google lays off thousands and then does billions of stock buybacks ...)... The employer-employee relationship needs to evolve.

Re: Why did Google Brain exist?

#74

Post by metadat »

tdullien wrote: post #38
vl wrote: post #4

>PyTorch/Nvidia GPUs easily overtaking TensorFlow/Google TPUs. TF lost to PyTorch, and this is Google’s fault - TF APIs are both insane and badly documented. But nothing comes close to performance of Google’s TPU exaflop mega-clusters. Nvidia is not even in the same ballpark.

TF in its first version was stellarly misdesigned. It was infuriatingly difficult to use, particularly if you were of the "I just want to write code and have it autodiffed + SGDed" school, I found it crazy to use Python to manually construct a computational graph...

What's the meaning of SDG in this context?

Edit:

Hypothesis: Stochastic Gradient Descent

Re: Why did Google Brain exist?

#75

Post by light_hue_1 »

sudosysgen wrote: post #61
light_hue_1 wrote: post #51

Earlier quoted context omitted.

> Quite a ridiculous statement. Google has inserted ML all over their products. Maybe you just don't notice, to their credit. But for example the fact that YouTube can automatically generate subtitles for any written language from any spoken language is a direct outcome of Google ML research. There are lots of machine-inferred search ranking signals. Google Sheets will automatically fill in your formulas, that's in-h…

Automatic subtitles and translation is actually a huge feature which is very useful to the many people that don't speak English. It definitely did provide Google with a lot of value.

> Automatic subtitles and translation is actually a huge feature which is very useful to the many people that don't speak English. It definitely did provide Google with a lot of value.

It lost Google immense value.

Before Google Brain the only speech recognizers that halfway worked were at Google, IBM and Amazon. And Amazon had to buy a company to get access.

After Google Brain, anyone can run a speech recognizer. One that is state of the art. There are many models out there that just work well enough.

Google went from having an ok speech recognizer that sort of worked in a few languages and gave YouTube an advantage that no company aside from IBM and Amazon could touch. Neither of which compete with Google much. No startup could have anything like Google's captioning. It was untouchable. Like, speech recognition researchers actively avoided this competition, that's how inferior everyone was.

To now, post Google Brain, any startup can have captions that are as good as YouTube's captions. You can run countless models on your laptop today.

This is a huge competitive loss for Google.

They got a minor feature for YouTube and lost one of the key ML advantages they had.

Re: Why did Google Brain exist?

#76

Post by QuercusMax »

jeffbee wrote: post #23
light_hue_1 wrote: post #3

> The next obvious reason for Google to invest in pure research is for the breakthrough discoveries it has yielded and can continue to yield. As a rudimentary brag sheet, Brain gave Google TensorFlow, TPUs, significantly improved Translate, JAX, and Transformers. Except that these advances have made other companies an existential threat for Google. 2 years ago it was hard to imagine what could topple Google. Now a lo…

> Basically nothing has spun out of it to benefit Google Quite a ridiculous statement. Google has inserted ML all over their products. Maybe you just don't notice, to their credit. But for example the fact that YouTube can automatically generate subtitles for any written language from any spoken language is a direct outcome of Google ML research. There are lots of machine-inferred search ranking signals. Google Sheet…

A ton of the new stuff in the pixel cameras is ML powered, along with a lot of Google Photos.

Re: Why did Google Brain exist?

#77

Post by ragebol »

metadat wrote: post #74
tdullien wrote: post #38

Earlier quoted context omitted.

TF in its first version was stellarly misdesigned. It was infuriatingly difficult to use, particularly if you were of the "I just want to write code and have it autodiffed + SGDed" school, I found it crazy to use Python to manually construct a computational graph...

What's the meaning of SDG in this context? Edit: Hypothesis: Stochastic Gradient Descent

Parent typed SGD, which means Stochastic Gradient Descent. An optimization method.

Re: Why did Google Brain exist?

#78

Post by dragonwriter »

jeffbee wrote: post #48
marricks wrote: post #32

Earlier quoted context omitted.

Google has since fired other folks on her team and was in crisis mode to protect Dean. Like, I’m not really going to give them the benefit of the doubt on this. When people brought Dean up Timnit came up as something to consider, it’s interesting to see how all anyone has to say in these threads is reverence towards him. People should try to see the whole picture.

Timnit and the other ex-ML ethics crowd who got fired from Google seem like some of the most ignorant people around. I don't defend Dean reflexively, it just seems like he is on the right side of the issue. For example, here is Emma Strubell accusing Dean of creating a "toxic workplace culture" after he and David Patterson had to refute her paper in print. https://twitter.com/strubell/status/1634164078691098625?lang.…

> Timnit and the other ex-ML ethics crowd

Timnit is ex-Google, but very much not ex-ML ethics (fouded Distributed AI Research Institute focussed on the field in late 2021). Very much also true of Margaret Mitchell, who has been at Hugging Face since 2021.

Re: Why did Google Brain exist?

#79

Post by choppaface »

> Neither side “won” this merger. I think both Brain and DeepMind lose. I expect to see many project cancellations, project mergers, and reallocations of headcount over the next few months, as well as attrition.

This merger will be a big test for Sundar, who has openly admitted years ago to there being major trust issues [1]. Can Sundar maintain the perspective of being the alpha company while bleeding a ton of talent that doesn't actively contribute to tech dominance? Or will he piss off the wrong people internally? It's OK to have a Google Plus / Stadia failure if the team really wanted to do the project. If the team does _not_ want to work together though, and they fail, then Sundar's request that the orgs work together to save the company is going to get totally ignored in the finger-pointing.

[1] https://www.axios.com/2019/10/26/google-trust-employee-immig... .

Re: Why did Google Brain exist?

#80

Post by crazygringo »

khazhoux wrote: post #59

> I sat on it because I wasn’t sure of the optics of posting such an essay while employed by Google Brain. But then Google made my decision easier by laying me off in January. My severance check cleared... I'm really baffled by how people think it's OK to write public accounts of their previous ( and sometime current! ) employers' inner workings. This guy got paid a shitload of money to do work and to keep all intern…

How do you know he was paid to keep all internal details private after he leaves? Do you have knowledge of the employment contract, can you share the relevant language with us?

All he says is his concern about "optics", which has nothing to do with contract.

If Google has a problem with his post, they can go after him, but that's an issue between Google and him, not with you or me or the rest of the internet.

I'm definitely struggling to see what any of this has to do with personal integrity, betrayal, or squeezing personal benefit. To the contrary, it simply seems informative and he's sharing knowledge just to be helpful. Unless I've missed something, I don't see anything revealed that would harm Google or his former teammates here. No leaks of what's coming down the pipeline, no scandals, nothing of the sort.

People are allowed to share opinions of their previous employment and generally describe the broad outlines of their work and where they worked. This isn't a situation of working for the CIA with top-secret clearance.

Post reply on HN