Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why did Google Brain exist?

moderndescartes.com

Post reply on HN
Print view View original

121–130 of 328 posts

Re: Why did Google Brain exist?

#121

Post by dekhn »

tdullien wrote: post #38
vl wrote: post #4

>PyTorch/Nvidia GPUs easily overtaking TensorFlow/Google TPUs. TF lost to PyTorch, and this is Google’s fault - TF APIs are both insane and badly documented. But nothing comes close to performance of Google’s TPU exaflop mega-clusters. Nvidia is not even in the same ballpark.

TF in its first version was stellarly misdesigned. It was infuriatingly difficult to use, particularly if you were of the "I just want to write code and have it autodiffed + SGDed" school, I found it crazy to use Python to manually construct a computational graph...

You need something to construct a graph. Why not pick a well-known language already used in scientific computing and stats/data science? The other options are: pick a lesser known language (lua, julia) or a language not traditionally used for scientific computing (php, ruby), or a compiled language most researchers don't know (C++), or a raw config file format (which you would then use code to generate).

What's really crazy is using Pure, Idiomatic Python which is then Traced to generate a graph (what Jax does). I want my model definitions to be declarative, not implict in the code.

Re: Why did Google Brain exist?

#122

Post by nologic01 »

> it is becoming increasingly apparent to Google that it does not know how to capture that value

To paraphrase, its the business model, stupid.

Inventing algorithms, building powerful tools and infrastructure etc is actually a tractable problem: you can throw money and brains at it (and the latter typically follows the former). While the richness of research fields is not predictable, you can bet that the general project of employing silicon to work with information will keep bearing fruits for a long time. So creating that value is not the problem.

The problem with capitalizing (literally) on that intellectual output is that it can only be done 1) within a given business model that can channel effectively it or 2) through the invention of totally new business models. 1) is a challenge: These billions of users on which AI goodies can surface are not customers, they are product. They don't pay for anything and they don't create any virtuous circle of requirements and solutions. Alas, option 2) inventing major new business models is highly non-trivial. The track record is poor: the only major alternative business model to adtech (cloud unit) was not invented there anyway and in any case selling sophisticated IT services whether to consumers or enterprise is a can of worms that others have much more experience in.

For a industrial research unit to thrive, its output must be congruent with what the organization is doing. Not necessarily in the detail, but definitely in the big picture.

Re: Why did Google Brain exist?

#123

Post by confounded »

khazhoux wrote: post #86
crazygringo wrote: post #80

Earlier quoted context omitted.

How do you know he was paid to keep all internal details private after he leaves? Do you have knowledge of the employment contract, can you share the relevant language with us? All he says is his concern about "optics", which has nothing to do with contract. If Google has a problem with his post, they can go after him, but that's an issue between Google and him, not with you or me or the rest of the internet. I'm def…

> Do you have knowledge of the employment contract, can you share the relevant language with us? I actually could dig up my own contract from years ago (ugh, the effort though) but Confidentiality clause is in there, and it was made clear during on-boarding what is expected from employees: don't share any internal info unless you're an authorized company representative.

You should write to to the author, to scold them directly!

Re: Why did Google Brain exist?

#124

Post by asciimov »

dr_dshiv wrote: post #7

Lots of great insight. Here’s one: “Given the long timelines of a PhD program, the vast majority of early ML researchers were self-taught crossovers from other fields. This created the conditions for excellent interdisciplinary work to happen. This transitional anomaly is unfortunately mistaken by most people to be an inherent property of machine learning to upturn existing fields. It is not. Today, the vast majority…

> "[...] I’ve seen repeatedly that it’s much harder for a ML PhD to learn chemistry than for a chemist to learn ML.”

That's good ol' academic gatekeeping for ya, available wherever PhD's are found.

Re: Why did Google Brain exist?

#125

Post by simonster »

I work for Google Brain. I remember meeting Brian at a conference and I have nothing but good things to say about him. That said, I think Brian is underestimating the extent to which the Brain/DeepMind merger is happening because it's what researchers want. Many of us have a strong sense that the future of ML involves models built by large teams in industry environments. My impression is that the goal of the merger is to create a better, more coordinated environment for that kind of research.

Re: Why did Google Brain exist?

#126

Post by khazhoux »

brilee wrote: post #115
khazhoux wrote: post #59

> I sat on it because I wasn’t sure of the optics of posting such an essay while employed by Google Brain. But then Google made my decision easier by laying me off in January. My severance check cleared... I'm really baffled by how people think it's OK to write public accounts of their previous ( and sometime current! ) employers' inner workings. This guy got paid a shitload of money to do work and to keep all intern…

I've been quite careful not to divulge anything confidential, and anything that is remotely close to sensitive has publicly accessible citations. My opinions about Google are tantamount to discussions about workplace conditions, and it would be very bad for society if ex-employees were not allowed to discuss those.

But all your opinions are informed by your years of TGIFs and internal emails and discussions and presentations and your insider perspective. When you talk about promotions, or internal values and prioritizations, you are leveraging info gained privately.

If I'm wrong and nothing in your contract or on-boarding said you shouldn't talk about internals, then my bad. But I suspect they were as clear with you as they were with me, that it's not ok to post anything based on inside info. And in your opening paragraph you say:

> As somebody with a unique perspective

Your unique perspective was your access as an employee.

> and the unique freedom to share it

Your unique freedom is that you're done receiving money from them. But contractually, this doesn't matter.

Re: Why did Google Brain exist?

#127

Post by knorker »

dr_dshiv wrote: post #7

Lots of great insight. Here’s one: “Given the long timelines of a PhD program, the vast majority of early ML researchers were self-taught crossovers from other fields. This created the conditions for excellent interdisciplinary work to happen. This transitional anomaly is unfortunately mistaken by most people to be an inherent property of machine learning to upturn existing fields. It is not. Today, the vast majority…

> I’ve seen repeatedly that it’s much harder for a ML PhD to learn chemistry than for a chemist to learn ML.

Haha, I've seen that for so many topics. "It's much easier for someone used to circuit switched phone networks to learn IP than the other way around", says the person who started with circuit switched.

I just thought "dude, you're literally the worst at IP networking that I've ever met. Your misunderstandings are dug into everything I've seen you do with IP".

Re: Why did Google Brain exist?

#128

Post by anonymouskimmer »

khazhoux wrote: post #86
crazygringo wrote: post #80

Earlier quoted context omitted.

How do you know he was paid to keep all internal details private after he leaves? Do you have knowledge of the employment contract, can you share the relevant language with us? All he says is his concern about "optics", which has nothing to do with contract. If Google has a problem with his post, they can go after him, but that's an issue between Google and him, not with you or me or the rest of the internet. I'm def…

> Do you have knowledge of the employment contract, can you share the relevant language with us? I actually could dig up my own contract from years ago (ugh, the effort though) but Confidentiality clause is in there, and it was made clear during on-boarding what is expected from employees: don't share any internal info unless you're an authorized company representative.

General working conditions are not "internal info"; it is beneficial to society to discuss working conditions (which can be pretty detailed), and is thus a protected activity under various laws. Nothing in any contract can obviate this lawful right (and, to some, a basic duty of citizenship, since the country is more important than any single company within the country). At best contracts can highlight what is privileged information that there is a duty to keep secret.

Re: Why did Google Brain exist?

#129

Post by frozenport »

ghaff wrote: post #39
kevviiinn wrote: post #25

Earlier quoted context omitted.

That's because application and research are quite different. If one does a PhD in ML they learn how to research ML. Someone with a PhD in chemistry learns how to research chemistry, they only need to apply ML to that research

Back when O'Reilly was still hosting events (sigh), at one of their AI conferences, someone from Google gave a talk about differences between research/academic AI and applied AI. I think she had a PhD in the field herself but basically she made the argument that someone who is just looking to more or less apply existing tools to business or other problems mostly doesn't need a lot of the math-heavy theory you'll get…

>>math-heavy theory you'll get in a PhD program

Lol. With the exception of niche groups in compressed sensing, math doesn't get too hard. Furthermore, ML isn't math driven in the sense people are trying things and somebody tries to come up with the explanation after the fact.

Re: Why did Google Brain exist?

#130

Post by khazhoux »

pessimizer wrote: post #110
khazhoux wrote: post #100

Earlier quoted context omitted.

I totally get why people act this way. Because it's Very Important That Everyone Knows What I Think. But there is no inequity or conflict of interest here. None of that about being trapped in a capitalist economy is to the point here. He has probably a couple million dollars in his bank account that wasn't there before, and the deal was to not talk publicly about internals (which includes promotions process, internal…

You don't think there's any inequity between somebody with a couple million in the bank and google, or any conflict of interest between my desire to talk about my work and my employer's desire that I do not? Your position is valid enough without being willfully obtuse.

Once you agree to not talk about it (as part of the employment terms), then there is zero conflict of interest.

"Yes, I agree to not share internal info, in exchange for this money. And by the way, I will still share internal info, because inequity."

Post reply on HN