Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why did Google Brain exist?

moderndescartes.com

Post reply on HN
Print view View original

201–210 of 328 posts

Re: Why did Google Brain exist?

#201

Post by bbor »

khazhoux wrote: post #130
pessimizer wrote: post #110

Earlier quoted context omitted.

You don't think there's any inequity between somebody with a couple million in the bank and google, or any conflict of interest between my desire to talk about my work and my employer's desire that I do not? Your position is valid enough without being willfully obtuse.

Once you agree to not talk about it (as part of the employment terms), then there is zero conflict of interest. "Yes, I agree to not share internal info, in exchange for this money. And by the way, I will still share internal info, because inequity."

You're thinking about employment contracts like an abstract economic exchange between two free parties, which is very micro. Try thinking about it instead like a bargain with the (macro) devil.

In other words, consider someone's perspective who has society split into two camps: the people who do all the work, and the corrupt elite that make their living through theft and oppression. In such a world, signing a contract with an employer (i.e. capitalist i.e. elite) is more of a practical step than a sacrosanct bond. There's a level of "what's reasonable" and "what legally enforceable" beyond the basic "never break a promise" level you're working at, IMO.

No ones endorsing publishing trade secrets randomly, but you're treating all disclosures like they're equivalent.

Re: Why did Google Brain exist?

#203

Post by ironman1478 »

"I’ve seen repeatedly that it’s much harder for a ML PhD to learn chemistry than for a chemist to learn ML. (This may be survivorship bias; the only chemists I encounter are those that have successfully learned ML, whereas I see ML researchers attempt and fail to learn chemistry all the time.)"

This is something that rings really true to me. I work in imaging and it's just very clear that there are groups of people in ML that don't want to learn how things actually work and just want to throw a model at it (this is a generalization obviously, but it's more often than not the case). It only gets you 80% there, which is fine usually, but not fine when the details are make or break for a company. Unfortunately that last 20% requires understanding of the domain and people just don't like digging into a topic to actually understanding things.

Re: Why did Google Brain exist?

#204

Post by oofbey »

anonylizard wrote: post #193
gowld wrote: post #175

Earlier quoted context omitted.

The goal of the merger is for execs to look like they are doing something to drive progress. Actual progress comes from the researchers and developers.

Well, where exactly is this progress? Where is Google's answer to GPT-4? Why weren't the 'researchers and developers' making a GPT-4 equivalent? Turns out you sometimes you need a top down, centralised vision to execute on projects. When the goal is undefined, you can allow researchers to run free and explore, now its full on wartime, with clear goals (make GPT-5,6,7....).

Google is fundamentally allergic to top-down management. Most googlers will reject any attempt to be told what to do as wrong, because lots of IC's voting with their feet are smarter than any (google) exec at figuring out what to do.

Last time Google got spooked by a competitor was Facebook, and they built Google Plus in response. We all know that was an utter failure. Googlers could escape that one with their egos in tact because winning in "social" is just some UX junk, not hard-core engineering like ML.

It's gonna be super hard for them to come to grips with the fact that they are way behind on something that they should be good at. Plan for lots of cognitive dissonance ahead.

Re: Why did Google Brain exist?

#205

Post by drakenot »

ironman1478 wrote: post #203

"I’ve seen repeatedly that it’s much harder for a ML PhD to learn chemistry than for a chemist to learn ML. (This may be survivorship bias; the only chemists I encounter are those that have successfully learned ML, whereas I see ML researchers attempt and fail to learn chemistry all the time.)" This is something that rings really true to me. I work in imaging and it's just very clear that there are groups of people i…

This seems to kind of be the opposite opinion of The Bitter Lesson[0].

[0] http://www.incompleteideas.net/IncIdeas/BitterLesson.html

Re: Why did Google Brain exist?

#206

Post by sushid »

light_hue_1 wrote: post #3

> The next obvious reason for Google to invest in pure research is for the breakthrough discoveries it has yielded and can continue to yield. As a rudimentary brag sheet, Brain gave Google TensorFlow, TPUs, significantly improved Translate, JAX, and Transformers. Except that these advances have made other companies an existential threat for Google. 2 years ago it was hard to imagine what could topple Google. Now a lo…

It truly feels like Google Brain could be considered Google's equivalent of Bell Labs in the 70s.

Re: Why did Google Brain exist?

#207

Post by quickthrower2 »

dekhn wrote: post #148
dr_dshiv wrote: post #7

Lots of great insight. Here’s one: “Given the long timelines of a PhD program, the vast majority of early ML researchers were self-taught crossovers from other fields. This created the conditions for excellent interdisciplinary work to happen. This transitional anomaly is unfortunately mistaken by most people to be an inherent property of machine learning to upturn existing fields. It is not. Today, the vast majority…

As somebody who has crossed the line between ML and chemistry many times, I would love to see: more ML researchers who know chemistry, more chemistry researchers who know ML, and best of all, fully cross-disciplinary researchers who are both masters of chemistry and ML, as those are the ones who move the field farthest, fastest.

Society is not structured to encourage this. Getting a job sooner is more lucrative. Any breakthough you make having studied for a couple of decades is property of a corporation not you.

Re: Why did Google Brain exist?

#208

Post by oofbey »

tdullien wrote: post #38
vl wrote: post #4

>PyTorch/Nvidia GPUs easily overtaking TensorFlow/Google TPUs. TF lost to PyTorch, and this is Google’s fault - TF APIs are both insane and badly documented. But nothing comes close to performance of Google’s TPU exaflop mega-clusters. Nvidia is not even in the same ballpark.

TF in its first version was stellarly misdesigned. It was infuriatingly difficult to use, particularly if you were of the "I just want to write code and have it autodiffed + SGDed" school, I found it crazy to use Python to manually construct a computational graph...

TF1 was pretty rough to use, but beat the pants off Theano for usability, which was really the best thing going before it. Sure it was slow as dirt ("tensorslow") even though the awkward design was justified on being able to make it fast. But it was by far the best thing going for a long time.

Google really killed TF with the transition to TF2. Backwards incompatible everything? This only makes sense if you live in a giant monorepo with tools that rewrite everybody's code whenever you change an interface. (e.g. inside google). On the outside it took TF's biggest asset and turned it into a liability. Every library, blog post, stackoverflow post, etc talking about TF was now wrong. So anybody trying to figure out how to get started or build something was forced into confusion. Not sure about this, but I suspect it's Chollet's fault.

Re: Why did Google Brain exist?

#209

Post by Quekid5 »

dr_dshiv wrote: post #7

Lots of great insight. Here’s one: “Given the long timelines of a PhD program, the vast majority of early ML researchers were self-taught crossovers from other fields. This created the conditions for excellent interdisciplinary work to happen. This transitional anomaly is unfortunately mistaken by most people to be an inherent property of machine learning to upturn existing fields. It is not. Today, the vast majority…

Do you have any anecdata on topologists?

Re: Why did Google Brain exist?

#210

Post by smaddox »

kenjackson wrote: post #199
dr_dshiv wrote: post #7

Lots of great insight. Here’s one: “Given the long timelines of a PhD program, the vast majority of early ML researchers were self-taught crossovers from other fields. This created the conditions for excellent interdisciplinary work to happen. This transitional anomaly is unfortunately mistaken by most people to be an inherent property of machine learning to upturn existing fields. It is not. Today, the vast majority…

Chemistry is a centuries old discipline, that people study undergrad a full four years before getting a PhD in the field of chemistry. ML is a, practically speaking, 15 year old field that PhDs often begin to study after a couple of AI courses in undergrade and a specific track in grad school (while they study other parts of CS as part of their early graduate CS work). There's just way less context in ML than Chemist…

I don't think that's really true, but you don't need the full context in ML in order to apply it to your field of interest. Just like you don't need the full context in Physics to apply it to an engineering problem.

Some of the most successful ML researchers have read several decades of research papers. It's not uncommon to see references to papers from the 70s or 90s.

Edit: No doubt that the relevant parts of stats, random matrix theory, and ML is a newer field than Physics or Chemistry, though.

Post reply on HN