Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why did Google Brain exist?

moderndescartes.com

Post reply on HN
Print view View original

241–250 of 328 posts

Re: Why did Google Brain exist?

#241

Post by j45 »

dr_dshiv wrote: post #7

Lots of great insight. Here’s one: “Given the long timelines of a PhD program, the vast majority of early ML researchers were self-taught crossovers from other fields. This created the conditions for excellent interdisciplinary work to happen. This transitional anomaly is unfortunately mistaken by most people to be an inherent property of machine learning to upturn existing fields. It is not. Today, the vast majority…

Interdisciplinary and intersectional skillsets are critically valuable.

Separation of concerns especially at the beginning innovation stages can be more of an inhibitor than accelerator of success. Scaling and growth is another thing.

A similar pattern existed in the late 90s with web developers coming from many different industries with their domain knowledge and domain insight.

The code and frameworks were early but the insights of what was a problem most pressing to solve.

Re: Why did Google Brain exist?

#242

Post by throwaway2037 »

epolanski wrote: post #182
kergonath wrote: post #159

Earlier quoted context omitted.

> I’ve seen repeatedly that it’s much harder for a ML PhD to learn chemistry than for a chemist to learn ML I can confirm. We regularly look for people to write some computational physics code, and recently for people using ML to solve solid state physics problems. It’s way easier to bring a good physicist or chemist to a decent CS level (either ML or HPC) than the other way around.

It's the same reason analysts come from math rather than economy degrees. You can teach a mathematician what he needs to know about finance, you can hardly do the opposite.

In my experience, people with degrees in math, physics, chemistry, biology/medicine, astronomy, computer science have higher/stronger intellectual rigor that can be re-applied to other fields, e.g., finance. Also, those areas of study are much harder than economics. Partly, it is a self-selection process. Yes, there are some with economics degrees whom are very bright, but they probably could have majored in any of the sciences mentioned earlier.

Re: Why did Google Brain exist?

#243

Post by a_bonobo »

jltsiren wrote: post #235
mattkrause wrote: post #176

Earlier quoted context omitted.

There’s more to it than that. CS is unusually easy to learn on your own. You can mess around, build intuition, and check your progress—-all on your own and in your pyjamas. It’s easy to roll things back if you make a mistake, and hard to do lasting damage. There are tons of useful resources, often freely available. Thus, you can get to an intermediate level quickly and cheaply. Wet-lab fields have none of that. Hands…

This is more about ML than CS. ML is fundamentally about developing general-purpose algorithms applicable to a wide range of problems. If your job is using ML to solve problems in chemistry, it's more about chemistry than ML, and a chemistry background is more important than an ML background. It's unlikely that you have to develop novel ML methods for the problems you are facing. I've seen the opposite in bioinformat…

I've seen both in bioinformatics, because the field is so wide now.

1) Bioinformatics as tool-building, algorithm-dev: you're right, you don't need to know much biology there if the problem is defined well.

2) Bioinformatics as a tool to answer biological questions: here I've seen ML-background people really struggle, either developing stuff that's not useful or reinventing-the-wheel-but-now-it's-deep-learning. I've seen ML people present their fancy plant disease image detector which turned out to be pretty good at spotting 'yellow' - very good at training accuracy and benchmarks, does not add anything to what people in the field are doing.

Re: Why did Google Brain exist?

#244

Post by throwaway2037 »

ramraj07 wrote: post #232
qumpis wrote: post #15

Earlier quoted context omitted.

I'm yet to see an ML PhD be required to learn chemistry to a similar extent that chemists would need to doing ML (especially at research level)

I’m pretty sure the author is implying that the new crop of ML PhDs are just not a smart group of people - at least the level of intelligence required to do truly transformative things with ML in any field. I think what you’re saying is a commonly found attitude that relates to this topic: it’s pretty limiting to think a cursory knowledge of a field is sufficient to go change it. That’s likely why most “use ML to sol…

    AlphaFold succeed because the ML engineers truly understood the fundamental tenets of the topic and exploited it
What makes you think it wasn't chemists or biologists who learned enough ML to solve the problem?

Re: Why did Google Brain exist?

#245

Post by sundarurfriend »

seatsniffer wrote: post #163
nighthawk454 wrote: post #157

Earlier quoted context omitted.

It's become somewhat of a meme, where Schmidhuber seemingly tries to claim credit for nearly everything. I _think_ it's because he published ideas back in the 90s or so that weren't fully executable/realized at the time, and later people figured out how to actually flesh them out do it, and supposedly didn't cite him appropriately/enough. Often the ideas weren't exactly the same - but rather he claims they're derivat…

Thanks for taking the time to explain. I'll check out the link also.

Just to make it clear, you're hearing one side of the argument. There are also a lot of people who firmly believe that Schmidhuber was wrongfully deprived of recognition he deserved.

I don't have a strong opinion one way or another, but the issue isn't as clear cut as your parent comment might have made it sound like.

Re: Why did Google Brain exist?

#246

Post by theGnuMe »

dr_dshiv wrote: post #7

Lots of great insight. Here’s one: “Given the long timelines of a PhD program, the vast majority of early ML researchers were self-taught crossovers from other fields. This created the conditions for excellent interdisciplinary work to happen. This transitional anomaly is unfortunately mistaken by most people to be an inherent property of machine learning to upturn existing fields. It is not. Today, the vast majority…

The ML folks pretty much nailed protein folding though. Where any of them phds in molecular bio or chemistry?

Re: Why did Google Brain exist?

#247

Post by Nevermark »

Hmm. Anything that slows Google down and maintains a diversity of leaders in the field is ok with me.

Imagine a host of "helpful" Google AI's, Facebook AI's, Amazon AI's, etc., that know their very existence depends them monetizing you more effectively than competitive AI's.

Of course, the first versions will be very helpful. But continuous efforts to remain "the most helpful" will cost a lot, and eventually need to pay for themselves.

Re: Why did Google Brain exist?

#248

Post by vl »

sytelus wrote: post #238
vl wrote: post #4

>PyTorch/Nvidia GPUs easily overtaking TensorFlow/Google TPUs. TF lost to PyTorch, and this is Google’s fault - TF APIs are both insane and badly documented. But nothing comes close to performance of Google’s TPU exaflop mega-clusters. Nvidia is not even in the same ballpark.

NVidia A100 DGX Superpod is equivalent to exaflop TPU pod. No?

Perhaps NVidia is close now. A bit hard to say without specific hardware info.

Google’s were already available 5-6 years ago. And probably current versions are even faster. They have super fast optical interconnects in torus or hyper-torus configuration that allow synchronous weight updates on 1k+ TPUs. This leads to dramatically lower training times and less noise, which leads to better-performing models. I.e. you can’t even train model to the same level on traditional GPUs.

Once they started to get deployed, models that trained for 3 weeks on 30 GPUs were trained in 30 minutes on 1k TPU cluster.

All this reiterated main point in the article - Google had tremendous lead and wasted it due to the lack of vision and product execution ability.

Re: Why did Google Brain exist?

#249

Post by throwaway2037 »

disgruntledphd2 wrote: post #41
belval wrote: post #13

Earlier quoted context omitted.

There is a first mover handicap there though. TF1.0 included a bunch of things that were harder to understand like tf.Session(). PyTorch was inspired from the good parts and "we will eager-everything". Internally I'm sure there was a lot of debate in the TF team that culminated with TF2.0, but by that time the damage was done and people saw PyTorch as easier.

Nope, Pytorch was inspired by the Lua version of Torch which well predates Tensor flow. To be fair, basically every other DL framework made the same mistake though. Also, tensorflow was a total nightmare to install while Pytorch was pretty straightforward, which definitely shouldn't be discounted.

    tensorflow was a total nightmare to install while Pytorch was pretty straightforward
Hat tip for this comment. On HN, I read some great commentary about "time to achieve first HTTP 200 with your REST API". Regarding installed software libraries, lower friction to achieve "Hello, World!" is important.

Re: Why did Google Brain exist?

#250

Post by sulam »

cauthon wrote: post #234
epolanski wrote: post #182

Earlier quoted context omitted.

It's the same reason analysts come from math rather than economy degrees. You can teach a mathematician what he needs to know about finance, you can hardly do the opposite.

That’s the opposite of what OP is observing. It’s easier to teach a domain expert “good enough” quantitative and technical skills than to teach a pure quant “good enough” domain expertise (and the corresponding intuition)

I think you’re reading this backwards, or perhaps it was edited? Mathematics is surely the more pure discipline compared with economics, so physics : CS :: mathematics : finance is the right ordering.
Post reply on HN