Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why did Google Brain exist?

moderndescartes.com

Post reply on HN
Print view View original

211–220 of 328 posts

Re: Why did Google Brain exist?

#211

Post by fancy_pantser »

Quekid5 wrote: post #209
dr_dshiv wrote: post #7

Lots of great insight. Here’s one: “Given the long timelines of a PhD program, the vast majority of early ML researchers were self-taught crossovers from other fields. This created the conditions for excellent interdisciplinary work to happen. This transitional anomaly is unfortunately mistaken by most people to be an inherent property of machine learning to upturn existing fields. It is not. Today, the vast majority…

Do you have any anecdata on topologists?

Topologists will occasionally come to work with their clothes on inside out and insist nothing is wrong.

Re: Why did Google Brain exist?

#212

Post by cubefox »

oofbey wrote: post #204
anonylizard wrote: post #193

Earlier quoted context omitted.

Well, where exactly is this progress? Where is Google's answer to GPT-4? Why weren't the 'researchers and developers' making a GPT-4 equivalent? Turns out you sometimes you need a top down, centralised vision to execute on projects. When the goal is undefined, you can allow researchers to run free and explore, now its full on wartime, with clear goals (make GPT-5,6,7....).

Google is fundamentally allergic to top-down management. Most googlers will reject any attempt to be told what to do as wrong, because lots of IC's voting with their feet are smarter than any (google) exec at figuring out what to do. Last time Google got spooked by a competitor was Facebook, and they built Google Plus in response. We all know that was an utter failure. Googlers could escape that one with their egos i…

I don't think they are behind just because they have released less stuff. They had LaMDA way before ChatGPT, and they had multimodal models (both by DeepMind and by Google Brain) well before OpenAI.

Re: Why did Google Brain exist?

#213

Post by sdenton4 »

kevviiinn wrote: post #71
selimthegrim wrote: post #66

Earlier quoted context omitted.

Well I think the issue is more of if you’re Genentech and you need ML people and can’t afford to pay them you’re probably better off retraining chemistry PhDs.

I think you missed my point. Genentech, AFAIK, was not doing research on machine learning as in the principles of how machine learning works and how to make it better. They do biotech research which uses applied machine learning. You don't need a PhD in ML to apply things that are already known

Sadly a lot of foundational ML research works for single-label image classification and not much else. ImageNet is a niche problem and way too much ML research is over-indexed on it. If you can make your problem look like ImageNet, you're going to do OK, but if not you effectively need to re-invent the wheel...

Re: Why did Google Brain exist?

#214

Post by cratermoon »

mattkrause wrote: post #176
asciimov wrote: post #124

Earlier quoted context omitted.

> "[...] I’ve seen repeatedly that it’s much harder for a ML PhD to learn chemistry than for a chemist to learn ML.” That's good ol' academic gatekeeping for ya, available wherever PhD's are found.

There’s more to it than that. CS is unusually easy to learn on your own. You can mess around, build intuition, and check your progress—-all on your own and in your pyjamas. It’s easy to roll things back if you make a mistake, and hard to do lasting damage. There are tons of useful resources, often freely available. Thus, you can get to an intermediate level quickly and cheaply. Wet-lab fields have none of that. Hands…

On the flip side, software development and engineering rigor is largely absent in academics, as has been discussed previously here on HN. This is enough of an issue to make replicating research even when the code and data are provided, but it's an even bigger issue trying to turn academic code into a product.

Re: Why did Google Brain exist?

#215

Post by cratermoon »

fancy_pantser wrote: post #211
Quekid5 wrote: post #209

Earlier quoted context omitted.

Do you have any anecdata on topologists?

Topologists will occasionally come to work with their clothes on inside out and insist nothing is wrong.

Topologists can't tell the difference between a coffee cup and a donut, and will occasionally make a mess trying to dunk their donut in their coffee.

Re: Why did Google Brain exist?

#216

Post by oofbey »

cubefox wrote: post #212
oofbey wrote: post #204

Earlier quoted context omitted.

Google is fundamentally allergic to top-down management. Most googlers will reject any attempt to be told what to do as wrong, because lots of IC's voting with their feet are smarter than any (google) exec at figuring out what to do. Last time Google got spooked by a competitor was Facebook, and they built Google Plus in response. We all know that was an utter failure. Googlers could escape that one with their egos i…

I don't think they are behind just because they have released less stuff. They had LaMDA way before ChatGPT, and they had multimodal models (both by DeepMind and by Google Brain) well before OpenAI.

They are behind because what they have released sucks. Have you tried Bard? It's dumb. Like you're talking to some 20th century gimmick dumb. GPT4 is far from perfect, but when it makes mistakes and you point them out, it understands and tries to adapt. Bard just repeats itself saying the same stupid things like a casette-tape answering machine.

If you ask a googler about this, they typically assume GPT is just as stupid as bard. Or say something like "so GPT is just trained on more data - we can do that." As if nothing's wrong.

Re: Why did Google Brain exist?

#217

Post by cratermoon »

simonster wrote: post #125

I work for Google Brain. I remember meeting Brian at a conference and I have nothing but good things to say about him. That said, I think Brian is underestimating the extent to which the Brain/DeepMind merger is happening because it's what researchers want. Many of us have a strong sense that the future of ML involves models built by large teams in industry environments. My impression is that the goal of the merger i…

I'm having trouble keeping Brain/Brian straight.

Re: Why did Google Brain exist?

#218

Post by cratermoon »

alphabetting wrote: post #196
dekhn wrote: post #187

Earlier quoted context omitted.

I have infinite respect for the process that Waymo followed to get to where they are. And I'm impressed that Google continued to fund the project and move it forward even when it represents such a long-term bet. but it's not a product that has any real revenue. and most car companies keep their distance from google self-driving tech because they're afraid. afraid google wants to put them out of business. It's unclear…

I was just disputing "X exists as a press-release generation system, not as a real technology creation system." Definitely agree the path to profitability will be tough.

If the technology is not in production, is the creation complete?

Re: Why did Google Brain exist?

#219

Post by jpace121 »

drakenot wrote: post #205
ironman1478 wrote: post #203

"I’ve seen repeatedly that it’s much harder for a ML PhD to learn chemistry than for a chemist to learn ML. (This may be survivorship bias; the only chemists I encounter are those that have successfully learned ML, whereas I see ML researchers attempt and fail to learn chemistry all the time.)" This is something that rings really true to me. I work in imaging and it's just very clear that there are groups of people i…

This seems to kind of be the opposite opinion of The Bitter Lesson[0]. [0] http://www.incompleteideas.net/IncIdeas/BitterLesson.html

I don’t think it’s really the opposite opinion.

To pick on the game of Go as an example, the insight from the Bitter Lesson is the best method to use for Go is probably going to be the most broad method that abuses the magic of brute force the most, compared to a method that really encodes the best existing strategies of Go.

I think what OP is referring to and what I’ve observed is that in some fields you have to have a certain amount of expertise to just understand the rules of the game, and there are a lot of applications where someone approaching a problem with the goal of applying ML can’t quite get over the hump of understanding all the rules.

Re: Why did Google Brain exist?

#220

Post by sidibe »

drakenot wrote: post #205
ironman1478 wrote: post #203

"I’ve seen repeatedly that it’s much harder for a ML PhD to learn chemistry than for a chemist to learn ML. (This may be survivorship bias; the only chemists I encounter are those that have successfully learned ML, whereas I see ML researchers attempt and fail to learn chemistry all the time.)" This is something that rings really true to me. I work in imaging and it's just very clear that there are groups of people i…

This seems to kind of be the opposite opinion of The Bitter Lesson[0]. [0] http://www.incompleteideas.net/IncIdeas/BitterLesson.html

Someone still needs to have enough domain knowledge to figure out what needs solving and how to validate that it was solved.

Being able to frame problems, and being able to cobble together the data and infra for the AI to solve them are the most valuable things in the medium term.

Post reply on HN