Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why did Google Brain exist?

moderndescartes.com

Post reply on HN
Print view View original

291–300 of 328 posts

Re: Why did Google Brain exist?

#291

Post by jltsiren »

ramraj07 wrote: post #233
anonymouskimmer wrote: post #58

Earlier quoted context omitted.

> Second, there is a small chance you're absolutely right, and you've just obviated a whole field from first principles. Mostly when I read about things like this happening, it's happening to a formerly intractable problem in mathematics. Do you have examples outside of math?

The only time I’ve seen this happen is AlphaFold. Every other time some math or software guy came into biology it worked exactly like in xkcd.

Biology has already been transformed by mathematics, statistics, and CS for decades. These days, if something like in that xkcd happens, those people are probably not even familiar with the relevant parts of math / stats / CS.

Re: Why did Google Brain exist?

#292

Post by ukuina »

oofbey wrote: post #208
tdullien wrote: post #38

Earlier quoted context omitted.

TF in its first version was stellarly misdesigned. It was infuriatingly difficult to use, particularly if you were of the "I just want to write code and have it autodiffed + SGDed" school, I found it crazy to use Python to manually construct a computational graph...

TF1 was pretty rough to use, but beat the pants off Theano for usability, which was really the best thing going before it. Sure it was slow as dirt ("tensorslow") even though the awkward design was justified on being able to make it fast. But it was by far the best thing going for a long time. Google really killed TF with the transition to TF2. Backwards incompatible everything? This only makes sense if you live in a…

Echoes the AngularJS 1.x to Angular2 transition.

Every attempt at a "clean break" new version of a commonly-used platform leads to such long-term weakness, yet the temptation to piggyback off of the mindshare/existing branding forces companies to avoid calling it a new platform.

Re: Why did Google Brain exist?

#293

Post by kergonath »

selimthegrim wrote: post #237
kergonath wrote: post #159

Earlier quoted context omitted.

> I’ve seen repeatedly that it’s much harder for a ML PhD to learn chemistry than for a chemist to learn ML I can confirm. We regularly look for people to write some computational physics code, and recently for people using ML to solve solid state physics problems. It’s way easier to bring a good physicist or chemist to a decent CS level (either ML or HPC) than the other way around.

What sort of problems are you trying to solve?

Lots of things, but mostly things like designing software architectures for massively parallel codes implementing fancy physics or coming up with ML models to predict complicated properties (things like wear, radiation, or corrosion resistance for which we just do not have any comprehensive model) or explore humongous problem spaces (things like predicting simple properties of very complex materials with 5 to 8 elements and complex microstructures like superalloys).

Neither the Physics nor the CS are cutting edge (that’s why we do not necessarily reject people with limited experience in Physics, though they need to show motivation and abilities to learn); what is is the combination of both.

We’d like to be as close to the cutting edge on ML as possible, though, because it’s a significant competitive advantage to be able to use fancy new techniques before our friendly competitors. But as I said it seems to be easier to train a Physics or Chemistry undergrad to get some feeling about how ML works than to train a CS undergrad to have some intuition about the Physics. And intuition is critical to detect when models hallucinate and get off the rails.

Re: Why did Google Brain exist?

#294

Post by fakedang »

stolsvik wrote: post #287
cubefox wrote: post #225

Earlier quoted context omitted.

Bard uses a smaller model currently, which was announced before release. > We’re releasing it initially with our lightweight model version of LaMDA. This much smaller model requires significantly less computing power, enabling us to scale to more users, allowing for more feedback. [1] > Bard is powered by a research large language model (LLM), specifically a lightweight and optimized version of LaMDA, and will be upd…

”We’re releasing something useless and uninteresting, so that we can get more user feedback” Is it only me that sees a problem here?

"We are releasing something that's too underdeveloped, because Wall Street demands we release something. We're using a smaller training set because we have to rush to market with our smaller, stupider model ASAP."

Re: Why did Google Brain exist?

#295

Post by ramraj07 »

jltsiren wrote: post #291
ramraj07 wrote: post #233

Earlier quoted context omitted.

The only time I’ve seen this happen is AlphaFold. Every other time some math or software guy came into biology it worked exactly like in xkcd.

Biology has already been transformed by mathematics, statistics, and CS for decades. These days, if something like in that xkcd happens, those people are probably not even familiar with the relevant parts of math / stats / CS.

What exactly has it transformed? Nothing has fundamentally changed in biology, you still have to run western blots and give mice cancer. A great example of “math” solving biology is super resolution - which was just a dud.

Re: Why did Google Brain exist?

#296

Post by qumpis »

a_bonobo wrote: post #243
jltsiren wrote: post #235

Earlier quoted context omitted.

This is more about ML than CS. ML is fundamentally about developing general-purpose algorithms applicable to a wide range of problems. If your job is using ML to solve problems in chemistry, it's more about chemistry than ML, and a chemistry background is more important than an ML background. It's unlikely that you have to develop novel ML methods for the problems you are facing. I've seen the opposite in bioinformat…

I've seen both in bioinformatics, because the field is so wide now. 1) Bioinformatics as tool-building, algorithm-dev: you're right, you don't need to know much biology there if the problem is defined well. 2) Bioinformatics as a tool to answer biological questions: here I've seen ML-background people really struggle, either developing stuff that's not useful or reinventing-the-wheel-but-now-it's-deep-learning. I've…

In regards to 2), it sounds a bit directionless to be proposing stuff people don't need? Isn't that more of a problem of selecting relevant problems to solve, and getting supervision on your ideas?

Re: Why did Google Brain exist?

#297

Post by qumpis »

cubefox wrote: post #225
oofbey wrote: post #216

Earlier quoted context omitted.

They are behind because what they have released sucks. Have you tried Bard? It's dumb. Like you're talking to some 20th century gimmick dumb. GPT4 is far from perfect, but when it makes mistakes and you point them out, it understands and tries to adapt. Bard just repeats itself saying the same stupid things like a casette-tape answering machine. If you ask a googler about this, they typically assume GPT is just as st…

Bard uses a smaller model currently, which was announced before release. > We’re releasing it initially with our lightweight model version of LaMDA. This much smaller model requires significantly less computing power, enabling us to scale to more users, allowing for more feedback. [1] > Bard is powered by a research large language model (LLM), specifically a lightweight and optimized version of LaMDA, and will be upd…

I remember the CEO of google saying that more capable models will be releases a few months (a month?) ago already. Either they delayed or the model is just as bad (maybe they released after the announcement of coding in 20 languages?)

Re: Why did Google Brain exist?

#298

Post by activitypea »

liuyipei wrote: post #162

Google has good engineers and a long history of high throughput computing. This, combined with a lack of understanding what ML research is like (versus deployment), led to the original TF1 API. Also, the fact that google has good engineers working in a big bureaucracy probably hid a lot of the design problems as well. TF2 was a total failure, in that TF1 can do a few things really well when you get the hang of it, bu…

> google has good engineers working in a big bureaucracy probably hid a lot of the design problems as well.

I feel like this is true of every google product in the last decade, maybe more. Their customer products, but especially their dev tools like Angular and k8s screams "We were so preoccupied with whether we could, we didn’t stop to think if we should."

Re: Why did Google Brain exist?

#299

Post by jltsiren »

ramraj07 wrote: post #295
jltsiren wrote: post #291

Earlier quoted context omitted.

Biology has already been transformed by mathematics, statistics, and CS for decades. These days, if something like in that xkcd happens, those people are probably not even familiar with the relevant parts of math / stats / CS.

What exactly has it transformed? Nothing has fundamentally changed in biology, you still have to run western blots and give mice cancer. A great example of “math” solving biology is super resolution - which was just a dud.

Genomics, for example. The entire field could not exist without extensive algorithmic research done over several decades. Even today, many of the key people in the field have a background in CS and mathematics.

Theoretical ecology is a bit more old-school answer. Many mathematicians have been involved in that field.

Re: Why did Google Brain exist?

#300

Post by activitypea »

khazhoux wrote: post #126
brilee wrote: post #115

Earlier quoted context omitted.

I've been quite careful not to divulge anything confidential, and anything that is remotely close to sensitive has publicly accessible citations. My opinions about Google are tantamount to discussions about workplace conditions, and it would be very bad for society if ex-employees were not allowed to discuss those.

But all your opinions are informed by your years of TGIFs and internal emails and discussions and presentations and your insider perspective. When you talk about promotions, or internal values and prioritizations, you are leveraging info gained privately. If I'm wrong and nothing in your contract or on-boarding said you shouldn't talk about internals, then my bad. But I suspect they were as clear with you as they wer…

bootlicker lol
Post reply on HN