Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why did Google Brain exist?

moderndescartes.com

Post reply on HN
Print view View original

271–280 of 328 posts

Re: Why did Google Brain exist?

#271

Post by mirker »

disgruntledphd2 wrote: post #41
belval wrote: post #13

Earlier quoted context omitted.

There is a first mover handicap there though. TF1.0 included a bunch of things that were harder to understand like tf.Session(). PyTorch was inspired from the good parts and "we will eager-everything". Internally I'm sure there was a lot of debate in the TF team that culminated with TF2.0, but by that time the damage was done and people saw PyTorch as easier.

Nope, Pytorch was inspired by the Lua version of Torch which well predates Tensor flow. To be fair, basically every other DL framework made the same mistake though. Also, tensorflow was a total nightmare to install while Pytorch was pretty straightforward, which definitely shouldn't be discounted.

PyTorch examples were also cleaner. torchvision had ResNet training batteries included while TF had role your own or clone some weird Keras repository.

Re: Why did Google Brain exist?

#272

Post by petesergeant »

quickthrower2 wrote: post #255
randmeerkat wrote: post #224

Earlier quoted context omitted.

> Society is not structured to encourage this. I wish more people understood this and the value that it would add to society. For some reason most people understand how roads and transit improve commerce, but fail to understand how education and social support for the individual does as well.

Sport incentives work out nicely for getting a supply of top talent. Big money through sponsorship/ads/tickets, fairly formulaic system to get better at a sport (top coaches/teams, get fitter, practice a lot), an easy way to identify progress (wins games, scores goals). But for sciences, someone could do great work for 20 years and produce obscure papers that might stay obscure, or may unlock the mystery of the unive…

Would employees of non-sports industries accept the weird-ass ways that sports contracts work? Major League Pharma Research is certainly intriguing

Re: Why did Google Brain exist?

#273

Post by anonymouskimmer »

ramraj07 wrote: post #233
anonymouskimmer wrote: post #58

Earlier quoted context omitted.

> Second, there is a small chance you're absolutely right, and you've just obviated a whole field from first principles. Mostly when I read about things like this happening, it's happening to a formerly intractable problem in mathematics. Do you have examples outside of math?

The only time I’ve seen this happen is AlphaFold. Every other time some math or software guy came into biology it worked exactly like in xkcd.

That's a good one.

Re: Why did Google Brain exist?

#274

Post by mirker »

bitL wrote: post #27
belval wrote: post #13

Earlier quoted context omitted.

There is a first mover handicap there though. TF1.0 included a bunch of things that were harder to understand like tf.Session(). PyTorch was inspired from the good parts and "we will eager-everything". Internally I'm sure there was a lot of debate in the TF team that culminated with TF2.0, but by that time the damage was done and people saw PyTorch as easier.

I think the main problem was debugging tensors on the fly, impossible with TF/Keras, but completely natural to PyTorch. Most researchers needed to sequentially observe what is going on in tensors (histograms etc.) and even doing backprop for their newly constructed layers by hand and that was difficult with TF.

Nah, TF has had dynamic execution since TF2 and it’s still losing users, it seems. The execution model and API is simply more complicated. What’s a session, placeholder, constant, tensor, …? PyTorch was sold as numpy with GPU support and it is pretty close to that. JAX is an attempt to approach language simplicity and purity.

Re: Why did Google Brain exist?

#275

Post by VirusNewbie »

mattkrause wrote: post #176
asciimov wrote: post #124

Earlier quoted context omitted.

> "[...] I’ve seen repeatedly that it’s much harder for a ML PhD to learn chemistry than for a chemist to learn ML.” That's good ol' academic gatekeeping for ya, available wherever PhD's are found.

There’s more to it than that. CS is unusually easy to learn on your own. You can mess around, build intuition, and check your progress—-all on your own and in your pyjamas. It’s easy to roll things back if you make a mistake, and hard to do lasting damage. There are tons of useful resources, often freely available. Thus, you can get to an intermediate level quickly and cheaply. Wet-lab fields have none of that. Hands…

That's because academia is ass backwards with it's attempt at gatekeeping high knowledge/low skill fields.

Re: Why did Google Brain exist?

#276

Post by mirker »

oofbey wrote: post #208
tdullien wrote: post #38

Earlier quoted context omitted.

TF in its first version was stellarly misdesigned. It was infuriatingly difficult to use, particularly if you were of the "I just want to write code and have it autodiffed + SGDed" school, I found it crazy to use Python to manually construct a computational graph...

TF1 was pretty rough to use, but beat the pants off Theano for usability, which was really the best thing going before it. Sure it was slow as dirt ("tensorslow") even though the awkward design was justified on being able to make it fast. But it was by far the best thing going for a long time. Google really killed TF with the transition to TF2. Backwards incompatible everything? This only makes sense if you live in a…

The APIs were messed up early on, which is a reason TF2 happened. Every team started making their own random implementations of stuff. You had the TF Slim API, you had Keras, etc. The API just got fatter and fatter and then libraries would make cross dependencies to bake in the API mistakes.

Re: Why did Google Brain exist?

#277

Post by PartiallyTyped »

kenjackson wrote: post #270
PartiallyTyped wrote: post #267

Earlier quoted context omitted.

DL is a 15 year old field, ML is far older than that.

That’s why I said practically speaking. And realistically I don’t think you can even generously consider anything prior to perceptrons.

ML is so much more than DL / NNs. I recommend that you open a book like Bishop or ESL, the overwhelming majority of ML has been non DL going back to the 70s.

Re: Why did Google Brain exist?

#278

Post by _delirium »

kenjackson wrote: post #270
PartiallyTyped wrote: post #267

Earlier quoted context omitted.

DL is a 15 year old field, ML is far older than that.

That’s why I said practically speaking. And realistically I don’t think you can even generously consider anything prior to perceptrons.

> realistically I don’t think you can even generously consider anything prior to perceptrons

I mean, sure, but for the practical reason that perceptrons (proposed 1943, implemented 1958) are about as old as digital computers, and ML as a field is tied up pretty strongly with the existence of computers.

Re: Why did Google Brain exist?

#279

Post by deltree7 »

ramraj07 wrote: post #233
anonymouskimmer wrote: post #58

Earlier quoted context omitted.

> Second, there is a small chance you're absolutely right, and you've just obviated a whole field from first principles. Mostly when I read about things like this happening, it's happening to a formerly intractable problem in mathematics. Do you have examples outside of math?

The only time I’ve seen this happen is AlphaFold. Every other time some math or software guy came into biology it worked exactly like in xkcd.

The biggest innovation of this decade, LLMs was created by Math/CS not linguistics

Re: Why did Google Brain exist?

#280

Post by deltree7 »

anonymouskimmer wrote: post #58
javajosh wrote: post #26

Earlier quoted context omitted.

That's a great xkcd, but there are 2 upsides to this arrogant approach. First, arrogance is a nerd-snipe maximizer. Second, there is a small chance you're absolutely right, and you've just obviated a whole field from first principles. It doesn't happen often, but when it does happen and there is no clout like "emporer's new clothes" clout. EDIT: The downside, of course, is that you appear arrogant, and people won't l…

> Second, there is a small chance you're absolutely right, and you've just obviated a whole field from first principles. Mostly when I read about things like this happening, it's happening to a formerly intractable problem in mathematics. Do you have examples outside of math?

LLMs weren't invented by Naom Chomsky but Math/CS guy with a simple "Attention is all you need"
Post reply on HN