Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why did Google Brain exist?

moderndescartes.com

Post reply on HN
Print view View original

101–110 of 328 posts

Re: Why did Google Brain exist?

#101

Post by snapcaster »

asdfman123 wrote: post #95
snapcaster wrote: post #63

Earlier quoted context omitted.

I keep hearing this, but Bard sucks so badly when i've tried to use it like GPT-4 or compare results its like night and day. What makes you so confident they have "secret" LLMs that are superior?

Have you tried PaLM? I work for Google and have been playing with it. It’s pretty good. The decision to release Bard, an LLM that was clearly not as good as ChatGPT, struck me as reactive and is why people think Google is behind. I’d think so too if I had just demoed Bard.

No, but would love to try it. I'm using these models 20-30 times a day throughout the average work day for random tasks so have a pretty good sense of performance levels. Didn't think it was available to public yet but just saw it's apparently on google cloud now, i'll have to try it out. How do you compare Palm with GPT4 if you've had a chance to try both?

Re: Why did Google Brain exist?

#102

Post by snapcaster »

jeffbee wrote: post #99
snapcaster wrote: post #63

Earlier quoted context omitted.

I keep hearing this, but Bard sucks so badly when i've tried to use it like GPT-4 or compare results its like night and day. What makes you so confident they have "secret" LLMs that are superior?

Bard is in full-scale production to all U.S. users for free. GPT-4 costs $20/month. Rather a big difference in the economics of the situation. Also it's pretty clear that even the $20 is highly subsidized. Microsoft is willing to incinerate almost any amount of money to harm Google.

Free but unuseably bad seriously have you tried it? compared it to even GPT-3? it really really sucks

Re: Why did Google Brain exist?

#103

Post by return_to_monke »

potatolicious wrote: post #54
asdfman123 wrote: post #46

My theory is that broadly, tech learned not to act like Microsoft in the 90s -- closed off, anti-competitive, unpopular -- but swung too far in the opposite direction. Google has been basically giving away technology for free, which was easy because of all the easy money. It's good for reputation and attracting the best talent. That is, until a competitor starts to threaten to overtake you with the technology you gav…

Ehh, I mildly disagree. I'm not entirely bought-in on the notion that giving one's technical innovations for free is obviously the right move, but I don't think it's why the company is in trouble. Chrome is experiencing unprecedented competition because it faltered on the product. Chrome went from synonymous with fast-and-snappy to synonymous with slow-and-bloated. Likewise Google invented transformers - but the sin…

> Chrome is experiencing unprecedented competition

From where? 65% globally use Chrome (https://en.m.wikipedia.org/wiki/Usage_share_of_web_browsers).

The only widespread competition is from Safari and FF, both of which have been around longer than it.

Re: Why did Google Brain exist?

#104

Post by seatsniffer »

logicchains wrote: post #72
Scea91 wrote: post #43

Earlier quoted context omitted.

Seems more than a nitpick to me. I find the essay interesting but this line raised some distrust in me. How can someone have these deep insights into Google's ML strategy and the evolution of the field and simultaneously think LSTMs were invented by Google in 2014?

>How can someone have these deep insights into Google's ML strategy and the evolution of the field and simultaneously think LSTMs were invented by Google in 2014? It may not have been accidental; there's a deliberate movement among some people in the ML community to deny Jürgen Schmidhuber credit for inventing LSTMs and GANs.

That's something I hadn't heard about. Is there any particular reason for this?

Re: Why did Google Brain exist?

#105

Post by jrochkind1 »

khazhoux wrote: post #59

> I sat on it because I wasn’t sure of the optics of posting such an essay while employed by Google Brain. But then Google made my decision easier by laying me off in January. My severance check cleared... I'm really baffled by how people think it's OK to write public accounts of their previous ( and sometime current! ) employers' inner workings. This guy got paid a shitload of money to do work and to keep all intern…

> he's just going for internet brownie points. It's just an attempt to squeeze a bit more personal benefit out of your (now-ended) employment.

So, many researcher types (and not only them, but also including many of us) are motivated by — find it personally rewarding at a psychological level — to share their thoughts in a dialog with a community. They just find this to be an enjoyable thing to do that makes them feel like they are a valuable member of society contributing to a general collaborative practice of knowledge-creation.

I hope it doesn't feel like I'm explaining the obvious; but it occurs to me that to ask the question the way you did, this is probably _not_ a motivation you have, not something you find personally rewarding. Which is fine, we all are driven by different things.

But I don't think it's quite the same thing as "internet brownie points" -- while if you are especially good at it, you will gain respect and admiration, which you will probably appreciate it -- you aren't thinking "if I share my insight gained working at Google, then maybe more people will think I'm cool," you're just thinking that it's a natural urge to share your insight and get feedback on it, because that itself is something enjoyable and gives you a sense of purpose.

Which is to say, i don't think it's exactly a motivation for "personal benefit" either, except in the sense that doing things you enjoy and find rewarding are a "personal benefit", that having a sense of purpose is a "personal benefit", sure.

I'm aware that not everyone works this way. I'm aware that some people on HN seem to be motivated primarily by maximizing income, for instance. That, or some other orientation, may lead to thinking that one should never share anything at all publicly about one's job, because it can only hurt and never help whatever one's goals are (maximizing income or what have you).

(Although... here you are commenting on HN; why? For internet brownie points?)

But that is not generally how academic/researcher types are oriented.

I think it's a sad thing if it becomes commonplace to think that there's something _wrong_ with people who find purpose and meaning in sharing their insights in dialog with a community.

Re: Why did Google Brain exist?

#106

Post by ternaus »

dr_dshiv wrote: post #7

Lots of great insight. Here’s one: “Given the long timelines of a PhD program, the vast majority of early ML researchers were self-taught crossovers from other fields. This created the conditions for excellent interdisciplinary work to happen. This transitional anomaly is unfortunately mistaken by most people to be an inherent property of machine learning to upturn existing fields. It is not. Today, the vast majority…

Loved this argument as well.

With respect to more mature research fields the entry point to ML is much lower.

Hence I always recommended people to have major in Physics, Chemistry, Biology etc but look for projects in these fields that could benefit in ML (I have a number of them about Physics).

So that argument was not novel.

But the fact that pure ML PhDs will have significantly lower multidisciplinary knowledge is a good one. It could be compensated by the fact that ML is growing fast and all kinds of people join the ride, but still.

Re: Why did Google Brain exist?

#107

Post by dekhn »

kevviiinn wrote: post #71
selimthegrim wrote: post #66

Earlier quoted context omitted.

Well I think the issue is more of if you’re Genentech and you need ML people and can’t afford to pay them you’re probably better off retraining chemistry PhDs.

I think you missed my point. Genentech, AFAIK, was not doing research on machine learning as in the principles of how machine learning works and how to make it better. They do biotech research which uses applied machine learning. You don't need a PhD in ML to apply things that are already known

Genentech has several ML groups that do mostly applied work, but some do fairly deep research into the model design itself, rather than just applying off-the-shelf systems. For example, they acquired Prescient Design which builds fairly sophisticated protein models (https://nips.cc/Conferences/2022/ScheduleMultitrack?event=59...) and one of the coauthors is the head of Genentech Research (which itself is very similar to Google Research/Brain/DeepMind), and came from the Broad Institute having done ML for decades ('before it was cool').

They have a few other groups as well (https://nips.cc/Conferences/2022/ScheduleMultitrack?event=60... and https://neurips.cc/Conferences/2022/ScheduleMultitrack?event... and https://neurips.cc/Conferences/2022/ScheduleMultitrack?event...).

I can't say I know anybody there who is doing what I would describe as truly pure research into ML; it's not in the DNA of the company (so to speak) to do that.

Re: Why did Google Brain exist?

#108

Post by brilee »

protastus wrote: post #88
khazhoux wrote: post #59

> I sat on it because I wasn’t sure of the optics of posting such an essay while employed by Google Brain. But then Google made my decision easier by laying me off in January. My severance check cleared... I'm really baffled by how people think it's OK to write public accounts of their previous ( and sometime current! ) employers' inner workings. This guy got paid a shitload of money to do work and to keep all intern…

I've never worked at Google Brain, but I've been a research manager in tech for a decade, and nothing here seems surprising to me. It discusses the archetype of the well funded industry lab that is too academic and ultimately winds down. The post makes sensible but generic statements based on the view of an IC. It tries to work back from the conclusion (Google struggles to move academic research into product) and pro…

You're right that I don't know enough as an IC to comment on the thinking, actions, and promises at the exec level that kept Brain funded. But even if I did, to the GP's point, I would not talk about that part at all.

I'm curious about your perspective as a research manager in tech - would you be willing to chat privately?

Re: Why did Google Brain exist?

#109

Post by jeffbee »

snapcaster wrote: post #102
jeffbee wrote: post #99

Earlier quoted context omitted.

Bard is in full-scale production to all U.S. users for free. GPT-4 costs $20/month. Rather a big difference in the economics of the situation. Also it's pretty clear that even the $20 is highly subsidized. Microsoft is willing to incinerate almost any amount of money to harm Google.

Free but unuseably bad seriously have you tried it? compared it to even GPT-3? it really really sucks

Yes I think it has less utility than the free version of ChatGPT, but it also has some nice points, is faster, and has fewer outages.

For my use case none of them is worth using. All three of the ones we've mentioned in this thread will just make up language features that would be useful but don't exist, and all three of them will hallucinate imaginary sections of the C++ standard to explain them. Bard loves `std::uint128_t`. GPT-4 will make up GIS coordinate reference systems that don't exist. For me they are all more trouble than they are worth, on the daily.

Re: Why did Google Brain exist?

#110

Post by pessimizer »

khazhoux wrote: post #100
ShamelessC wrote: post #94

Earlier quoted context omitted.

That's a very optimistic take on how new generations perceive the ethics surrounding confidentiality. Are you really _baffled_ by this? I understand that it's a common position, but it's a position that is so clearly tainted by the conflicts of interest between employer and employee. And a keen awareness of those conflicting interests _only_ serves to better an employee's ability to serve themselves best in a capital…

I totally get why people act this way. Because it's Very Important That Everyone Knows What I Think. But there is no inequity or conflict of interest here. None of that about being trapped in a capitalist economy is to the point here. He has probably a couple million dollars in his bank account that wasn't there before, and the deal was to not talk publicly about internals (which includes promotions process, internal…

You don't think there's any inequity between somebody with a couple million in the bank and google, or any conflict of interest between my desire to talk about my work and my employer's desire that I do not? Your position is valid enough without being willfully obtuse.
Post reply on HN