Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Advancements in machine learning for machine learning

blog.research.google

Post reply on HN
Print view View original

81–90 of 162 posts

Re: Advancements in machine learning for machine learning

#81

Post by ben_w »

Fbnkigffb66tfbj wrote: post #60
sublinear wrote: post #41

Earlier quoted context omitted.

Have you ever worked in tech and had to deal with the typical illiteracy and incompetence of management and execs? If LLMs got this good, the brick wall these orgs will hit is what will really ruffle feathers. Leadership will have to be replaced by their technical workers in order for the company to continue existing. There's simply not enough information in the very high level plain english requirements they're used…

> you very likely cannot feed that half-assed junk to any LLM no matter how advanced and expect useful results Why don't you think that a sufficiently advanced AI can do the same as what technical humans do today with vague directions from managers?

Indeed.

I can give a vague, poorly written, poorly spelled request to the free version of ChatGPT and it still gives me a correct response.

As correct as usual at least (85-95%), but that's a different problem.

Re: Advancements in machine learning for machine learning

#82

Post by staunton »

code51 wrote: post #77
sbierwagen wrote: post #20

Earlier quoted context omitted.

It is interesting how persistently dominant GPT-4 is: https://twitter.com/lmsysorg/status/1735729398672716114 Off the top of my head, I can think for at least five foundation models (Llama, Claude, Gemini, Falcon, Mistral) that are all trading blows, but GPT is still a head above them and has been for a year now. Transformer LLMs are simple enough that, demonstrably, anyone with a million bucks of GPU time can make o…

Their special sauce is most probably the quality of data and the amount of data cleaning effort they put in. I’m speculating here but I think Google always refrains from getting into the manual side of things. With LLMs, it became obvious so fast that data is what matters. Seeing Microsoft’s phi-2 play, I’m convinced more about this. DeepMind understood the properties, came up with Chinchilla but DeepMind couldn’t in…

Indeed, making poor people in 3rd world countries rate the worst sludge of the internet for 8+h a day might backfire on your marketing... OpenAI could risk it, Google maybe doesn't want to...

Re: Advancements in machine learning for machine learning

#84

Post by blowski »

staunton wrote: post #82
code51 wrote: post #77

Earlier quoted context omitted.

Their special sauce is most probably the quality of data and the amount of data cleaning effort they put in. I’m speculating here but I think Google always refrains from getting into the manual side of things. With LLMs, it became obvious so fast that data is what matters. Seeing Microsoft’s phi-2 play, I’m convinced more about this. DeepMind understood the properties, came up with Chinchilla but DeepMind couldn’t in…

Indeed, making poor people in 3rd world countries rate the worst sludge of the internet for 8+h a day might backfire on your marketing... OpenAI could risk it, Google maybe doesn't want to...

Given that many western companies hire poor people to do all sorts of horrible work I doubt it’s that. More likely it’s to avoid suggestions of bias across their product range.

Re: Advancements in machine learning for machine learning

#85

Post by staunton »

dindobre wrote: post #76
kccqzy wrote: post #67

Earlier quoted context omitted.

If I recall right, before ChatGPT launched Google already had LaMDA which an employee believed to be sentient and was subsequently fired. The foundation model was definitely done, but to launch Bard, Google needed a kick in the ass in additional RLHF, safety and groundedness work. Ultimately though, it's futile to argue which model got done first, as long as the models were behind closed doors. But ChatGPT launched b…

The LaMDA is sentient guy gave me the impression of being a bit nuts. I'm sure google would show their weight and out-compete openai if they could. We all know all this "AI safety" is for show, right?

> We all know all this "AI safety" is for show, right?

No. A lot of people think it really matters

A lot of other people pretend to care about it because it also enables stifling the competition and attempting regulatory capture. But it's not all of them.

Re: Advancements in machine learning for machine learning

#86

Post by rybosome »

fjkdlsjflkds wrote: post #34
smabie wrote: post #24

Earlier quoted context omitted.

And they generally get outcompeted sooner or latter. All disciplines evolve over time and those who fail or refuse to keep up will be left behind.

Just because you can have a robot/machine that can efficiently churn out 1000 frozen lasagnas a second doesn't necessarily mean that italian restaurants have been "outcompeted" or "left behind" by not using such a machine in their business. Sometimes quality and responsibility matter. Even if a machine is really good at producing bug-free code, often someone is going to have to read/understand the code that the machi…

Ok, but to continue this analogy, industrialization and the ability to create 1000 frozen lasagnas a second had an enormous impact on the world. Not only on the economics of production, but ultimately on human society.

Sure, handmade lasagna still exists, but the world looks nothing like it did 200 years ago.

Re: Advancements in machine learning for machine learning

#87

Post by Jensson »

IanCal wrote: post #56
tommychillfiger wrote: post #25

Earlier quoted context omitted.

At great risk of sounding completely ignorant, this approach is basically what I thought the point of machine learning was - cleverly using feedback loops to improve things automatically. The thing that sticks out to me as particularly cool about FunSearch is the use of programs as inputs/outputs and the fact that they managed to automate feedback. I'm pretty naive in terms of granular understanding here as I am bare…

The important thing is "how do you change X so that it heads towards the goal". And "how to do it quickly and efficiently". Otherwise the description is the same as "select randomly, keep the best, iterate". The goal is also complex. You might be thinking of "find the most efficient program" but that's not what we're doing here iiuc. We're trying to get a program that makes other unseen programs more efficient. That'…

> Otherwise the description is the same as "select randomly, keep the best, iterate".

That is what they did though. The LLM didn't know what problem it was "solving".

Re: Advancements in machine learning for machine learning

#88

Post by Jensson »

rvnx wrote: post #72
xbmcuser wrote: post #63

Earlier quoted context omitted.

The first 3 have and did result as of today in trillions in dollars of economic activity. And have changed societies, politics, political participation, access to knowledge etc worldwide for good and bad. So I don't get why you are so dismissive of them.

AI is definitively here to stay forever. It's not a hype, it's 100% here for the long-term. The hype may be specific to some companies for now, but AI is deeply going to change many industries, especially due to open-source, specialized chips to allow running in local, and new hardware (I strongly hope a clone of H100 A80G comes quickly). The next step is to add limbs to the LLMs. Then we get Tesla bot who is going t…

> It's not a hype, it's 100% here for the long-term.

You mean it isn't overhyped, hype is just what expectations people have it is underhyped or overhyped that says how those expectations related to reality.

Re: Advancements in machine learning for machine learning

#89

Post by ChatGTP »

ben_w wrote: post #71
nodogoto wrote: post #52

Earlier quoted context omitted.

>In order for an AI to evaluate the effect of a small molecule on the brain, it would have to... simulate the operation of a human brain in a simulated environment. Humans aren't capable of doing this, but still make useful drug discoveries. AI can be empowered to conduct research in the real world, it doesn't need to simulate everything.

We start by doing them on mice (well, in vitro first, mice as the first in vivo), who have no say in the matter; and as mice are only rough analogues of humans, the human trials are still cautious once the animal trials are over.

We also develop a lot of drugs with a which have side-effects, which will is probably better than no drugs in most cases, the side effects are because it's a lot of educated guesswork.

Re: Advancements in machine learning for machine learning

#90

Post by vbezhenar »

kypro wrote: post #13
SirMaster wrote: post #11

Earlier quoted context omitted.

I don’t see why not. I like programming how I do now. I don’t plan to stop. People do lots of things manually that machines have been able to do for a long time.

You can do what you want, you just won't be paid to do it anymore.

I still get paid for project involving Oracle 9i running on Itanium HPUX and Delphi application running on Windows XP. This project is not going anywhere in the next 5 years. And there are numerous other projects which will not go anywhere either. I just don't believe that programming landscape will change much. May be in California startup world. My world moves slower.

I don't think anything significantly changed in my approach to the code since 2013. We will see how 2033 goes, but I don't expect nothing big either. ChatGPT is just Google replacement, Copilot is just smart autocomplete. I can use Google instead of ChatGPT and I could use Google 10 years ago. I can use vim macros instead of Copilot. This AI stuff helps me to save few hours a months, I guess, so it worth few bucks of subscription, but nothing groundbreaking so far.

Post reply on HN