Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Advancements in machine learning for machine learning

blog.research.google

Post reply on HN
Print view View original

101–110 of 162 posts

Re: Advancements in machine learning for machine learning

#101

Post by dmarchand90 »

sbierwagen wrote: post #20
GreedClarifies wrote: post #9

How’s Gemini looking?

It is interesting how persistently dominant GPT-4 is: https://twitter.com/lmsysorg/status/1735729398672716114 Off the top of my head, I can think for at least five foundation models (Llama, Claude, Gemini, Falcon, Mistral) that are all trading blows, but GPT is still a head above them and has been for a year now. Transformer LLMs are simple enough that, demonstrably, anyone with a million bucks of GPU time can make o…

I kinda wonder if maybe it's at least partially due to openai hitting a kind of hyperparameter lottery. When each experiment costs millions it might be that (aside from good/ unique data) they just have a good set of hyperparameters used in training and it's too expensive for a competitor to find equal or better settings

Re: Advancements in machine learning for machine learning

#102

Post by porompompero »

dmarchand90 wrote: post #101
sbierwagen wrote: post #20

Earlier quoted context omitted.

It is interesting how persistently dominant GPT-4 is: https://twitter.com/lmsysorg/status/1735729398672716114 Off the top of my head, I can think for at least five foundation models (Llama, Claude, Gemini, Falcon, Mistral) that are all trading blows, but GPT is still a head above them and has been for a year now. Transformer LLMs are simple enough that, demonstrably, anyone with a million bucks of GPU time can make o…

I kinda wonder if maybe it's at least partially due to openai hitting a kind of hyperparameter lottery. When each experiment costs millions it might be that (aside from good/ unique data) they just have a good set of hyperparameters used in training and it's too expensive for a competitor to find equal or better settings

Sorry for my ignorance: why does each experiment cost millions?

Re: Advancements in machine learning for machine learning

#103

Post by larodi »

d3m0t3p wrote: post #75
dalbasal wrote: post #45

Can anyone bring this down to earth for me? What's the actual state of these "ML compilers" currently, and what is rhe near term promise?

One of the easiest approache is torch.compile, it's the latest iteration of pytorch compiler (previous methods were : TorchScript and FX Tracing.) You simply write model = torch.compile(model) "Across these 163 open-source models torch.compile works 93% of time, and the model runs 43% faster in training on an NVIDIA A100 GPU. At Float32 precision, it runs 21% faster on average and at AMP Precision it runs 51% faster…

Thanks for this summary

Re: Advancements in machine learning for machine learning

#104

Post by DeathArrow »

xbmcuser wrote: post #63
DeathArrow wrote: post #33

Earlier quoted context omitted.

For me it's just another gold rush after dotcom, mobile, cloud, VR.

The first 3 have and did result as of today in trillions in dollars of economic activity. And have changed societies, politics, political participation, access to knowledge etc worldwide for good and bad. So I don't get why you are so dismissive of them.

I am not dismissive. I think, though, that for a hundred companies engaged in the race one or less might succeed.

Re: Advancements in machine learning for machine learning

#105

Post by Jensson »

porompompero wrote: post #102
dmarchand90 wrote: post #101

Earlier quoted context omitted.

I kinda wonder if maybe it's at least partially due to openai hitting a kind of hyperparameter lottery. When each experiment costs millions it might be that (aside from good/ unique data) they just have a good set of hyperparameters used in training and it's too expensive for a competitor to find equal or better settings

Sorry for my ignorance: why does each experiment cost millions?

Because training a model costs millions, so each time you experiment with trying to create a new kind of model it costs millions.

Re: Advancements in machine learning for machine learning

#106

Post by jazarwil »

sbierwagen wrote: post #20
GreedClarifies wrote: post #9

How’s Gemini looking?

It is interesting how persistently dominant GPT-4 is: https://twitter.com/lmsysorg/status/1735729398672716114 Off the top of my head, I can think for at least five foundation models (Llama, Claude, Gemini, Falcon, Mistral) that are all trading blows, but GPT is still a head above them and has been for a year now. Transformer LLMs are simple enough that, demonstrably, anyone with a million bucks of GPU time can make o…

You cannot compare GPT 4 to Gemini Pro. They are different classes of models.

Re: Advancements in machine learning for machine learning

#107

Post by owlbite »

These ML-compilers are being overhyped. It's all the same trade-off as a traditional compiler: you get a lot more throughput than hiring a specialist performance programmer, but the latter will typically outperform, possibly by orders of magnitude.

These things are inferior at many levels: - Algorithmic: These things aren't feeding back to their human masters tips and tricks on how to modify the network to go faster beyond some very basic signals. - Loss of intent: ML network designers are specifying architecture in python, and by the time it's gone through many layers of lowering, you can get some complete garbage. Highly efficient garbage, but still garbage. (recent example, we caught one of these compilers doing a slice update operation by first forming the range of all possible indices to the array, slicing that to get indices to update, and then doing a scatter; we replaced it with a single memcpy call). - Inefficient kernels. Every time we see the output of these compilers go head-to-head with an expert assembly programmer, the compiler loses, often by 30%+. This always seems like the sort of thing that should be easy to solve, but given no-one seems to have cracked it in the past 50 years, it's obviously not as simple as it sounds.

Re: Advancements in machine learning for machine learning

#108

Post by PartiallyTyped »

dalbasal wrote: post #45

Can anyone bring this down to earth for me? What's the actual state of these "ML compilers" currently, and what is rhe near term promise?

The near term promise is that you can use AMD, CUDA, TPUs, CPUs etc without explicit vendor support for the framework on which the model was developed.

Disclaimer: I will be very handwavey, reality is complex.

This is achieved by compiling the graph into some intermediate representation. And then implementing the right backend. For projects here, look at stableHLO, IREE, openXLA.

You can argue that Jax's jit compiler is a form of such compiler, mapping the traced operations down to XLA, which then does its own bit of magic to make it work on your backend.

It's transformations and abstractions all the way down.

Re: Advancements in machine learning for machine learning

#109

Post by IanCal »

Jensson wrote: post #87
IanCal wrote: post #56

Earlier quoted context omitted.

The important thing is "how do you change X so that it heads towards the goal". And "how to do it quickly and efficiently". Otherwise the description is the same as "select randomly, keep the best, iterate". The goal is also complex. You might be thinking of "find the most efficient program" but that's not what we're doing here iiuc. We're trying to get a program that makes other unseen programs more efficient. That'…

> Otherwise the description is the same as "select randomly, keep the best, iterate". That is what they did though. The LLM didn't know what problem it was "solving".

That's not really true unless you're ignoring the rest of my points. The process did not just uniformly randomly create programs.

They also don't just keep the best and search from that point but feed the resulting programs and their scores into an LLM.

Re: Advancements in machine learning for machine learning

#110

Post by hotstickyballs »

owlbite wrote: post #107

These ML-compilers are being overhyped. It's all the same trade-off as a traditional compiler: you get a lot more throughput than hiring a specialist performance programmer, but the latter will typically outperform, possibly by orders of magnitude. These things are inferior at many levels: - Algorithmic: These things aren't feeding back to their human masters tips and tricks on how to modify the network to go faster…

Hardware (and performance) can always be improved without involvement of users so this is actually pretty useful.
Post reply on HN