Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Advancements in machine learning for machine learning

blog.research.google

Post reply on HN
Print view View original

121–130 of 162 posts

Re: Advancements in machine learning for machine learning

#121

Post by snewman »

dindobre wrote: post #76
kccqzy wrote: post #67

Earlier quoted context omitted.

If I recall right, before ChatGPT launched Google already had LaMDA which an employee believed to be sentient and was subsequently fired. The foundation model was definitely done, but to launch Bard, Google needed a kick in the ass in additional RLHF, safety and groundedness work. Ultimately though, it's futile to argue which model got done first, as long as the models were behind closed doors. But ChatGPT launched b…

The LaMDA is sentient guy gave me the impression of being a bit nuts. I'm sure google would show their weight and out-compete openai if they could. We all know all this "AI safety" is for show, right?

I'm personally devoting my career to AI safety, on a volunteer basis, because I think it's is legitimately of high importance. (See my blog, e.g. https://amistrongeryet.substack.com/p/implications-of-agi, if you want to understand where I'm coming from.)

What makes you think it is for show?

Re: Advancements in machine learning for machine learning

#122

Post by Tao3300 »

falcor84 wrote: post #54
DeathArrow wrote: post #33

Earlier quoted context omitted.

For me it's just another gold rush after dotcom, mobile, cloud, VR.

I'm not sure what the purpose of the word "just" there is. There indeed seems to be quite a lot of gold to be had by whoever gets a foothold.

"just" is there to distinguish a gold rush from a singularity

Re: Advancements in machine learning for machine learning

#123

Post by staunton »

Palmik wrote: post #120
staunton wrote: post #82

Earlier quoted context omitted.

Indeed, making poor people in 3rd world countries rate the worst sludge of the internet for 8+h a day might backfire on your marketing... OpenAI could risk it, Google maybe doesn't want to...

This is a naive take. How do you think Google collects or collected data for their safe-search classifiers? Now that's a sludge. Or how do you think Google evaluates search-ranking changes (or gather data for training various ad-ranking & search-ranking models).

I don't know. How do they?

Re: Advancements in machine learning for machine learning

#124

Post by blackbear_ »

sbierwagen wrote: post #59
AnthonyMouse wrote: post #49

Earlier quoted context omitted.

> In order for an AI to evaluate the effect of a small molecule on the brain, it would have to... simulate the operation of a human brain in a simulated environment. Similarly, to avoid Thalidomide-style disasters, it would have to simulate the conception, development and growth to adulthood of a human. This is how the human doctors who have cured things in the past have done it, is it? The way this is going to work,…

Then AI will have no effect on the drug industry at all. The rate limiting step isn't "thinking up molecules." The University of Bern enumerated all possible molecules composed only of hydrogen, carbon, nitrogen, oxygen, sulfur and chlorine, up to 17 atoms. That produced 166 billion molecules. https://pubs.acs.org/doi/10.1021/ci300415d There are commercial drugs considerably larger than that. We've got molecular stru…

> The problem is the clinical trial. [...] That's the part that takes years and tens of millions of dollars.

Clinical trials only start after about five years of research and development. While they do represent a large part of the budget (even in the hundreds of millions of dollars), there are countless of other necessary steps before, during, and after trials to ensure that drugs are both safe and effective. The problem is that we still don't understand how most of these molecules behave in the body, and how we can produce them reliably and efficiently enough, which brings me to the next point:

> [...] but did it do anything for the problem of putting chemicals in people?

Yes, there are plenty of problems that AI and computational chemistry already help with in the pharmaceutical industry, including predicting solubility, stability, crystallization, granulation, toxicity, pharmacokinetics, developing the formulation, optimizing and scaling up both the synthesis and production process, developing appropriate techniques for quality control, and so on.

In all these cases and more, AI can help reduce the amount of experiments that need to be done in the lab, which require highly specialized equipment, personnel, and a lot of time. Oh and design of experiments is also a very important topic, again aiming at reducing the amount of lab time needed.

Admittedly, most of these things aim at ensuring that we do not put the wrong chemical in people, but they do represent most of the R&D effort spent in pharma, and reducing everything to clinical trials is not correct. There is a very wide gap between "AI will design drugs entirely on its own" and "AI is useless".

Re: Advancements in machine learning for machine learning

#125

Post by aconz2 »

summary: improve prediction of run-time performance of a computation graph using GNN, they use an embedding dictionary for each node's opcode along with some other node features (eg shape, bits, window size, see [1]), they released a big dataset of these graphs in [2] with varying XLA compilation configurations and their resulting perf on TPUs, they did some stuff to improve prediction on bigger graphs than before in [3] by partitioning the graph (METIS graph partition, new to me) and other training things

This is only about predicting performance of a given graph and not about improving/suggesting/editing a new equivalent graph. As in FunSearch, models which have decent predictive power could be used with evolutionary search.

[1] https://github.com/google-research-datasets/tpu_graphs#featu...

[2] TpuGraphs: A Performance Prediction Dataset on Large Tensor Computational Graphs https://arxiv.org/abs/2308.13490

[3] Learning Large Graph Property Prediction via Graph Segment Training https://arxiv.org/abs/2305.12322

Re: Advancements in machine learning for machine learning

#126

Post by dbecker »

owlbite wrote: post #107

These ML-compilers are being overhyped. It's all the same trade-off as a traditional compiler: you get a lot more throughput than hiring a specialist performance programmer, but the latter will typically outperform, possibly by orders of magnitude. These things are inferior at many levels: - Algorithmic: These things aren't feeding back to their human masters tips and tricks on how to modify the network to go faster…

> These ML-compilers are being overhyped. It's all the same trade-off as a traditional compiler

Funny you should say that. Because traditional compilers have been incredibly useful.

Re: Advancements in machine learning for machine learning

#127

Post by dwaltrip »

sbierwagen wrote: post #20
GreedClarifies wrote: post #9

How’s Gemini looking?

It is interesting how persistently dominant GPT-4 is: https://twitter.com/lmsysorg/status/1735729398672716114 Off the top of my head, I can think for at least five foundation models (Llama, Claude, Gemini, Falcon, Mistral) that are all trading blows, but GPT is still a head above them and has been for a year now. Transformer LLMs are simple enough that, demonstrably, anyone with a million bucks of GPU time can make o…

Note, Gemini Ultra, which they claim is competitive with or possibly even better than GPT-4, isn’t out yet. They have released a weaker model, Gemini Pro.

It will be interesting to see how capable Gemini Ultra actually is. For now we wait.

Re: Advancements in machine learning for machine learning

#128

Post by asah »

stabbles wrote: post #112
owlbite wrote: post #107

These ML-compilers are being overhyped. It's all the same trade-off as a traditional compiler: you get a lot more throughput than hiring a specialist performance programmer, but the latter will typically outperform, possibly by orders of magnitude. These things are inferior at many levels: - Algorithmic: These things aren't feeding back to their human masters tips and tricks on how to modify the network to go faster…

Take a look at the chess engine Stockfish: they tossed out years and years of human written heuristics in board evaluation, to a small neural net that does the same but better. Now consider all the heuristics for inlining, loop unrolling, vectorization etc in compilers, certainly a neural net can be beneficial and possibly easier to maintain than tons of human written heuristics.

big +1 - IMHO the future of optimizers (and probably compilers...) are almost certainly ML-based.

Re: Advancements in machine learning for machine learning

#129

Post by ldjkfkdsjnv »

stabbles wrote: post #112
owlbite wrote: post #107

These ML-compilers are being overhyped. It's all the same trade-off as a traditional compiler: you get a lot more throughput than hiring a specialist performance programmer, but the latter will typically outperform, possibly by orders of magnitude. These things are inferior at many levels: - Algorithmic: These things aren't feeding back to their human masters tips and tricks on how to modify the network to go faster…

Take a look at the chess engine Stockfish: they tossed out years and years of human written heuristics in board evaluation, to a small neural net that does the same but better. Now consider all the heuristics for inlining, loop unrolling, vectorization etc in compilers, certainly a neural net can be beneficial and possibly easier to maintain than tons of human written heuristics.

Humans designing algorithms by hand will go the way of the dodo bird

Re: Advancements in machine learning for machine learning

#130

Post by JyB »

owlbite wrote: post #107

These ML-compilers are being overhyped. It's all the same trade-off as a traditional compiler: you get a lot more throughput than hiring a specialist performance programmer, but the latter will typically outperform, possibly by orders of magnitude. These things are inferior at many levels: - Algorithmic: These things aren't feeding back to their human masters tips and tricks on how to modify the network to go faster…

Comment seem extremely dismissive and close minded.
Post reply on HN