Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AlphaFold: a solution to a 50-year-old grand challenge in biology

deepmind.com

Post reply on HN
Print view View original

331–340 of 683 posts

Re: AlphaFold: a solution to a 50-year-old grand challenge in biology

#331

Post by robocat »

beowulfey wrote: post #254
treis wrote: post #143

Earlier quoted context omitted.

That is an impressive improvement, but I think you've missed the most important point: >a score of around 90 GDT is informally considered to be competitive with results obtained from experimental methods So DeepMind is to the point where it's a question of whether their generated model or the experimentally determined structure is closest to the actual physical structure.

I don’t think you can say DeepMind could ever be more accurate to the true physical structure since it was built on the same experimental structures that it is being compared to. The limit of accuracy is the experimental data. However, I think we can say that a DeepMind prediction could at least be as good as a new experimental structure.

But DeepMind could be used to find errors in the training set.

Let’s say you have 100000 proteins in the training set. Now remove #1 and train on 99999, and then check that it still predicts the same protein result for #1 as the experimental result.

Or remove from training whole sets of proteins by particular teams to find systematic errors made by teams?

Re: AlphaFold: a solution to a 50-year-old grand challenge in biology

#332

Post by dash2 »

zaroth wrote: post #319
zaroth wrote: post #224

Earlier quoted context omitted.

> The price of training the final architecture is meaningless. The research is the giant shoulders you stand on, the compute cost is the price of the tool you need to do the present-day work. Both are relevant but the shoulder’s of giants are generally more accessible, particularly if we’re talking about published research and not proprietary tech. A competing team is not starting from the same place the DeepMind tea…

To expand on this, after fully reading AlQuraishi's "What Just Happened" post from a couple years ago, was this point that he made; > I don’t think we would do ourselves a service by not recognizing that what just happened presents a serious indictment of academic science. There are dozens of academic groups, with researchers likely numbering in the (low) hundreds, working on protein structure prediction. We have bee…

To be fair, I'm not sure that they are "simplistic" in the sense that, e.g., writing a neural network to recognise cat pictures is now simplistic. I don't know how many people have Deepmind levels of expertise in ML, or could implement what they have done, but I doubt it is many, and they are thinly spread amongst many interesting problems.

Re: AlphaFold: a solution to a 50-year-old grand challenge in biology

#333

Post by garmaine »

the8472 wrote: post #7
partingshots wrote: post #3

I continue to be impressed by how quickly DeepMind has managed to progress in such a short time. CASP13 was a shocker to all of us I think, but many were skeptical as to the longevity of the performance DeepMind was able to achieve. I believe with CASP14 rankings now released, it's safe to say that they've proven themselves. Congratulations to the team! This work will have far reaching impacts, and I hope that you co…

> but many were skeptical as to the longevity of the performance DeepMind was able to achieve For a non-biologist, on what is this skepticism based? Just purely based on following ML news it looks like the trend for ML solutions has been that they've overtaken expert-systems once they've gained a solid foodhold in a field. Maybe this is some perception bias. Are there any cases where ML performed decently but then hi…

> Are there any cases where ML performed decently but then hit a ceiling while expert systems kept improving?

Yes, this describes entire history of AI including several boom-bust cycles. In particular the 80's come to mind. Yes the practitioners think that there's no technical barriers stopping them from eating the world, but that's exactly what people thought about other so-called revolutionary advances.

Although to be pedantic, "expert systems" is the technology behind AI boom of the 80's. At the time people were saying expert systems can't be as good as existing algorithms (including what we would now call "machine learning" techniques), then suddenly the expert systems were better and there was rampant speculation real AI was around the corner. Then they plateaued.

We appear to be at the tail end of the maximum hype part of the boom-bust cycle. Thinking that the rapid gains being made by the current deep learning approaches will soon hit a wall is a reasonable outside-view prediction to make: nearly every time we've had a similarly transformative technology in the AI space and elsewhere, hitting the wall is exactly what happened. The onus would be on practitioners to show that this time really is different.

Re: AlphaFold: a solution to a 50-year-old grand challenge in biology

#334

Post by garmaine »

whimsicalism wrote: post #299
klmr wrote: post #269

Earlier quoted context omitted.

> It'll be genetics next. Which part of genetics are you thinking of? Much of genetics isn’t amenable to this kind of ML, because it isn’t some kind of optimisation problem. And many other parts don’t require ML because they can be modelled very closely using exact methods. ML does get used here, and sometimes to great effect (e.g. DeepVariant, which often outperforms other methods, but not by much — not because Deep…

What do you mean? Genetics is amenable because the genome is a sequence that can be language modeled/auto-regressed for depth of understanding by the network. There are plenty of inferences that you would want to do on genetic sequences that we can't model exactly and there is some past work on doing stuff like this, although biology is usually a few years behind. https://www.nature.com/articles/s41592-018-0138-4 e:…

This is word salad.

Re: AlphaFold: a solution to a 50-year-old grand challenge in biology

#335

Post by dekhn »

mjn wrote: post #308
throwawayiionqz wrote: post #204

Earlier quoted context omitted.

This is the cost of training the final architecture with all the refinements enabled by years of research. These years of research involved trying many different architectures, many of which received as much or more compute time than the final system. The price of training the final architecture is meaningless. Researching and training AlphaGo was expensive but it enabled the ideas and development of AlphaZero which…

Even if you try to account for the overall R&D cost, DeepMind isn't that large an organization by the standards of biomedical research. It's very big and well funded for a computer science research organization, yes, and most CS departments can't match its resources. But the NIH budget is $40 billion, and private pharmaceutical companies do another $80 billion in annual R&D. It's interesting that this kind of breakth…

DeepMind is taking advantage of NIH's funding. For example, Anfinsen who demonstrated that proteins fold spontaneously and reproducibly (https://en.wikipedia.org/wiki/Anfinsen%27s_dogma) ran a lab at NIH. Levinthal (who postulated an early and easily refutable model of protein folding) was funded by NIH for decades. Most of the competitors at CASP are supported by NIH and its investments have contributed to the modern results significantly.

That said I think the academic and pharma communities had engineered themselves into a corner and weren't going to see huge gains (even thogh they are exploring similar ideas) for a number of banal reasons.

Re: AlphaFold: a solution to a 50-year-old grand challenge in biology

#336

Post by timr »

treis wrote: post #143
mabbo wrote: post #30

Sometimes announcements like this are a bit over-the-top. But what really, to me, cements the 'big-deal' of this is the "Median Free-Modelling Accuracy" graph half way down the page. Scores of 30-45 for 15 years. Now scores of 87-92. This isn't a minor improvement, it's a leap forward.

That is an impressive improvement, but I think you've missed the most important point: >a score of around 90 GDT is informally considered to be competitive with results obtained from experimental methods So DeepMind is to the point where it's a question of whether their generated model or the experimentally determined structure is closest to the actual physical structure.

"So DeepMind is to the point where it's a question of whether their generated model or the experimentally determined structure is closest to the actual physical structure."

While this is an accomplishment, nobody is going to be confusing these models for structures produced experimentally. The CASP metric is for backbone atoms. To have a useful model of protein structure, you really need to have the positions of the protein side-chain atoms modeled correctly. Experimental methods will do that, but this method, as I understand it, does not.

Re: AlphaFold: a solution to a 50-year-old grand challenge in biology

#337

Post by klmr »

whimsicalism wrote: post #299
klmr wrote: post #269

Earlier quoted context omitted.

> It'll be genetics next. Which part of genetics are you thinking of? Much of genetics isn’t amenable to this kind of ML, because it isn’t some kind of optimisation problem. And many other parts don’t require ML because they can be modelled very closely using exact methods. ML does get used here, and sometimes to great effect (e.g. DeepVariant, which often outperforms other methods, but not by much — not because Deep…

What do you mean? Genetics is amenable because the genome is a sequence that can be language modeled/auto-regressed for depth of understanding by the network. There are plenty of inferences that you would want to do on genetic sequences that we can't model exactly and there is some past work on doing stuff like this, although biology is usually a few years behind. https://www.nature.com/articles/s41592-018-0138-4 e:…

I meant, which specifics are you thinking of?

> Genetics is amenable because it is a sequence

Not sure what you mean by that. Genetics is a field of research. The genome is a sequence. And yes, that sequence can be modelled for various purposes but without a specific purpose there’s no point in doing so (and furthermore doing so without specific purpose is trivial — e.g. via markov chains or even simpler stochastic processes — but not informative).

> There are plenty of inferences that you would want to do on genetic sequences

I’m aware (I’m in the field). But, again, I was looking for specific examples where you’d expect ML to provide breakthroughs. Because so far, the reason why ML hasn’t provided many breakthroughs in less about the lack of research and more because it’s not as suitable here as for other hard questions. For instance, polygenic risk scores (arguably the current “hotness” in the general field of genetics) can already be calculated fairly precisely using GWAS, it just requires a ton of clinical data. GWAS arguably already uses ML but, more to the point, throwing more ML at the problem won’t lead to breakthroughs because the problem isn’t compute bound or vague, it’s purely limited by data availability.

I could imagine that ML can help improve spatial resolution of single-cell expression data (once again ML is already used here) but, again, I don’t think we’ll see improvements worthy of called breakthroughs, since we’re already fairly good.

Re: AlphaFold: a solution to a 50-year-old grand challenge in biology

#338

Post by kevincox »

lacksconfidence wrote: post #315
curiousllama wrote: post #311

Earlier quoted context omitted.

Actually, no! Or at least, the budget they used ( In other words, it's less like GPT3 and more like ImageNet.

Is the cost to train really the relevant metric for developing this? It seems like the salary's involved are probably at least 10x whatever they spent on hardware.

Additionally the training test of all of the models during development.

Re: AlphaFold: a solution to a 50-year-old grand challenge in biology

#339

Post by alquemist »

whimsicalism wrote: post #5
partingshots wrote: post #3

I continue to be impressed by how quickly DeepMind has managed to progress in such a short time. CASP13 was a shocker to all of us I think, but many were skeptical as to the longevity of the performance DeepMind was able to achieve. I believe with CASP14 rankings now released, it's safe to say that they've proven themselves. Congratulations to the team! This work will have far reaching impacts, and I hope that you co…

Progress like this was, in my view, inevitable after the invention of unsupervised transformers. It'll be genetics next. e: although AlphaFold appears to be convolutionally based! I suspect that'll change soon.

FWIW, transformers is to sequences what convnets is to grids, modulo important considerations like kernel size and normalization. Think of transformers as really wide (N) and really short (1) convolutions. Both are instances of graphnets with a suitable neighbor function. Once normalization was cracked by transformers, all sort of interesting graphnets became possible, though it's possible that stacked k-dimensional convolutions are sufficient in practice.

Re: AlphaFold: a solution to a 50-year-old grand challenge in biology

#340

Post by sseagull »

Seanambers wrote: post #250
tonfa wrote: post #226

Earlier quoted context omitted.

> So it wasn't out of reach for academia, pharmaceuticals, or others with a bit of resources. How much does hiring a deepmind-like team cost though? (massively more than the TPU resources?) Still within reach of pharmaceutical industry I guess, but maybe not so easy for academia.

From what I can gather, Google bought Deepmind for 500 million USD in 2014, they have outstanding debt to its parent company as of 2019 of 1.3 billion USD. And they had income around 100 million in 2019 but it's all against Google, so looks like a 2 billion +/- 0.5 operation so far, and who knows if they pay for compute. Other articles place the runrate at 500 million per year in 2019. Which means 500 million * 6 yea…

It would still be a significant amount of money for a lot of companies.

Remember, we are looking in hindsight that it seemingly paid off. A few years ago, this was just an educated bet; only the richest companies with money to burn (from selling ads) would be willing to take on that kind of a risk.

Post reply on HN