Biological Function Emerges from Unsupervised Learning on 250M Protein Sequences
1–10 of 51 posts
Re: Biological Function Emerges from Unsupervised Learning on 250M Protein Sequences
#2Language, music, and now amino acid sequences. Attention is all you need.
Re: Biological Function Emerges from Unsupervised Learning on 250M Protein Sequences
#3Language, music, and now amino acid sequences. Attention is all you need.
I would say you also need a fair bit of data too...
Re: Biological Function Emerges from Unsupervised Learning on 250M Protein Sequences
#4Language, music, and now amino acid sequences. Attention is all you need.
I would say you also need a fair bit of data too...
Well, yes. But I think lucidrains was referring to:
Re: Biological Function Emerges from Unsupervised Learning on 250M Protein Sequences
#5Re: Biological Function Emerges from Unsupervised Learning on 250M Protein Sequences
#6This blog post seems to anticipate this happening: https://moalquraishi.wordpress.com/2019/04/01/the-future-of-...
Re: Biological Function Emerges from Unsupervised Learning on 250M Protein Sequences
#7Like some of the other ML/AI posts that made it to the top page today, this research too does not give any clear way to reproduce the results. I looked through the pre-print page as well as the full manuscript itself.
Without reproducibility and transparency in the code and data, the impact of this research is ultimately limited. No one else can recreate, iterate, and refine the results, nor can anyone rigorously evaluate the methodology used (besides giving a guess after reading a manuscript).
The year is 2019, many are finally realizing it's time to back up your results with code, data, and some kind of specification of the computing environment you're using. Science is about sharing your work for others in the research community to build upon. Leave the manuscript for the pretty formality.
Re: Biological Function Emerges from Unsupervised Learning on 250M Protein Sequences
#8I find these emergent behaviours fascinating:
https://youtu.be/gaFKqOBTj9w
Re: Biological Function Emerges from Unsupervised Learning on 250M Protein Sequences
#9This blog post seems to anticipate this happening: https://moalquraishi.wordpress.com/2019/04/01/the-future-of-...
> It does a surprisingly good job of predicting protein function across a diverse set of tasks, including ones structural in nature, like the induction of a single neuron that is able, with some degree of accuracy (ρ = 0.33) to distinguish between α helices and β strands (I suspect the network as a whole is far more performant at this task than the single neuron we’ve identified, but we didn’t push this aspect of the analysis as the problem is well tackled using specialized approaches.)
I hate to be that guy, but distinguishing between alpha helices and beta strands is not really that hard.
It's a good start though. I would propose the following test: Let's see if we can use the activations from the neurons to predict the luminosity of a 'base' GFP molecule (under a fixed set of experimental conditions). Train the set on 10,000 mutations (this could maybe be done in very high throughput by tethering the XNA to a bead, synthesizing, and then measuring the beads one by one), and see if can extrapolate the effects of 10k more, or heck, just by doing it brute-forcedly, we've got high throughput robots, right?
Re: Biological Function Emerges from Unsupervised Learning on 250M Protein Sequences
#10I find these emergent behaviours fascinating: https://youtu.be/gaFKqOBTj9w
It would be neat to run this with a few more rules, on a larger world, and for a longer time to see what emerges!