11–20 of 30 posts
Cool. Could we train a "potential oncoprotein" classifier on Orthrus embeddings? IMO self serve diagnosis and detection is a far larger market than synthesis.
I am totally onboard with the premise (as a TechBio-adjacent person), and some of the approaches you're taking (focused domain-specific models like Orthrus, rather than massive foundation models like Evo2). I'm curious about what your strategy is for data collection to fuel improved algorithmic design. Are you building out experimental capacity to generate datasets in house, or is that largely farmed out to partners?
For the data - Orthrus is trained on non experimentally collected data so our pre-training dataset is large by biological standards. It adds up to about 45 million unique sequences and assuming 1k tokens per sequence it's about 50b tokens.
We're thinking about this as large pre-training run on a bunch of annotation data from Refseq and Gencode in conjunction with more specialized Orthology datasets that are pooling data across 100s of species.
Then for specific applications we are fine tuning or doing linear probing for experimental prediction. For example we can predict half life using publicly available data collected by the awesome paper from: https://genomebiology.biomedcentral.com/articles/10.1186/s13...
Or translation efficency: https://pubmed.ncbi.nlm.nih.gov/39149337/
Eventually as we ramp up out wet lab data generation we're thinking about what does post-training look like? There is an RL analog here that we can use on these generalizable embeddings to demonstrate "high quality samples".
There are some early attempts at post-training in bio and I think it's a really exciting direction
Maybe another application could be the ranking of candidate variants for cancer immunotherapy? As far as I know, lncRNAs are sometimes assessed.
founders@blankbio.com
Fun to see talk of "a compiler for DNA"---I've been hoping for that for a long time. I have to admit, at a _glance_ this feels like a promising idea with few results and lots of marketing. I'll try to be clear about my confusion, feel free to explain if I'm off base. - There's not a lot of talk of your "ground truth" for evaluations. Are you using mRNABench? - Has you mRNABench paper been peer reviewed? You linked a…
> ground truth Hey yes, the ground truth for our evaluations is measured experimental data. Our models are benchmarked using mRNABench, which aggregates results from high-throughput wet lab experiments. Our goal, however, is to move beyond predicting existing experimental outcomes. We intend to design novel sequences and validate their function in our own lab. At that stage, the functional success of the RNA we desig…
Just curious, in other areas of ML, I think it's widely acknowledged that benchmarks have pretty limited real world value, just end up getting saturated, and (my view) are all pretty correlated, regardless of their ostensible speciality and don't really tell you that much.
Do you think mRNABench is different, or where do you see the limitations? Do you imagine this or any benchmark will be useful for anything beyond comparing how different models do on the benchmark?
Fun to see talk of "a compiler for DNA"---I've been hoping for that for a long time. I have to admit, at a _glance_ this feels like a promising idea with few results and lots of marketing. I'll try to be clear about my confusion, feel free to explain if I'm off base. - There's not a lot of talk of your "ground truth" for evaluations. Are you using mRNABench? - Has you mRNABench paper been peer reviewed? You linked a…
> ground truth Hey yes, the ground truth for our evaluations is measured experimental data. Our models are benchmarked using mRNABench, which aggregates results from high-throughput wet lab experiments. Our goal, however, is to move beyond predicting existing experimental outcomes. We intend to design novel sequences and validate their function in our own lab. At that stage, the functional success of the RNA we desig…
Cool. Could we train a "potential oncoprotein" classifier on Orthrus embeddings? IMO self serve diagnosis and detection is a far larger market than synthesis.
This is a really interesting direction. There is this big field of Cell Free (cfRNA) cancer detection. We talked to a few people in the field and think that embedding sequences for this direction could be really valuable. One challenge here is that it's hard to set up evaluation tasks since the public data is scarce
Earlier quoted context omitted.
> ground truth Hey yes, the ground truth for our evaluations is measured experimental data. Our models are benchmarked using mRNABench, which aggregates results from high-throughput wet lab experiments. Our goal, however, is to move beyond predicting existing experimental outcomes. We intend to design novel sequences and validate their function in our own lab. At that stage, the functional success of the RNA we desig…
Hi, I'm the lead author of the human 5' UTR paper. It was a nice surprise seeing it linked on HN and I'm happy to see that it's providing value for you all. Looking forward to watching your team's progress!
> As compilers freed programmers from low-level details, we're building the abstraction layer for RNA.
That’s all fun and games when it’s literally fun and games. When it’s mRNA injected into living beings it’s the stuff of nightmares.
Will technologists ever _ever_ stop and think for a second?
Earlier quoted context omitted.
> ground truth Hey yes, the ground truth for our evaluations is measured experimental data. Our models are benchmarked using mRNABench, which aggregates results from high-throughput wet lab experiments. Our goal, however, is to move beyond predicting existing experimental outcomes. We intend to design novel sequences and validate their function in our own lab. At that stage, the functional success of the RNA we desig…
> mRNABench Just curious, in other areas of ML, I think it's widely acknowledged that benchmarks have pretty limited real world value, just end up getting saturated, and (my view) are all pretty correlated, regardless of their ostensible speciality and don't really tell you that much. Do you think mRNABench is different, or where do you see the limitations? Do you imagine this or any benchmark will be useful for anyt…
We think the situation is similar here - one the challenges is aligning the benchmark with the function of the models. Genomic benchmarks for gLMs and RNA foundation models have been very resistant to staturation.
I think in NLP the problem is that they are victims of their own success where the models can be overfit to particular benchmarks really fast.
In genomics we're a bit behind. A good paper on this is DartEval where they provide levels of complexity https://arxiv.org/abs/2412.05430
in RNA the models work much better than DNA prediction but it's key to have benchmarks to measure progress.
Literally the stuff of nightmares. Why are we doing this? > As compilers freed programmers from low-level details, we're building the abstraction layer for RNA. That’s all fun and games when it’s literally fun and games. When it’s mRNA injected into living beings it’s the stuff of nightmares. Will technologists ever _ever_ stop and think for a second?
From where we sit - there are people with diseases and mRNA is an effective way to revert them to a healthy state.
I'd be interested to hear more where you're coming from