We built an end-to-end protein AI pipeline covering structure prediction, sequence design, and codon optimization. After comparing multiple transformer architectures for codon-level language modeling, CodonRoBERTa-large-v2 emerged as the clear winner with a perplexity of 4.10 and a Spearman CAI correlation of 0.40, significantly outperforming ModernBERT. We then scaled to 25 species, trained 4 production models in 55 GPU-hours, and built a species-conditioned system that no other open-source project offers. Complete results, architectural decisions, and runnable code below.
Training mRNA Language Models Across 25 Species for $165
1–10 of 46 posts
Re: Training mRNA Language Models Across 25 Species for $165
#2Re: Training mRNA Language Models Across 25 Species for $165
#3gray goo of the future
Re: Training mRNA Language Models Across 25 Species for $165
#4[deleted]
Re: Training mRNA Language Models Across 25 Species for $165
#5full article: https://huggingface.co/blog/OpenMed/training-mrna-models-25-...
What makes this dataset or problem worth solving compared to other health datasets? Would the results on this task be broadly useful to health?
Re: Training mRNA Language Models Across 25 Species for $165
#6> In Progress: CodonJEPA
JEPA is going to break the whole industry :D
Re: Training mRNA Language Models Across 25 Species for $165
#7What makes these Domain specific models work when we don’t have good domain models for health care, chemistry, economics and so on
Re: Training mRNA Language Models Across 25 Species for $165
#8> In Progress: CodonJEPA JEPA is going to break the whole industry :D
Can you explain this? I haven't heard of JEPA, and from a quick search it seems to be vision/robotics based?
Re: Training mRNA Language Models Across 25 Species for $165
#9> In Progress: CodonJEPA JEPA is going to break the whole industry :D
Can you explain this? I haven't heard of JEPA, and from a quick search it seems to be vision/robotics based?
Re: Training mRNA Language Models Across 25 Species for $165
#10Can someone explain what one might use this model for? As a developer with a casual interest in biology it would be fun to play with but honestly not sure what I would do