How to train your own large language models

Post by moltar » Thu, Apr 20, 2023, 4:31 PM UTC

dskhudia wrote: post #60

moltar wrote: post #46

Earlier quoted context omitted.

Do you have any examples on how to train a model that can write code but in a specific domain? Eg I only want to train it on a specific set of code. Eg let’s say functional React components in TypeScript.

We recently released 1B parameter model trained on a mix of data.[1] If you got your domain-specific data, our platform can cover the rest. [1]: https://twitter.com/jefrankle/status/1649060478910357504?s=4...

Post by MrMovl » Tue, May 02, 2023, 11:21 AM UTC

kkielhofner wrote: post #14

zacksiri wrote: post #11

This is interesting. I think the future of AI will not be re-creating something like ChatGPT, but using variations of these methodologies to train AI models for specific tasks. There are some advantages to not having to make an LLM that impresses every human being on the planet. Imagine training the AI to be good in only one specific thing. I think it will become much more precise and deterministic. This is just my h…

There is plenty of less attention-grabbing work being done on "domain specific LLMs" like BioMedLM[0], Med-PaLM[1], BloombergGPT[2], etc. That reminds me - I saw a somewhat-clever acronym variant for LLM that communicated this the other day but it escapes me ATM... [0] - https://www.mosaicml.com/blog/introducing-pubmed-gpt [1] - https://cloud.google.com/blog/topics/healthcare-life-science... [2] - https://dev.to/ream…