Developing an LLM: Building, Training, Finetuning (A 1h Video Explainer)
1–10 of 14 posts
Re: Developing an LLM: Building, Training, Finetuning (A 1h Video Explainer)
#2Seems very good, thank you.
The channel: https://www.youtube.com/@SebastianRaschka/videos
contains hundreds of video lessons, originally seemingly originating from Sebastian Raschka teaching at Wisconsin-Madison Uni (before he went full-time entrepreneur).
Re: Developing an LLM: Building, Training, Finetuning (A 1h Video Explainer)
#3Seems very good, thank you. The channel: https://www.youtube.com/@SebastianRaschka/videos contains hundreds of video lessons, originally seemingly originating from Sebastian Raschka teaching at Wisconsin-Madison Uni (before he went full-time entrepreneur).
Thanks, glad that this is helpful!
Re: Developing an LLM: Building, Training, Finetuning (A 1h Video Explainer)
#4Not Sebastian (who I assume is the OP), but his blog/substack is also a great resource
Re: Developing an LLM: Building, Training, Finetuning (A 1h Video Explainer)
#5Is anyone training LLMs outside of Meta, OpenAI, etc... ?
I don't much get the point. For huge models, it's impossible to outcompete them. For smaller models, isn't mistral or LLaMa good enough?
What are other startups finetuning LLMs for?
Re: Developing an LLM: Building, Training, Finetuning (A 1h Video Explainer)
#6Is anyone training LLMs outside of Meta, OpenAI, etc... ? I don't much get the point. For huge models, it's impossible to outcompete them. For smaller models, isn't mistral or LLaMa good enough? What are other startups finetuning LLMs for?
I wouldn't pretrain from scratch, but continued pretraining is pretty popular for adapating LLMs to recent and/or custom data. (Sometimes this is referred to 'finetuning', however, not to be confused with 'instruction finetuning').
Re: Developing an LLM: Building, Training, Finetuning (A 1h Video Explainer)
#7Not Sebastian (who I assume is the OP), but his blog/substack is also a great resource https://magazine.sebastianraschka.com/
thanks for mentioning, that makes me super happy to hear!
Re: Developing an LLM: Building, Training, Finetuning (A 1h Video Explainer)
#8Is anyone training LLMs outside of Meta, OpenAI, etc... ? I don't much get the point. For huge models, it's impossible to outcompete them. For smaller models, isn't mistral or LLaMa good enough? What are other startups finetuning LLMs for?
I find it can be nice to have an academic understanding of things you work with even if you don't have to develop it directly yourself.
Re: Developing an LLM: Building, Training, Finetuning (A 1h Video Explainer)
#9Is anyone training LLMs outside of Meta, OpenAI, etc... ? I don't much get the point. For huge models, it's impossible to outcompete them. For smaller models, isn't mistral or LLaMa good enough? What are other startups finetuning LLMs for?
I find it can be nice to have an academic understanding of things you work with even if you don't have to develop it directly yourself.
Agreed, understanding how a method works and how it would be done helps with developing an intuition for its limitations -- what it can and what it can't do
Re: Developing an LLM: Building, Training, Finetuning (A 1h Video Explainer)
#10[deleted]