Ask HN: How to Learn LLM Fundamentals

#1

Post by AETackaberry » Thu, Mar 30, 2023, 4:54 AM UTC

As a Software Engineer that is interested in learning LLM fundamentals, what is your recommended course of action? Are there any courses or curricula you would recommend?

Re: Ask HN: How to Learn LLM Fundamentals

#2

Post by hislaziness » Thu, Mar 30, 2023, 5:06 AM UTC

I really liked this series. https://karpathy.ai/zero-to-hero.html

Re: Ask HN: How to Learn LLM Fundamentals

#3

Post by AETackaberry » Thu, Mar 30, 2023, 5:12 AM UTC

hislaziness wrote: post #2

I really liked this series. https://karpathy.ai/zero-to-hero.html

I am definitely planning to start with this series, seems like a no-brainer!

The part I'm unsure of is what next? Seems like diving into papers might be a decent leap. Who knows, maybe GPT-4 can help me bridge the gap

Re: Ask HN: How to Learn LLM Fundamentals

#4

Post by quickthrower2 » Thu, Mar 30, 2023, 5:28 AM UTC

hislaziness wrote: post #2

I really liked this series. https://karpathy.ai/zero-to-hero.html

Thanks that series looks quite fun to do

Re: Ask HN: How to Learn LLM Fundamentals

#5

Post by hislaziness » Thu, Mar 30, 2023, 6:07 AM UTC

AETackaberry wrote: post #3

hislaziness wrote: post #2

I really liked this series. https://karpathy.ai/zero-to-hero.html

I am definitely planning to start with this series, seems like a no-brainer! The part I'm unsure of is what next? Seems like diving into papers might be a decent leap. Who knows, maybe GPT-4 can help me bridge the gap

Papers would be good next step. The concept and the algorithms do not seem very complex. The training and fine tuning is what needs effort and infrastructure.

Re: Ask HN: How to Learn LLM Fundamentals

#6

Post by seydor » Thu, Mar 30, 2023, 10:05 AM UTC

There s this guy who has made in depth videos about transformers https://www.youtube.com/watch?v=O3xbVmpdJwU

Re: Ask HN: How to Learn LLM Fundamentals

#7

Post by is_true » Thu, Mar 30, 2023, 12:08 PM UTC

I think this blog posts is nice overview to help you go deeper in the topics mentioned: https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-...

Re: Ask HN: How to Learn LLM Fundamentals

#8

Post by jytechdevops » Thu, Mar 30, 2023, 5:33 PM UTC

What a coincidence, I actually ran across this github repo for LLM fundamentals today: https://gist.github.com/rain-1/eebd5e5eb2784feecf450324e3341...

gl on your journey

Re: Ask HN: How to Learn LLM Fundamentals

#9

Post by TroyZ » Fri, Mar 31, 2023, 2:24 AM UTC

For a high level overview of transformers/GPT, you can check out this article: https://medium.com/design-bootcamp/how-chatgpt-really-works-...

The article provides references to the original papers and other articles explaining the subject, which can be great sources to dive deeper.

Re: Ask HN: How to Learn LLM Fundamentals

#10

Post by felipelalli » Fri, Mar 31, 2023, 5:48 AM UTC

http://www.iasylum.net/writings/2023-03-29-if-you-are-curiou...