Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stanford A.I. Courses

ai.stanford.edu

Post reply on HN
Print view View original

61–70 of 81 posts

Re: Stanford A.I. Courses

#61

Post by nocoder »

tonmoy wrote: post #12
ripvanwinkle wrote: post #9

Looking for guidance here. There are a lot of courses out there on AI from esteemed institutions at that. What do people recommend as a curriculum for someone with a formal univ education in CS albeit from a while ago and who has programmed extensively though not in Python. The goal at the end is to have a deep understanding of the LLM space and its adjacencies.

Although I myself am not related to the industry or academia pertaining to AI, I have heard many people speak highly of the zero to hero course by Andrej Karpathy: https://youtube.com/playlist?list=PLAqhIrjkxbuWI23v9cThsA9Gv... I myself loved it and learned a lot, but YMMV

Would you mind sharing, what level of programming & mathematical background do I need? I know basic python (read python for data analysis) & currently half way through elements of statistical learning. What else do I need to learn?

Re: Stanford A.I. Courses

#62

Post by lazybusy »

itissid wrote: post #14

I think the way courses are taught can give you some needed grounding, like you should always take a good linear regression class. But I think that is as far as it gets you, a theoretical base. Honestly the issue is that most ML programs are taught as being some kind of additive skill set: the more courses you take the better or selection of the right kind of courses gets you some where. In reality: 1. most real worl…

[flagged]

Re: Stanford A.I. Courses

#63

Post by balthigor »

godelski wrote: post #58
blueyoda wrote: post #50

Earlier quoted context omitted.

Thanks for the helpful advice. What would you recommend to someone who is interested in learning about diffusion models? I have a CS degree but I have 0 knowledge about AI. Things like Stable Diffusion have blown my mind and I’m really interested in learning about this field. Lots of courses out there but I lack the expertise to discern which one is good.

Yeah no problem, this is even closer to my area of focus! What do you know about physics and thermodynamics? I'd say a good intro for low background is from Tomczak[0]. He has a book, but the blog posts are nearly identical. He did a post doc with Max Welling (someone you should learn about if you want to get deep, like I was suggesting before). So I'd switch things up slightly. I'd go Intro -> Autoregressive -> Flow…

The quality of these recommendations reflects favourably on OP.

Re: Stanford A.I. Courses

#64

Post by ra7 »

AJRF wrote: post #28

I've moved from "traditional" software engineering to a role of working with ML (building + deploying models used in product features) and of the team I work with - and my extended communication with developers at other companies making the same transition - every single person has said the Francis Chollet book (Deep Learning with Python) is all they really needed. It walks a very thin line between too little info an…

Thanks for the book recommendation! I’m interested in making the same transition. Can I ask what you did to be considered for a role in ML coming from a software engineering background? Did you showcase any personal projects in your resume?

Re: Stanford A.I. Courses

#65

Post by tczMUFlmoNk »

mellavora wrote: post #57
Exuma wrote: post #26

Care to explain cross entropy simply? That’s where I paused currently

Shooting from the hip: entropy of a single signal, say a sequence of letters, "ababababab" is the scaled "average" surprise per letter. So if they are uniformly distributed, each letter is equally likely/unlikely to come next in the sequence, where if instead one letter only 1/1000th of the time (aaa....aaa...aa..a.z.aaaa), then when the rare beast shows up, it is a big surprise, so the total amount of surprise avail…

Mostly yes, but to your second paragraph:

> if instead one letter only 1/1000th of the time (aaa....aaa...aa..a.z.aaaa), then when the rare beast shows up, it is a big surprise, so the total amount of surprise available in the sequence is high

…when a Bernoulli distribution is skewed, the maximum surprise is high, yes, but the average surprise (= entropy) is low. The entropy of a Bernoulli distribution is maximized when p = 0.5 and falls off to either end:

https://en.wikipedia.org/wiki/Binary_entropy_function

For your examples, if the sequence is uniformly distributed (Bernoulli(1/2)), the entropy is log(2) ≈ 0.693 bits per symbol; if instead one letter occurs 1/1000th of the time, the entropy is about 0.0079 bits per symbol.

Re: Stanford A.I. Courses

#66

Post by aaarrm »

itissid wrote: post #14

I think the way courses are taught can give you some needed grounding, like you should always take a good linear regression class. But I think that is as far as it gets you, a theoretical base. Honestly the issue is that most ML programs are taught as being some kind of additive skill set: the more courses you take the better or selection of the right kind of courses gets you some where. In reality: 1. most real worl…

So in your opinion, what is the best way to learn ML in order to try and avoid these issues

Re: Stanford A.I. Courses

#67

Post by AJRF »

ra7 wrote: post #64
AJRF wrote: post #28

I've moved from "traditional" software engineering to a role of working with ML (building + deploying models used in product features) and of the team I work with - and my extended communication with developers at other companies making the same transition - every single person has said the Francis Chollet book (Deep Learning with Python) is all they really needed. It walks a very thin line between too little info an…

Thanks for the book recommendation! I’m interested in making the same transition. Can I ask what you did to be considered for a role in ML coming from a software engineering background? Did you showcase any personal projects in your resume?

So this might not work for you, but I will tell you my path anyway.

1. Was one of the first members of the #ai slack channel inviting some people I had in person conversations about AI with.

2. I posted _a lot_ in there. Stuff about regulatory updates, people using co-pilot, cool github repos, little demo projects I was working on.

3. Now this was pure luck and probably the best thing to push me over the boundary, there was a hackathon. I thought "Hmm if I make a kick ass demo showcasing generative AI here, a lot of high up people will see it" - that 100% happened, CTO reached out to me saying demo was great and that people will be in touch.

4. I started really digging in to how I could provide value to our existing data team - be that code, deploying things, bringing some of my engineering know how to that team. This point the #ai channel really started to grow and the head of data and engineering started talking to me and directing people my way based on what they saw at the hackathon.

5. Did a demo of my hack in the company all hands which the CEO was MC'ing.

6. Started having fortnightly 1 to 1s with head of data at this point

7. Floated idea of team taking a little subset of good and motivated people from other teams for a short time to investigate and implement LLMs in some small way into our apps. That team has now grown to effectively investigate any and all use cases (internal and external) for generative AI.

8. I started reading more theory and also following a bit of a road map for things I should learn to have a better picture of how to actually bring LLMs in some form to production (fine-tuning, vector dbs, functions, guard rails).

9. Now I am just building some quick feature in the mobile app to show case the value of the team to exec as quick as I can, which should give us few months cover to work on the thing I am really interested in - multi-arm bandit LLM that uses our existing models.

This was pretty much it. Seems trivial, but in between each points was lots of reading, tinkering, working on weekends, but its totally possible. The ML + AI focused PhD's in your company likely need help from engineering but don't know it - bringing those two groups together quickly shows how you can be useful.

This post was helpful; https://blog.gregbrockman.com/how-i-became-a-machine-learnin...

Re: Stanford A.I. Courses

#68

Post by itissid »

giuscri wrote: post #49
itissid wrote: post #14

I think the way courses are taught can give you some needed grounding, like you should always take a good linear regression class. But I think that is as far as it gets you, a theoretical base. Honestly the issue is that most ML programs are taught as being some kind of additive skill set: the more courses you take the better or selection of the right kind of courses gets you some where. In reality: 1. most real worl…

so what’s a good course / book or way to learn more?

I don't know about ML but if you want to learn applied stats I would look up andrew gelman's or one of the newer books on Bayesian Inference ones using Stan and do them cover to cover.

Re: Stanford A.I. Courses

#70

Post by ra7 »

AJRF wrote: post #67
ra7 wrote: post #64

Earlier quoted context omitted.

Thanks for the book recommendation! I’m interested in making the same transition. Can I ask what you did to be considered for a role in ML coming from a software engineering background? Did you showcase any personal projects in your resume?

So this might not work for you, but I will tell you my path anyway. 1. Was one of the first members of the #ai slack channel inviting some people I had in person conversations about AI with. 2. I posted _a lot_ in there. Stuff about regulatory updates, people using co-pilot, cool github repos, little demo projects I was working on. 3. Now this was pure luck and probably the best thing to push me over the boundary, th…

Great response, thank you!
Post reply on HN