Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stanford A.I. Courses

ai.stanford.edu

Post reply on HN
Print view View original

41–50 of 81 posts

Re: Stanford A.I. Courses

#41

Post by 90d »

itissid wrote: post #14

I think the way courses are taught can give you some needed grounding, like you should always take a good linear regression class. But I think that is as far as it gets you, a theoretical base. Honestly the issue is that most ML programs are taught as being some kind of additive skill set: the more courses you take the better or selection of the right kind of courses gets you some where. In reality: 1. most real worl…

Large problem with social media ad targeting recommendations. They are completely unweighted.

What you think you are targeting : 'business owners researching startups and investing'.

And somehow your ads get pushed to 'people with interest in dogs' when you go with the recommendations.

Re: Stanford A.I. Courses

#42

Post by godelski »

itissid wrote: post #14

I think the way courses are taught can give you some needed grounding, like you should always take a good linear regression class. But I think that is as far as it gets you, a theoretical base. Honestly the issue is that most ML programs are taught as being some kind of additive skill set: the more courses you take the better or selection of the right kind of courses gets you some where. In reality: 1. most real worl…

I think this is because of a misalignment that is even common in plenty of other subjects as well. You know how once you've gained expertise in something that it is difficult to explain because it is so obvious? Kinda what is happening in education. Let me explain.

The reason a lot of the theoretical basis is taught is because you need to get the skills to learn why things work, when to use them, when they fail, when not to use them, and __most importantly__ their limitations. The problem is, most of this isn't explained explicitly. Maybe just this process happening for a few decades and momentum. Or that teaching isn't a priority and so no one tries to fix it. (there are exceptions to this. You've all probably met professors that are outstanding and make boring things seem fascinating)

But what you're talking about is part of this "when to use, what to use" part. It is also why those classes are so boring, because they aren't properly motivated. But it is also why we're running into so many problems: because evaluation is fucking hard. You see models perform really well on research papers but not in the real world but you'll also see researchers evaluating papers purely on singular benchmarks. "In reality" you're forced to come to terms with the limitations of the limitations of datasets, as datasets are just proxies and what you are about is the actual generalization. But if we're not discussing and evaluating on actual generalization in research then we get this dichotomy.

There's definitely more efficient (tractable) posterior estimators that work at large scale but just a lot of stuff isn't really known unless you're in that niche yourself. Statistics is often taught from the reference of "here's a bunch of tools and when to use them" rather than "here's the problems, our assumptions, and the main tool we use to solve them. It looks different in different settings, but they are actually the same thing." So it is kinda problematic, but then again, to get there requires a lot more work and most people aren't going to bother with things like metric theory. So a middle ground approach is taken and it gets jumbled.

Re: Stanford A.I. Courses

#43

Post by godelski »

ripvanwinkle wrote: post #9

Looking for guidance here. There are a lot of courses out there on AI from esteemed institutions at that. What do people recommend as a curriculum for someone with a formal univ education in CS albeit from a while ago and who has programmed extensively though not in Python. The goal at the end is to have a deep understanding of the LLM space and its adjacencies.

> The goal at the end is to have a deep understanding of the LLM space and its adjacency.

This is kinda a hard thing to quantify. How are we defining deep? Like you want to understand how they work? The Karpathy videos are good for that. But I wouldn't call this "deep".

If you want to get down into the weeds and into the mud, you need a hell of a lot more than 13hrs of education. You're also going to have a hard time doing this because most people are going from an engineering perspective of "enough to work with it" rather than "I fundamentally want to understand all inner workings". If you are the former, then the fastai course and others are great for you. If you want to really get deep though, you're going to need a lot more than programming. You're going to need some pretty advanced maths too: high dimensional statistics, metric theory, and optimization theory are some. (Most researchers aren't doing this btw) But if you do go down this path you'll also be able to understand the full spectrum of generative models and have a clearer picture. But I should also say that there is still a black box element to these models as they are so large that they are near impossible to analyze. But it is definitely achievable to learn a 2 layer transformer autoregressive network and fully understand its inner workings. But programming skills alone won't get you there.

Re: Stanford A.I. Courses

#44

Post by godelski »

bob88jg wrote: post #20
chongli wrote: post #13

Earlier quoted context omitted.

How long has it been since you studied/used university-level math? Calculus and linear algebra in particular. I ask because it’s pretty difficult to get through the math of backprop without a firm grasp of these. The Python part is trivial by comparison, the main difficulty being the matching of dimensions.

It's nothing more than the chain rule...University level it is not...the engineering aspect is the non trivial part IMHO...

While many learn calculus in high school, many also don't get it till uni. Not everyone is at your level, or took your same path, and that's okay. Don't shame people for not knowing when they're trying to learn.

Re: Stanford A.I. Courses

#45

Post by rahimnathwani »

samstave wrote: post #39
ripvanwinkle wrote: post #9

Looking for guidance here. There are a lot of courses out there on AI from esteemed institutions at that. What do people recommend as a curriculum for someone with a formal univ education in CS albeit from a while ago and who has programmed extensively though not in Python. The goal at the end is to have a deep understanding of the LLM space and its adjacencies.

This might be taboo, but I used ChatGPT to educate me on some basic concepts, then deeper concepts - and put together a learning-plan and a syllabus for me with also a glossary of terms.... The cool thing, is it helped me put a more structured thought process on how I should pursuing AI leanings... I couldnt find anything concise out there - and this helped me to better think through everything: If anything - its a g…

The link fails for me. I see a red banner saying: Unable to load conversation 7a36b5dc-0016-4b4c-bf1a-c3a66dac7c6d

Re: Stanford A.I. Courses

#46

Post by chongli »

bob88jg wrote: post #20
chongli wrote: post #13

Earlier quoted context omitted.

How long has it been since you studied/used university-level math? Calculus and linear algebra in particular. I ask because it’s pretty difficult to get through the math of backprop without a firm grasp of these. The Python part is trivial by comparison, the main difficulty being the matching of dimensions.

It's nothing more than the chain rule...University level it is not...the engineering aspect is the non trivial part IMHO...

The gradient of softmax is beyond most high school calculus students.

Re: Stanford A.I. Courses

#47

Post by mhh__ »

godelski wrote: post #42
itissid wrote: post #14

I think the way courses are taught can give you some needed grounding, like you should always take a good linear regression class. But I think that is as far as it gets you, a theoretical base. Honestly the issue is that most ML programs are taught as being some kind of additive skill set: the more courses you take the better or selection of the right kind of courses gets you some where. In reality: 1. most real worl…

I think this is because of a misalignment that is even common in plenty of other subjects as well. You know how once you've gained expertise in something that it is difficult to explain because it is so obvious? Kinda what is happening in education. Let me explain. The reason a lot of the theoretical basis is taught is because you need to get the skills to learn why things work, when to use them, when they fail, when…

The people with experience also aren't necessarily the ones that end up teaching - which isn't to say the same information can't be conveyed necessarily (e.g. good academics keep up to date with the field in industry) but there is powerful focus that practical experimentation brings.

Re: Stanford A.I. Courses

#48

Post by mhh__ »

antegamisou wrote: post #18

Why is Convex Optimization (EE364a) not included? https://stanford.edu/class/ee364a/ https://www.youtube.com/playlist?list=PL3940DD956CDF0622 It's one of the best courses to take if you want to obtain some fundamental understanding of the mathematical concepts behind AI. Yes there's much more to it than NNs/transformers/'Attention is all you need' paper/whatever else is trendy right now. No, don't expect to do import…

> Yes there's much more to it than NNs/transformers/'Attention is all you need' paper/whatever else is trendy right now. No, don't expect to do important, as in employable, work if you won't be spending some time truly understanding the mathematical foundations.

People only seem to want the former.

Re: Stanford A.I. Courses

#49

Post by giuscri »

itissid wrote: post #14

I think the way courses are taught can give you some needed grounding, like you should always take a good linear regression class. But I think that is as far as it gets you, a theoretical base. Honestly the issue is that most ML programs are taught as being some kind of additive skill set: the more courses you take the better or selection of the right kind of courses gets you some where. In reality: 1. most real worl…

so what’s a good course / book or way to learn more?

Re: Stanford A.I. Courses

#50

Post by blueyoda »

godelski wrote: post #43
ripvanwinkle wrote: post #9

Looking for guidance here. There are a lot of courses out there on AI from esteemed institutions at that. What do people recommend as a curriculum for someone with a formal univ education in CS albeit from a while ago and who has programmed extensively though not in Python. The goal at the end is to have a deep understanding of the LLM space and its adjacencies.

> The goal at the end is to have a deep understanding of the LLM space and its adjacency. This is kinda a hard thing to quantify. How are we defining deep? Like you want to understand how they work? The Karpathy videos are good for that. But I wouldn't call this "deep". If you want to get down into the weeds and into the mud, you need a hell of a lot more than 13hrs of education. You're also going to have a hard time…

Thanks for the helpful advice. What would you recommend to someone who is interested in learning about diffusion models? I have a CS degree but I have 0 knowledge about AI. Things like Stable Diffusion have blown my mind and I’m really interested in learning about this field. Lots of courses out there but I lack the expertise to discern which one is good.
Post reply on HN