31–40 of 81 posts
Looking for guidance here. There are a lot of courses out there on AI from esteemed institutions at that. What do people recommend as a curriculum for someone with a formal univ education in CS albeit from a while ago and who has programmed extensively though not in Python. The goal at the end is to have a deep understanding of the LLM space and its adjacencies.
Looking for guidance here. There are a lot of courses out there on AI from esteemed institutions at that. What do people recommend as a curriculum for someone with a formal univ education in CS albeit from a while ago and who has programmed extensively though not in Python. The goal at the end is to have a deep understanding of the LLM space and its adjacencies.
Honestly this is all moving so fast you can do well by reading the news, following a few reddits/substacks, and skimming the prompt engineering papers as they come out every week (!).
https://www.latent.space/p/ai-engineer provides an early manifesto for this nascent layer of the stack.
Zvi writes a good roundup (though he is concerned mostly with alignment so skip if you don’t like that angle): https://thezvi.substack.com/p/ai-18-the-great-debate-debates
Simon W has some good writeups too: https://simonwillison.net/
I strongly recommend playing with the OpenAI APIs and working with langchain in a Colab notebook to get a feel for how these all fit together. Also, the tools here are incredibly simple and easy to understand (very new) so looking at, say, https://github.com/minimaxir/simpleaichat/tree/main/simpleai... or https://github.com/smol-ai/developer and digging in to the prompts, what goes in system vs assistant roles, how you guide the LLM, etc.
I would not start with any course unless you had a project in mind that would take the knowledge from the course to get started. Otherwise you risk wasting a lot of time for knowledge that won't help you in any way and will get outdated in a few months anyway
I took a deep learning course in late 2019, during which we implemented transformers as described in Attention is All You Need and fine-tuned GPT-2. The output was amusing, but useless, but I still remember the basic principles. Now, a few years later, transformers are the tech, and GPT-2's successors are the most hyped technologies of the century so far. All of which is to say that I wouldn't assume that coursework…
Which course was this?
Earlier quoted context omitted.
There's no great answer to this question. It is a bunch of tricks. Fundamentally: If you're saying FizzBuzz doesn't work, presumably you mean that encoding the n directly doesn't work. Neither does encoding n from 0 to 1 or between -1 and 1 (and don't forget: obviously don't use relu with -1 to 1). It doesn't. Neural networks can do a LOT of things, but they cannot deal with numbers. And they certainly cannot deal wi…
Thanks for answering, what you wrote here is exactly the sort of thing I'm talking about. Something implicit that's known but not obvious if you look at the first few lectures of the first few courses (or blogs or announcements, etc). You mention bag of tricks and that's indeed one issue but its worse than that because it includes knowing what "silent problems" needs a trick applied to it in the first place! Indeed,…
Looking for guidance here. There are a lot of courses out there on AI from esteemed institutions at that. What do people recommend as a curriculum for someone with a formal univ education in CS albeit from a while ago and who has programmed extensively though not in Python. The goal at the end is to have a deep understanding of the LLM space and its adjacencies.
Although I myself am not related to the industry or academia pertaining to AI, I have heard many people speak highly of the zero to hero course by Andrej Karpathy: https://youtube.com/playlist?list=PLAqhIrjkxbuWI23v9cThsA9Gv... I myself loved it and learned a lot, but YMMV
I would not start with any course unless you had a project in mind that would take the knowledge from the course to get started. Otherwise you risk wasting a lot of time for knowledge that won't help you in any way and will get outdated in a few months anyway
I took a deep learning course in late 2019, during which we implemented transformers as described in Attention is All You Need and fine-tuned GPT-2. The output was amusing, but useless, but I still remember the basic principles. Now, a few years later, transformers are the tech, and GPT-2's successors are the most hyped technologies of the century so far. All of which is to say that I wouldn't assume that coursework…
Why is Convex Optimization (EE364a) not included? https://stanford.edu/class/ee364a/ https://www.youtube.com/playlist?list=PL3940DD956CDF0622 It's one of the best courses to take if you want to obtain some fundamental understanding of the mathematical concepts behind AI. Yes there's much more to it than NNs/transformers/'Attention is all you need' paper/whatever else is trendy right now. No, don't expect to do import…
(as the CS237A Principles of Robotic Autonomy I link seems to be broken)
Looking for guidance here. There are a lot of courses out there on AI from esteemed institutions at that. What do people recommend as a curriculum for someone with a formal univ education in CS albeit from a while ago and who has programmed extensively though not in Python. The goal at the end is to have a deep understanding of the LLM space and its adjacencies.
The cool thing, is it helped me put a more structured thought process on how I should pursuing AI leanings...
I couldnt find anything concise out there - and this helped me to better think through everything:
If anything - its a good primer for getting your own thought process on the subject going...
https://chat.openai.com/c/7a36b5dc-0016-4b4c-bf1a-c3a66dac7c...
Earlier quoted context omitted.
How long has it been since you studied/used university-level math? Calculus and linear algebra in particular. I ask because it’s pretty difficult to get through the math of backprop without a firm grasp of these. The Python part is trivial by comparison, the main difficulty being the matching of dimensions.
It's nothing more than the chain rule...University level it is not...the engineering aspect is the non trivial part IMHO...