Earlier quoted context omitted.
Quoted post unavailable.
Personal attacks will get you banned here. Please don't post like this again. https://news.ycombinator.com/newsguidelines.html
No post body was provided.
Practical Deep Learning for Coders 2022
71–80 of 86 posts
Re: Practical Deep Learning for Coders 2022
#72Earlier quoted context omitted.
I think you completely misunderstand his stance. You don't need the math in the beginning to train a model and get first results. Later, you will need the math and Jeremy clearly knows the math. He gives a great example: In sports, you don't start with learning about physiology and train individual muscles etc. (I paraphrase), you start playing basketball or baseball or soccer, and understand the overall game. And if…
It is good enough to not need heavy math to begin . Yeah, I know. But you need a lot of math to do Deep Learning. But I do not think Howard tries to communicate that. You can't show me people who knows high school math only and gets to work in FAANG, or PhD in DL/related, or CTO of an AI start-up, or anyhow "made it" in DL.
I think co-writing and co-teaching a math course at a deep learning company he co-founded, pinning that to the github repo and moving it close to the top of the home page makes it pretty clear he does see value in math in deep learning. I mean, if you need other evidence beyond than the fact that he teaches math needed to understand and build things from scratch in the deep learning course...
In the courses he has always been clear you don't need a ton of math to begin. He's also always been clear that as you progress you will encounter math that you need to learn to continue. He's always clear that that is ok if you don't know it before you start and it's ok to learn it when you need it.
Re: Practical Deep Learning for Coders 2022
#73Earlier quoted context omitted.
Not him nor will I talk about his course, but I’ve been in the field a reasonable amount of time (both on the academia and industry side). Honestly, applied maths will get you a long way and make it easier to digest the concepts (you might just see them as repackaged problems depending on your mileage). If you have good programming skills and discipline you practically have most of what you need. Re the course, I jus…
out of curiosity, how much applied math should one bone up on? (Obviously the more the better, but diminishing marginal returns and all that.)
Bare minimum is basic calculus, basic linear algebra and basic statistics. By basic, I probably mean first courses for those in most undergraduate programs.
I disagree with needing none and just going along as needed. That’s how you have machine learning models that look like they work but you don’t understand why they work so there might actually be problems.
Re: Practical Deep Learning for Coders 2022
#74The course uses PyTorch, which despite being a big played still has many issues running g on any of the latest AMD cards.
Windows not supported on AMD cards Navi series cards not supported in general. Heavily biased towards CUDA, despite AMD cards drivers being open sourced far more than Nvidia cards.
Remote machines and kaggle notebooks go some way to improving these limits for the course. I'm complaining a bit more in general here, I think.
Re: Practical Deep Learning for Coders 2022
#75Earlier quoted context omitted.
Thank you so much for this course. I plan to go through it properly. I have a search problem of my own and I have had a hard time applying what I have learnt (including the coursera DL specialization). The chief characteristics are: (a) It is a fuzzy search of a corpus that is in a non-English language. (b) The search should be able to run on a mobile phone _offline_. Is this possible? Can training be done elsewhere…
Have you tried... a) BM25 after some preprocessing (lemmatization etc.) b) fastText / GloVe (possibly weighted by BM25) The results can be surprisingly good. Often no need to bother with big language models or GPUs.
As far as I understand, BM25 is not for fuzzy searches. For a bit more context, the search terms are in the English script, but the words are basically the closest-sounding transcriptions of sounds in various Indian languages. Different people may render the same word differently in an English transcription. But there's enough crowd-sourced data to account for the ways in which words can vary.
For the same reason, GloVe is of not much use to me.
Re: Practical Deep Learning for Coders 2022
#76Earlier quoted context omitted.
Thank you so much for this course. I plan to go through it properly. I have a search problem of my own and I have had a hard time applying what I have learnt (including the coursera DL specialization). The chief characteristics are: (a) It is a fuzzy search of a corpus that is in a non-English language. (b) The search should be able to run on a mobile phone _offline_. Is this possible? Can training be done elsewhere…
If the volume of data fits on a mobile phone for it to be offline, perhaps you don't need deep learning?
Perhaps. What alternative would you suggest? The search terms are fuzzy, and there are too many variants (not exactly misspellings) for me to encode them explicitly. So I thought I'd rather learn from a crowd-sourced corpus.
In my case, I don't want the model to be general. I can afford for it to be like a database index, tailored to that data.
Re: Practical Deep Learning for Coders 2022
#77Earlier quoted context omitted.
Have you tried... a) BM25 after some preprocessing (lemmatization etc.) b) fastText / GloVe (possibly weighted by BM25) The results can be surprisingly good. Often no need to bother with big language models or GPUs.
As far as I understand, BM25 is not for fuzzy searches. For a bit more context, the search terms are in the English script, but the words are basically the closest-sounding transcriptions of sounds in various Indian languages. Different people may render the same word differently in an English transcription. But there's enough crowd-sourced data to account for the ways in which words can vary. For the same reason, Gl…
OK, I understand. That is a different problem indeed. Although I am not sure that BERT would be of much more help there either, unless you have quite a large training corpus. The simplest/cheapest approach might be some kind of transcription normalization. As you say, what is introducing ambiguity here is the act of transcribing these sounds into the English alphabet. There’s not a single source language/alphabet? Also, I am not sure if fastText/GloVe really would not pick up the semantic similarity in spite of differing transcriptions as long as you have a large enough training corpus. I’d experiment with the settings here (bag of words/skipgram, min/max lengths etc.).
Re: Practical Deep Learning for Coders 2022
#78Hi folks - I'm the creator/teacher of this course. I'd be happy to answer any questions that you have about the course, learning deep learning in general, or the state of deep learning in 2022.
Will it help me more or achieve more out of learning this course as compared to just directly using GPT-3 or Dall-E as paid user?
Re: Practical Deep Learning for Coders 2022
#79Hello Jeremy. Thanks for this great content. I have gone through your entire course and learned a ton from you.
Now moving on from here, do you have any resource recommendation where I can dive deeper into machine learning and deep learning theory? And also any resources to become a much much better programmer?
I am currently working in as an assistant in a research lab. My coding skills are not that great.
Re: Practical Deep Learning for Coders 2022
#80I haven’t seen this course content yet, but fully did the 2019 version. Extremely grateful to have found it. Changed the course of my life. I can vouch for it's quality. Jeremy is an excellent instructor. So much clarity in his teaching! I love that this is a hands-on course, and there are ZERO hand-wavings. I also really like the top-down approach of teaching. Now, whenever I try to communicate something or teach so…
Yes, updated second part of the course when? Any chance you will shift from Swift to Julia?