Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Daily practices for building AI/ML skills?

news.ycombinator.com

Post reply on HN
Print view View original

91–100 of 135 posts

Re: Ask HN: Daily practices for building AI/ML skills?

#91

Post by gwbas1c »

One thing to point out: Try not to let your imagination run away, or get overconfident in what AI/ML can do.

I worked for a major company on an ML project for 2 years. By the time I left, I realized that:

1: The project I was working on has no improvement over ordinary statistical methods; yet the ability for people to understand the statistics (over the black box of ML) meant that the project had no tangible improvement over the processes we were trying to replace.

2: A lot of the ML I was working on was a solution in search of a problem.

I personally found the ML system I was working on fascinating; but the overconfidence about what it can infer, and the way that non-developers thought ML could make magical inferences, frustrating.

---

One other thing: Make sure you understand how to use databases, both SQL and non-SQL. In order to use ML effectively, you will need to be excellent at programming with large volumes of data in a performant manner.

Re: Ask HN: Daily practices for building AI/ML skills?

#92

Post by xianshou »

Not a complete answer, but here are the most helpful resources for understanding transformer basics in particular:

Original transformer paper: https://arxiv.org/abs/1706.03762

Illustrated transformer: http://jalammar.github.io/illustrated-transformer/

Transformer visualization: https://bbycroft.net/llm

minGPT (Karpathy): https://github.com/karpathy/minGPT

---

Next, some foundational textbooks for general ML and deep learning:

Elements of Statistical Learning (aka the bible): https://hastie.su.domains/ElemStatLearn/

Probabilistic ML: https://probml.github.io/pml-book/book2.html

Deep Learning Book (Goodfellow/Bengio): https://www.deeplearningbook.org/

Understanding Deep Learning: https://udlbook.github.io/udlbook/

---

Finally, assorted tutorials/resources/intro courses:

Beyond the Illustrated Transformer: https://news.ycombinator.com/item?id=35712334

AI Zero to Hero: https://karpathy.ai/zero-to-hero.html

AI Canon: https://a16z.com/2023/05/25/ai-canon/

LLM University by Cohere: https://llm.university/

Practical Guide to LLMs: https://github.com/Mooler0410/LLMsPracticalGuide

Practical Deep Learning for Coders: https://course.fast.ai/Lessons/part2.html

---

Hope that helps!

Re: Ask HN: Daily practices for building AI/ML skills?

#93

Post by yCombLinks »

mickael-kerjean wrote: post #74
DeathArrow wrote: post #21

Earlier quoted context omitted.

ML is so much more than just neural networks. I would start by taking a free university level course in statistics. Then I would continue with the basics: SVM, linear regression, naive Bayes, gradient boosting, neural nets etc. I would not only train and fine tune them, but I would also build simple ones myself instead of just using libraries. Then I would continue to what you said, participate in Kaggle competitions…

> SVM, linear regression, naive Bayes When I studied ML in 2012, the very first course started with naive Bayes and went one from there. A decade after being away, I see a lot of people around me starting with neural nets to train a model that naive Bayes would be plenty enough for and never heard about naive Bayes. Is that only my experience?

Any data that is small enough to quickly iterate on for learning is small enough to use a simpler approach. The point is learning the techniques.

Re: Ask HN: Daily practices for building AI/ML skills?

#94

Post by d4rkp4ttern »

Specifically for LLMs— I recently gave a guest lecture on Intro to LLMs for non-CS (biomed) grad students. I wanted to assign a homework quiz but didn’t find any good ones, so I made a multiple choice quiz. It’s a bit “evil”: it trips you up if you don’t have a solid understanding. Several of the questions have nuances that both test your understanding and also help you learn by figuring out the right answer. It’s a google form that does NOT collect emails:

https://docs.google.com/forms/d/e/1FAIpQLScbWN3qwqeIc0b1cCRq...

Note this is for absolute LLM beginners, not if you’re already working with LLMs -- but even some of these folks have found it useful!

Hope you find this useful.

Re: Ask HN: Daily practices for building AI/ML skills?

#95

Post by owlbite »

mickael-kerjean wrote: post #74
DeathArrow wrote: post #21

Earlier quoted context omitted.

ML is so much more than just neural networks. I would start by taking a free university level course in statistics. Then I would continue with the basics: SVM, linear regression, naive Bayes, gradient boosting, neural nets etc. I would not only train and fine tune them, but I would also build simple ones myself instead of just using libraries. Then I would continue to what you said, participate in Kaggle competitions…

> SVM, linear regression, naive Bayes When I studied ML in 2012, the very first course started with naive Bayes and went one from there. A decade after being away, I see a lot of people around me starting with neural nets to train a model that naive Bayes would be plenty enough for and never heard about naive Bayes. Is that only my experience?

As with any super-hyped technology, people want to use the latest and greatest to solve it, even when traditional methods are cheaper, easier, more reliable and more accurate. See also: crypto.

Re: Ask HN: Daily practices for building AI/ML skills?

#97

Post by IKantRead »

janalsncm wrote: post #18

I got a masters degree in ML at a good school. I will say there’s pretty much nothing they taught me that I couldn’t have learned myself. That said, school focused my attention in ways I wouldn’t have alone, and provided pressure to keep going. The single thing which I learned the most from was implementing a paper. Lectures and textbooks to me are just words. I understand them in the abstract but learning by doing g…

> don’t waste your time writing your own neural net and backprop.

I don't think you should be combining writing a neural network with doing backprop since I don't know anyone working with serious ML who is not using some sort of automatic differentiation library to handling the backprop part for them. I'm not entirely sure people even know what they're saying when they talk about backprop these days, and I suspect they're confusing it with gradient optimization.

But anyone seriously interested in ML absolutely should be building their own models from scratch and training them with gradient descent, ideally start with building out your own optimization routine rather than using a prepackaged one.

This is hugely important since the optimization part of the learning is really the heart of modern machine learning. If you really want to understand ML you should have a strong intuition about various methods of optimizing a given model. Additionally there are lots of details and tricks behind these models that are ignored if you're only calling an api around these models.

There's a world of difference between implementing an LSTM and calling one. You learn significantly more about what's actually happening by doing the former.

Re: Ask HN: Daily practices for building AI/ML skills?

#98

Post by culi »

janalsncm wrote: post #18

I got a masters degree in ML at a good school. I will say there’s pretty much nothing they taught me that I couldn’t have learned myself. That said, school focused my attention in ways I wouldn’t have alone, and provided pressure to keep going. The single thing which I learned the most from was implementing a paper. Lectures and textbooks to me are just words. I understand them in the abstract but learning by doing g…

> My $0.02: don’t waste your time writing your own neural net and backprop.

I think it's worth doing a very simple implementation at least once to ensure you have the fundamentals memorized. It's not actually that complicated to implement a very simple one. Maybe a day or two -long project

Re: Ask HN: Daily practices for building AI/ML skills?

#99

Post by sanderjd »

uoaei wrote: post #47
sanderjd wrote: post #19

Earlier quoted context omitted.

This seems like great advice. You say don't write your own neural net and backprop implementation. That makes sense to me. What do you suggest using instead, for your suggested projects? I'm guessing tensorflow, based on your suggestions on profiling and debugging tools? Do the papers / projects you suggest map straightforwardly onto a tensorflow implementation, rather than a custom one?

Implementations in Tensorflow are widely considered to be technical debt. Internally, Google has mostly switched to JAX. PyTorch now has torch.compile and exports to ONNX so there's little reason to use Tensorflow these days except in niche cases.

This kind of information is why I was asking :)

Re: Ask HN: Daily practices for building AI/ML skills?

#100

Post by digging »

ekianjo wrote: post #75
wanderingmind wrote: post #32

Earlier quoted context omitted.

This type of bottom up approach is terrible idea for a fast moving area like ML. Ultimately to get a job and make money in the area, you need to solve customer problems, starting with libraries and fine tuning is what needs to happen first. However you should try to learn fundamentals as you go and when you get stuck. It will take a very long time if you have a full time job or a student studying something else and d…

> his type of bottom up approach is terrible idea for a fast moving area like ML. Of course not. When you learn this way you will realize there are a myriad of problems that can be solved with more simple algorithms. Trying to make every problem fit to neural networks is pure cargo culting.

That's true, but the goal is to learn to use neural networks, not to solve problems efficiently, isn't it?
Post reply on HN