Ask HN: Daily practices for building AI/ML skills? • Hacker News Board

Re: Ask HN: Daily practices for building AI/ML skills?

#41

Post by janalsncm » Thu, Dec 14, 2023, 8:49 AM UTC

atomicnature wrote: post #20

janalsncm wrote: post #18

I got a masters degree in ML at a good school. I will say there’s pretty much nothing they taught me that I couldn’t have learned myself. That said, school focused my attention in ways I wouldn’t have alone, and provided pressure to keep going. The single thing which I learned the most from was implementing a paper. Lectures and textbooks to me are just words. I understand them in the abstract but learning by doing g…

Love the suggestions, especially the one about implementing papers. Do you have any starters on how one selects papers in the early days, to implement? Also - any great papers you recommend beginners expose themselves to?

The best site imo is Papers With Code. State of the art benchmarks, the papers which achieved them (along with previous papers) and github repos to actual implementations.

I wouldn’t recommend papers to absolute beginners though. For them, it’s best to go to HuggingFace, find a model that seems interesting and play with it in a Jupyter notebook. You’ll get a lot more bang for your buck.

Re: Ask HN: Daily practices for building AI/ML skills?

#42

Post by dacryn » Thu, Dec 14, 2023, 8:50 AM UTC

janalsncm wrote: post #18

I got a masters degree in ML at a good school. I will say there’s pretty much nothing they taught me that I couldn’t have learned myself. That said, school focused my attention in ways I wouldn’t have alone, and provided pressure to keep going. The single thing which I learned the most from was implementing a paper. Lectures and textbooks to me are just words. I understand them in the abstract but learning by doing g…

I'd argue backprop is still handy just to learn the basics

It doesn't have to be production ready of course, but spending 3-4 hours to write it out in code, debug a few steps, ... are useful in my opinion.

or at least watch the karpathy video and try to follow along

Re: Ask HN: Daily practices for building AI/ML skills?

#43

Post by janalsncm » Thu, Dec 14, 2023, 8:52 AM UTC

spaniard89277 wrote: post #40

janalsncm wrote: post #18

I got a masters degree in ML at a good school. I will say there’s pretty much nothing they taught me that I couldn’t have learned myself. That said, school focused my attention in ways I wouldn’t have alone, and provided pressure to keep going. The single thing which I learned the most from was implementing a paper. Lectures and textbooks to me are just words. I understand them in the abstract but learning by doing g…

I only have old hardware at home. How viable is to practice this stuff on my own projects (and I'd like to touch JS as much as possible, despite everyone being on python)

You don’t need your own hardware. You can use Google Colab for free.

Most of the action happens in python. That being said, there’s a library called Tensorflow JS. It has some pre-trained models you can use off the shelf and run from your browser. Things like face detection and sentiment analysis.

Re: Ask HN: Daily practices for building AI/ML skills?

#44

Post by pknerd » Thu, Dec 14, 2023, 8:54 AM UTC

zmgsabst wrote: post #36

pknerd wrote: post #31

Earlier quoted context omitted.

Sorry! Just updated my comment. I was talking about usage to build products

I’m not an expert, but I just picked a project and used the OpenAI API — but with a wrapper that should let me swap out backends if/when I get a computer with a nice GPU. Python is great for mixing API calls, document formatting, and other data scraping. For myself, the problem was finding something interesting to do — in my case, generating videos from basic prompts.

With 2015 Macbook, I do not have such luxury

Re: Ask HN: Daily practices for building AI/ML skills?

#45

Post by hereonout2 » Thu, Dec 14, 2023, 8:54 AM UTC

Presuming you want to work in the field and already have software development experience why not look at the confluence between ML and engineering?

Things like ML ops, application of DevOps, testing and ci/cd in the ml space, how to train across multiple gpus, how to actually host an LLM especially at scale and affordably.

In my experience there are hundreds of candidates coming from academia with strong academic backgrounds in ML. There are very few experienced engineers available to help them realise their ambitions!

Re: Ask HN: Daily practices for building AI/ML skills?

#46

Post by TrackerFF » Thu, Dec 14, 2023, 9:06 AM UTC

Roughly speaking, the roadmap for a typical ML/AI student looks like this:

0) Learn the pre-requisites of math, CS, etc. That usually means calc 1-3, linear algebra, probability and statistics, fundamental cs topics like programming, OOP, data structures and algorithms, etc.

1) Elementary machine learning course, which covers all the classic methods.

2) Deep Learning, which covers the fundamental parts of DL. Note, though, this one changes fast.

From there, you kind of split between ML engineering, or ML research.

For ML engineering, you study more technical things that relate to the whole ML-pipeline. Big data, distributed computing, way more software engineering topics.

For ML research, you focus more on the science itself - which usually involves reading papers, learning topics which are relevant to your research. This usually means having enough technical skills to translate research papers into code, but not necessarily at a level that makes the code good enough to ship.

I'll echo what others have said, though, use to tools at hand to implement stuff. It is fun and helpful to implement things from scratch, for the learning, but it is easy to get extremely bogged down trying to implement every model out there.

When I tried to learn "practical" ML, I took some model, and tried to implement it in such a way that I could input data via some API, and get back the results. That came with some challenges:

- Data processing (typical ETL problem)

- Developing and hosting software (core software engineering problems)

- API development

And then you have the model itself, lots of work goes toward that alone.

Re: Ask HN: Daily practices for building AI/ML skills?

#47

Post by uoaei » Thu, Dec 14, 2023, 9:22 AM UTC

sanderjd wrote: post #19

janalsncm wrote: post #18

I got a masters degree in ML at a good school. I will say there’s pretty much nothing they taught me that I couldn’t have learned myself. That said, school focused my attention in ways I wouldn’t have alone, and provided pressure to keep going. The single thing which I learned the most from was implementing a paper. Lectures and textbooks to me are just words. I understand them in the abstract but learning by doing g…

This seems like great advice. You say don't write your own neural net and backprop implementation. That makes sense to me. What do you suggest using instead, for your suggested projects? I'm guessing tensorflow, based on your suggestions on profiling and debugging tools? Do the papers / projects you suggest map straightforwardly onto a tensorflow implementation, rather than a custom one?

Implementations in Tensorflow are widely considered to be technical debt. Internally, Google has mostly switched to JAX. PyTorch now has torch.compile and exports to ONNX so there's little reason to use Tensorflow these days except in niche cases.

Re: Ask HN: Daily practices for building AI/ML skills?

#48

Post by zmgsabst » Thu, Dec 14, 2023, 9:23 AM UTC

pknerd wrote: post #44

zmgsabst wrote: post #36

Earlier quoted context omitted.

I’m not an expert, but I just picked a project and used the OpenAI API — but with a wrapper that should let me swap out backends if/when I get a computer with a nice GPU. Python is great for mixing API calls, document formatting, and other data scraping. For myself, the problem was finding something interesting to do — in my case, generating videos from basic prompts.

With 2015 Macbook, I do not have such luxury

Does it not run Python?

I’m working off a 4GiB system with no dedicated GPU — and the Python APIs for OpenAI work fine, but admittedly, I have a Linux system.

Re: Ask HN: Daily practices for building AI/ML skills?

#49

Post by theusus » Thu, Dec 14, 2023, 9:24 AM UTC

RamblingCTO wrote: post #25

theusus wrote: post #22

Try this https://www.bishopbook.com/ and solve the exercises. I would not recommend doing many things at once.

That's only deep learning. There's so much more in machine learning and I think getting the basics right is more important than focusing only on one area.

It does claim to teach from beginning. Once comfortable they can try to cover other domains toi.

Re: Ask HN: Daily practices for building AI/ML skills?

#50

Post by camhart » Thu, Dec 14, 2023, 9:28 AM UTC

tsurba wrote: post #39

wanderingmind wrote: post #32

Earlier quoted context omitted.

This type of bottom up approach is terrible idea for a fast moving area like ML. Ultimately to get a job and make money in the area, you need to solve customer problems, starting with libraries and fine tuning is what needs to happen first. However you should try to learn fundamentals as you go and when you get stuck. It will take a very long time if you have a full time job or a student studying something else and d…

Getting a job to make money and solve customer problems as fast as possible was not a stated goal by the OP. Besides, you are also wrong, having good fundamentals in the maths will help you pick up new methods much faster as they pop up. And especially if you want to come up with new methods (as in research), there are no shortcuts.

They made a reasonable assumption.

Abstraction helps you to be productive. It's certainly good to understand everything all the way down to the actual physics behind computing, but its not necessary especially at the start. I don't have to know exactly how logic gates work to program in JS and make a difference. I assume the same applies to ML. Motivation is what prevents most people from learning difficult to learn skills. Finding ways to produce value early on can help keep you motivated.