Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: How to Break into AI Engineering

news.ycombinator.com

Post reply on HN
Print view View original

21–30 of 69 posts

Re: Ask HN: How to Break into AI Engineering

#21

Post by __rito__ »

- Have crystal clear Mathematical foundations, as in why this formula/method the way it is, rather than being able to solve college/HS test problems. Really solid footing in Differential Calculus and Linear Algebra is necessary.

- Know the Statistical language that you learn from a basic college-level Stat 101 course. Be able to translate normal sentences into those using Statistical notation, and be able to read easily. Also, know basic Statistics.

- You already know programming, I assume. Learn Python if you don't know already. It's really easy.

- There are a number of paths you can go from there. Here's what I did.

-- IBM Data Science Professional Certificate (not deep at all, but lays out the landscape well; did it in a week)

-- Machine Learning for Absolute Beginners by Oliver Theobald which you can finish in an evening.

-- Machine Learning Specialization by Andrew Ng on Coursera.

-- Deep Learning Specialization by Andrew Ng on Coursera.

-- fast.ai course.

- Learn PyTorch really well. I suggest Sebastian Raschka's book.

Now from here, you can chart your own path. You can choose NLProc, Vision, RL, or something else.

I went towards Vision. And I do Edge AI as hobby.

I was in the last year of college as a Physics undergrad, when I was hired to do Vision modelling/research for a non-flashy company in 2021. Finishing my CS Master's next month and starting to look for PhD. I worked in the same company for the ~2.5 years.

EDIT: If you want a job in big tech, grind Leetcode, and learn about system design, study Machine Learning systems, and be able to design them. Chip Huyen has a good book as I hear. 6-7 rounds of interview is common in Meta/Google. DL hackathon awards, open source contributions are significantly helpful.

Re: Ask HN: How to Break into AI Engineering

#22

Post by mathisfun123 »

hcks wrote: post #14
nikhizzle wrote: post #3

Assuming you have the math and algorithmic background, I would start by reading the “attention is all you need” paper. After reading, attempt to build a baby transformer model in PyTorch. After that, consider constructing some of the building blocks without libraries to understand how they work.

I read this exact advice often here on HN and I can’t help but wonder. Is the person writing it just repeating something they read? Is it just because they like the ´coding from first principles’ aesthetics? I mean let’s imagine that someone does read that paper, and manage to replicate the code (quite an effort from someone coming from outside AI and academia). Then what? I doubt it’s particularly illuminating. That…

> Is the person writing it just repeating something they read? Is it just because they like the ´coding from first principles’ aesthetics?

of course. you know how i know? absolutely no one except the wannabees has time to read papers - people working in the area have deadlines and meetings. we absorb the content of the paper by osmosis - convos, code bases, occasionally a talk at a conference.

it's especially horrible advice from the perspective of pedagogy to tell a n00b to read a paper (so the person giving the advice has immediately disqualified themselves from possibly being an expert) because papers are horribly written, omit critical details, and function purely as advertising for the authors, group, etc.

for every poor undergrad/n00b soul reading this comment, take this thing to heart that took me too long to unlearn (due to its constant perpetuation by people like gop): reading the paper is 100:1 waste-of-time:value-derived.

if i hear about/see some paper that makes strong claims that are relevant to my work, i will look for a github link and/or email the first author. 5/10 i get a response (ratio is going up as i'm getting to be more ingratiated in my community). the other times i just move on - none of these papers have some revolutionary cure for cancer in them so most of the time what i'm already doing is already close enough that i don't need to kill myself figuring out the new thing.

that paper in particular (attention) has nothing in it that is in the least bit interesting/revolutionary. the hard part of attention isn't writing down softmax(QK^T)V, the hard part is executing that matrix product fast enough that you're not waiting eons for your model to converge.

Re: Ask HN: How to Break into AI Engineering

#23

Post by electrondood »

71a54xd wrote: post #4

Has anyone here transitioned into an ai dev adjacent role, akin to something a bit more involved than "prompt engineering" potentially the "product eng" equivalent of AI?

That's me, currently. At a small startup and no one else seemed to be as interested in LLMs, so they made me the guy. Still a dev, deploying various models, integrating with our services, instrumentation, prompt management, etc.

Basically architecting LLM-related infrastructure to enable the product features they want, while managing expectations.

Re: Ask HN: How to Break into AI Engineering

#25

Post by tjpnz »

itake wrote: post #10
nikhizzle wrote: post #3

Assuming you have the math and algorithmic background, I would start by reading the “attention is all you need” paper. After reading, attempt to build a baby transformer model in PyTorch. After that, consider constructing some of the building blocks without libraries to understand how they work.

I worry that even if self-taught, I wouldn't have the credentials (job experience or degree), to do a full-time ML job. Our my concerns unfounded?

ML is a relatively new field. How do you think current engineers got their start?

Re: Ask HN: How to Break into AI Engineering

#26

Post by spmurrayzzz »

nikhizzle wrote: post #15
hcks wrote: post #14

Earlier quoted context omitted.

I read this exact advice often here on HN and I can’t help but wonder. Is the person writing it just repeating something they read? Is it just because they like the ´coding from first principles’ aesthetics? I mean let’s imagine that someone does read that paper, and manage to replicate the code (quite an effort from someone coming from outside AI and academia). Then what? I doubt it’s particularly illuminating. That…

The goal here is to actually understand the mechanics of the model and begin the process of intuitively understanding the space. I would put the effort here at a few weekends of focus. Also will add the models turned out to be a lot simpler to understand than I expected going in.

If developing intuitions is the goal, I really do think Jay Alammar's "Illustrated Transformer" [1] is at least a step function better in the service of that outcome than the academic paper itself.

(I totally realize this is subjective, but that has been my experience with my own learning in the space over the last few years as well as some folks I've mentored)

http://jalammar.github.io/illustrated-transformer/

Re: Ask HN: How to Break into AI Engineering

#27

Post by TrackerFF »

nikhizzle wrote: post #3

Assuming you have the math and algorithmic background, I would start by reading the “attention is all you need” paper. After reading, attempt to build a baby transformer model in PyTorch. After that, consider constructing some of the building blocks without libraries to understand how they work.

Just a quick reminder to everyone reading that AI / ML (let's face it, it's 0.1% AI and 99.9% ML) is still a ton more than just SOTA Deep Learning models. Depending on where you work, it could be all classical machine learning methods, and zero deep learning - or the other way around.

Having a broad enough understanding in ML would be a good starting point, along with solid SW engineering skills.

Re: Ask HN: How to Break into AI Engineering

#28

Post by thebigwinning »

hcks wrote: post #14
nikhizzle wrote: post #3

Assuming you have the math and algorithmic background, I would start by reading the “attention is all you need” paper. After reading, attempt to build a baby transformer model in PyTorch. After that, consider constructing some of the building blocks without libraries to understand how they work.

I read this exact advice often here on HN and I can’t help but wonder. Is the person writing it just repeating something they read? Is it just because they like the ´coding from first principles’ aesthetics? I mean let’s imagine that someone does read that paper, and manage to replicate the code (quite an effort from someone coming from outside AI and academia). Then what? I doubt it’s particularly illuminating. That…

> I doubt it’s particularly illuminating.

Everytime I do a foundational activity like this it does turn out to be illuminating. Why do you doubt you'll learn something?

Post reply on HN