Live data from Hacker News

The Engineer’s Guide to Deep Learning: Understanding the Transformer Model

interdb.jp

11–20 of 35 posts

Re: The Engineer’s Guide to Deep Learning: Understanding the Transformer Model

#11
post #7

question to experts of HN in ML/AI. Could you please share the beginner resources you think would worth for a person who wants to switch their domain from CRUD/backend APIs to ML/AI. There seems to be many branches of this domain, not sure where to start. Is my understanding correct? * ML engineer -> engineer who builds ML models with pytorch (or similar frameworks) * AI engineer -> engineer who builds applications o…

Kaggle is a good start

Re: The Engineer’s Guide to Deep Learning: Understanding the Transformer Model

#13

It uses keras, which is obsolete. Nobody uses that thing anymore. Stay away from this.

I wonder about Keras 3. It's now backend independent again, like in the early days, and supports JAX, TensorFlow, or PyTorch. It's a nice thing if you defined your model and can then easily switch between the frameworks, right? Or no-one cares about that, and everyone just uses PyTorch?

Re: The Engineer’s Guide to Deep Learning: Understanding the Transformer Model

#14
> When you send me an email, please provide at least two SNS [social networking service] addresses (e.g. LinkedIn, Twitter) for verification purposes. ... I no longer accept contact from anonymous individuals.

It's pretty sad to see that social networking is being adopted as an identification and trust mechanism even by technical people. It was bad enough when some governments began demanding social networking usernames for visa/immigrant screening, but we can't even send an email without social proof to other technical people now?

Re: The Engineer’s Guide to Deep Learning: Understanding the Transformer Model

#15

It uses keras, which is obsolete. Nobody uses that thing anymore. Stay away from this.

I prefer PyTorch myself, but to call Keras obsolete is quite the stretch. Just because academia has largely moved on from it, doesn't mean nobody uses it.

Also, the API isn't all that different from other libraries. The principles are the same.

Re: The Engineer’s Guide to Deep Learning: Understanding the Transformer Model

#16
post #7

question to experts of HN in ML/AI. Could you please share the beginner resources you think would worth for a person who wants to switch their domain from CRUD/backend APIs to ML/AI. There seems to be many branches of this domain, not sure where to start. Is my understanding correct? * ML engineer -> engineer who builds ML models with pytorch (or similar frameworks) * AI engineer -> engineer who builds applications o…

I'd call the "AI Engineer" an Application Engineer, albeit one that specialises in integrating ML into software.

Re: The Engineer’s Guide to Deep Learning: Understanding the Transformer Model

#17
post #14

> When you send me an email, please provide at least two SNS [social networking service] addresses (e.g. LinkedIn, Twitter) for verification purposes. ... I no longer accept contact from anonymous individuals. It's pretty sad to see that social networking is being adopted as an identification and trust mechanism even by technical people. It was bad enough when some governments began demanding social networking userna…

> I no longer accept contact from anonymous individuals.

This reminds of that joke, where a guy shows up at the Air Force HQ recruitment center. They ask, "Pilot license? Experience? Qualifications?" He replies, "Nope, just here to say: Don't count on me!"

Re: The Engineer’s Guide to Deep Learning: Understanding the Transformer Model

#18
post #10

There are many others that are better. 1/ The Annotated Transformer Attention is All You Need http://nlp.seas.harvard.edu/annotated-transformer/ 2/ Transformers from Scratch https://e2eml.school/transformers.html 3/ Andrej Karpathy has really good series of intros: https://karpathy.ai/zero-to-hero.html Let's build GPT: from scratch, in code, spelled out. https://www.youtube.com/watch?v=kCc8FmEb1nY GPT with Andrej Kar…

In addition, these websites are totally free.

The website listed here:

> I consider requests for full commercial use of all content on this site (and the github repository). For a complete buyout of all content rights, the cost is €10,000,000. > I’d like to ask you what problems you have by that I keep on having the copyright of my document.

+ no commercial-use without paying 20% royalty.

So fairly expensive for a Keras tutorial.

Re: The Engineer’s Guide to Deep Learning: Understanding the Transformer Model

#19
post #7

question to experts of HN in ML/AI. Could you please share the beginner resources you think would worth for a person who wants to switch their domain from CRUD/backend APIs to ML/AI. There seems to be many branches of this domain, not sure where to start. Is my understanding correct? * ML engineer -> engineer who builds ML models with pytorch (or similar frameworks) * AI engineer -> engineer who builds applications o…

85% of your ML project time will be spent on Data Quality and a little bit of Domain Feature Engineering.

If you want to make an impact, become excellent at those, you will be able to use these skills, for domains like Systems Integration and Business Analytics. Let the people who do Research bring you the Algorithms and nowadays even the trained Models.

Re: The Engineer’s Guide to Deep Learning: Understanding the Transformer Model

#20
post #15

It uses keras, which is obsolete. Nobody uses that thing anymore. Stay away from this.

I prefer PyTorch myself, but to call Keras obsolete is quite the stretch. Just because academia has largely moved on from it, doesn't mean nobody uses it. Also, the API isn't all that different from other libraries. The principles are the same.

The industry has also moved on from keras/tensorflow (apart from for legacy reasons).

Google itself has moved on to JAX.

Post reply on HN