Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MIT 6.S191: Recurrent Neural Networks, Transformers, and Attention [video]

youtube.com

Post reply on HN
Print view View original

51–60 of 63 posts

Re: MIT 6.S191: Recurrent Neural Networks, Transformers, and Attention [video]

#51

Post by credit_guy »

sudosysgen wrote: post #48
credit_guy wrote: post #47

Earlier quoted context omitted.

Ukraine has great fighters and great spirit. But fighters and spirit can take you only that far. Without the Javelins and Himars, Ukraine would have collapsed in a matter of weeks. Without a doubt some of the tech that goes in those Javelins and Himars was invented at MIT. This is not whataboutism. This is how you stop tyranny.

The weapons you're talking about killed over a million people in tyrannical wars and terrorism. They are weapons of tyranny.

I understand that you'd like to live in a world where everyone loved each other and there are no weapons, and people walk around carrying roses and tulips.

That world does not exist. The world we are living in is a world where Putin exists. A world where a disarmed US and Western Europe would be conquered in less time than it would take you to sing Kumbaya.

I prefer to live in a world where MIT helps the US military complex come up with better missiles than a world where Putin kills millions of men, women and children in the name of "denazification".

Re: MIT 6.S191: Recurrent Neural Networks, Transformers, and Attention [video]

#52

Post by kalimanzaro »

tysam_and wrote: post #41
matt_langston wrote: post #4

I wonder why that parameter is called "h"? Hmmm ... Why not just say the word "hysteresis" and bring some magnets to class for show-and-tell to help your students develop an intuition for the "h" parameter in RNNs.

Like jimsimmons said below, I believe it traditionally refers to 'hidden'. which was in vogue at the time for both feedforward nets and RNNs as well as any other other neural networks in the 90's or so and on. This trend actually continued for a while and I learned it in one of Hintons' main online classes which was made somewhere between 2012-2015 or so IIRC (though I opted to switch to reading and trying to impleme…

Hmm i read a tonne of RNN lit before 2020 and 'd never come across the term "hysteresis parameter" standing in for the hidden units. is it a recent trend? Google seem to suggest so

Re: MIT 6.S191: Recurrent Neural Networks, Transformers, and Attention [video]

#53

Post by woodson »

matt_langston wrote: post #50
matt_langston wrote: post #4

I wonder why that parameter is called "h"? Hmmm ... Why not just say the word "hysteresis" and bring some magnets to class for show-and-tell to help your students develop an intuition for the "h" parameter in RNNs.

As defined by the lecturer herself, "h(t) = h(t-1)", the very definition of hysteresis. My point is that the lecturer missed a golden opportunity to give her students a natural intuition of "h" that they can see, feel and touch and that will serve them well for their entire careers. The only thing "hidden" about "h" is that hysteresis is hidden in plain site in her lecture - maybe the lecturer did not know herself. N…

Back in the day, having taken some kind of statistical signal processing course would have been common before getting into neural networks. That would likely have covered a lot of intuitions.

Re: MIT 6.S191: Recurrent Neural Networks, Transformers, and Attention [video]

#54

Post by [deleted] »

credit_guy wrote: post #51
sudosysgen wrote: post #48

Earlier quoted context omitted.

The weapons you're talking about killed over a million people in tyrannical wars and terrorism. They are weapons of tyranny.

I understand that you'd like to live in a world where everyone loved each other and there are no weapons, and people walk around carrying roses and tulips. That world does not exist. The world we are living in is a world where Putin exists. A world where a disarmed US and Western Europe would be conquered in less time than it would take you to sing Kumbaya. I prefer to live in a world where MIT helps the US military…

[deleted]

Re: MIT 6.S191: Recurrent Neural Networks, Transformers, and Attention [video]

#55

Post by TylerLives »

credit_guy wrote: post #47
sudosysgen wrote: post #23

Earlier quoted context omitted.

Just because other militaries are bad doesn't make the US military good. It's perfectly analogous in its wars of agression, and supporting it is morally bad, no matter how hard you whatabout it.

Ukraine has great fighters and great spirit. But fighters and spirit can take you only that far. Without the Javelins and Himars, Ukraine would have collapsed in a matter of weeks. Without a doubt some of the tech that goes in those Javelins and Himars was invented at MIT. This is not whataboutism. This is how you stop tyranny.

Do you think the outcome of the war matters when evaluating the actions of the international community in Ukraine? For example, if millions of people die in Ukraine as a consequence of war that was only possible because NATO decided to arm Ukrainians, will it be justified, because they were fighting for democracy? What if Ukraine loses anyway?

Re: MIT 6.S191: Recurrent Neural Networks, Transformers, and Attention [video]

#56

Post by ramraj07 »

tysam_and wrote: post #42
ramraj07 wrote: post #13

Which is the best course or set of videos to learn the basics of neural networks and deep learning? Something that really gets the best explanation of things like backprop?

Karpathy's zero to hero series is excellent, and I really recommend it. I also made a few repos that are geared around readability and being a good 'working code demonstration' of certain best-practices in neural networks. If you're like me and you grok code better than symbols, this could be a helpful adjunct as well if you're wanting to dig deep a bit. https://github.com/tysam-code/hlb-CIFAR10 (convolutional networ…

Thanks, will check these out for sure. Already digging Karpathys code first approach in the first few hours!

Re: MIT 6.S191: Recurrent Neural Networks, Transformers, and Attention [video]

#57

Post by derangedHorse »

ramraj07 wrote: post #13

Which is the best course or set of videos to learn the basics of neural networks and deep learning? Something that really gets the best explanation of things like backprop?

Andrew Ng's machine learning course on Coursera is still my favorite introduction to neural networks

Re: MIT 6.S191: Recurrent Neural Networks, Transformers, and Attention [video]

#58

Post by sudosysgen »

credit_guy wrote: post #51
sudosysgen wrote: post #48

Earlier quoted context omitted.

The weapons you're talking about killed over a million people in tyrannical wars and terrorism. They are weapons of tyranny.

I understand that you'd like to live in a world where everyone loved each other and there are no weapons, and people walk around carrying roses and tulips. That world does not exist. The world we are living in is a world where Putin exists. A world where a disarmed US and Western Europe would be conquered in less time than it would take you to sing Kumbaya. I prefer to live in a world where MIT helps the US military…

I'm telling you that your argument if it was valid would justify the people at the Moscow Institute of Technology (MIT) making weapons for the Russian military because they were used by {choose country criminally invaded by the US} to defend themselves against invaders.

Your position is hypocritical unless you think that it's morally good to make weapons for Putin.

Re: MIT 6.S191: Recurrent Neural Networks, Transformers, and Attention [video]

#59

Post by tysam_and »

kalimanzaro wrote: post #52
tysam_and wrote: post #41

Earlier quoted context omitted.

Like jimsimmons said below, I believe it traditionally refers to 'hidden'. which was in vogue at the time for both feedforward nets and RNNs as well as any other other neural networks in the 90's or so and on. This trend actually continued for a while and I learned it in one of Hintons' main online classes which was made somewhere between 2012-2015 or so IIRC (though I opted to switch to reading and trying to impleme…

Hmm i read a tonne of RNN lit before 2020 and 'd never come across the term "hysteresis parameter" standing in for the hidden units. is it a recent trend? Google seem to suggest so

I didn't mention anything at all about a hysteresis parameter.

Re: MIT 6.S191: Recurrent Neural Networks, Transformers, and Attention [video]

#60

Post by lupire »

matt_langston wrote: post #50
matt_langston wrote: post #4

I wonder why that parameter is called "h"? Hmmm ... Why not just say the word "hysteresis" and bring some magnets to class for show-and-tell to help your students develop an intuition for the "h" parameter in RNNs.

As defined by the lecturer herself, "h(t) = h(t-1)", the very definition of hysteresis. My point is that the lecturer missed a golden opportunity to give her students a natural intuition of "h" that they can see, feel and touch and that will serve them well for their entire careers. The only thing "hidden" about "h" is that hysteresis is hidden in plain site in her lecture - maybe the lecturer did not know herself. N…

> As defined by the lecturer herself, "h(t) = h(t-1)", the very definition of hysteresis.

How is that a definition of hysteresis?

Hysteresis is when state is a function of previous state, not identical to previous state.

Post reply on HN