My collection of machine learning paper notes • Hacker News Board

Re: My collection of machine learning paper notes

#31

Post by yobibyte » Thu, Apr 22, 2021, 8:16 AM UTC

activatedgeek wrote: post #26

I think these notes are great, and Vitaly certainly seems like a great person from Twitter (been following for a while now). I just want to spell out the obvious - the biggest (and probably the only) beneficiary of such structured notes is the note-maker. The beginners who come in feeling excited that this will be a great learning resource are probably missing the point. Learning happens when you force yourself to cr…

> the biggest (and probably the only) beneficiary of such structured notes is the note-maker.

I would probably agree with the "biggest", but disagree with the "only": * The readers might use a note as an extended abstract when selecting a paper to read. This is like a short conference talk which is for advertising the paper and inviting people to the poster session. * The authors get feedback about their research, and some of them engage in a discussion as well.

Having said that, I agree that taking your own notes is better for you.

Re: My collection of machine learning paper notes

#32

Post by yobibyte » Thu, Apr 22, 2021, 8:25 AM UTC

albertzeyer wrote: post #30

yobibyte wrote: post #22

Earlier quoted context omitted.

Hi Albert! Long time no see =) I'll probably slow down at some point, but I think atm reading stuff gives me more ideas or general understanding what I want to work on and what not. There are drawbacks as well since some papers have a lot of time to read in depth, and a day is def not enough to get a proper understanding. Re your advice, that's a great point! How do you select a papers outside of your comfort zone?

Hm that's difficult. Automatic speech recognition (ASR) is probably by now my comfort zone. So already most pure DL papers are out of this zone, but I anyway many of them, when I find them interesting. Although I tend to find it a bit boring when you just adopt next-great-model (e.g. Transformer, or whatever comes next) to ASR, but most improvements in ASR are just due to that. You know, I'm also interested in all th…

Thanks for such an extended reply!

It's probably hard to estimate an impact of reading outside of your field, but this definitely sounds like a good idea. A positive bonus here is that you get more exposure to how people write and talk about research in different areas, and I find it super useful. I've recently read about Curry-Howard correspondence (https://en.wikipedia.org/wiki/Curry%E2%80%93Howard_correspon...), and it was mind-blowing both in terms of what they talk about and how they talk about it.

On the negative side, it's often quite hard to understand only because the terminology is different.

Re Neural Turing Machines, there's been an interesting resurgence of the field working on algorithmic tasks (check out this amazing survey https://arxiv.org/abs/2102.09544).

Re: My collection of machine learning paper notes

#33

Post by marwahaha » Thu, Apr 22, 2021, 10:43 AM UTC

yobibyte wrote: post #23

marwahaha wrote: post #18

I really like this. I recently started a similar project called the "arXiv wiki". Could you link future paper notes here? For example: https://arxiv.wiki/abs/2101.06861

Nice! Re linking, writing a bot scraping the links and adding PRs would be a great thing to have =)

Sure, I built one. I think it will work if you keep your paper collection page in the same format.

https://github.com/arxivwiki/kurin-paper-scraper https://github.com/arxivwiki/arxivwiki/blob/main/.github/wor... https://github.com/arxivwiki/arxivwiki/blob/main/.github/wor...

Re: My collection of machine learning paper notes

#34

Post by anon_tor_12345 » Thu, Apr 22, 2021, 1:31 PM UTC

activatedgeek wrote: post #29

anon_tor_12345 wrote: post #28

Earlier quoted context omitted.

>beneficiary of such structured notes is the note-maker. the same is true of most textbooks as well; most people write textbooks for themselves (and then publish them in order to not nothing to show for a year of work). i saw that somewhere and it's changed the way i approach reading textbooks (no longer do i take it for granted that one presentation is /the/ presentation).

I don't quite agree with this reduction. A good reference textbook is specifically designed to convey a clean linear story of the otherwise ugly conceptual development of research ideas. Notes are personal. Textbooks are a deliberate transform of those notes meant to convey structure in ideas to the average person in the target audience. I find it funny that someone would go through the pain of undertaking an endeavo…

>A good reference textbook is specifically designed to convey a clean linear story of the otherwise ugly conceptual development of research ideas.

Keyword: good. I said most and I stand by that: most textbooks suck and serve only to order the concepts in a way that makes sense to the author.

Re: My collection of machine learning paper notes

#35

Post by lobocinza » Fri, Apr 23, 2021, 12:20 AM UTC

activatedgeek wrote: post #29

anon_tor_12345 wrote: post #28

Earlier quoted context omitted.

>beneficiary of such structured notes is the note-maker. the same is true of most textbooks as well; most people write textbooks for themselves (and then publish them in order to not nothing to show for a year of work). i saw that somewhere and it's changed the way i approach reading textbooks (no longer do i take it for granted that one presentation is /the/ presentation).

I don't quite agree with this reduction. A good reference textbook is specifically designed to convey a clean linear story of the otherwise ugly conceptual development of research ideas. Notes are personal. Textbooks are a deliberate transform of those notes meant to convey structure in ideas to the average person in the target audience. I find it funny that someone would go through the pain of undertaking an endeavo…

It is a gradient. You can have notes where the author took that effort to great length and textbooks where it didn't.