Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Papers with Code

paperswithcode.com

Post reply on HN
Print view View original

11–20 of 42 posts

Re: Papers with Code

#11

Post by barefeg »

rg111 wrote: post #3

It is weird to see Papers with Code on the front page of HN. This site is the bread and butter of each Research Engineers and Scientists working in Deep Learning. You use the site almost everyday. Advanced learners also use the site regularly. You would just think that "everyone knows" and never think of sharing the site on HN.

I’m curious, how does it fit your daily workflow as an engineer? Is it somewhere where you get the “news” for the day? Or do you use it for getting information relevant to your current work projects?

Re: Papers with Code

#12

Post by 2bitencryption »

Question -

I get that your run-of-the-mill paper saying "Here we present a novel algorithm for xyz" will usually have the algorithm defined in simple psuedo-code, maybe with an implementation in a "real" language as a proof of concept.

But for the many papers describing novel ML models, how does that work? They seem to use images that diagram out the different layers of the model. But is that truly "universal" the way that a psuedo-code algorithm is universal? As in, if the authors use PyTorch (or whatever), can I take the exact model they describe in their paper and apply it in MyFavoriteMLToolkit and achieve similar results?

I guess my question is, what are the "primitives" of papers describing ML models? Is saying "convolutional layer" enough, or do they also describe the dozens of hyper-parameters, etc?

Re: Papers with Code

#13

Post by igorkraw »

barefeg wrote: post #11
rg111 wrote: post #3

It is weird to see Papers with Code on the front page of HN. This site is the bread and butter of each Research Engineers and Scientists working in Deep Learning. You use the site almost everyday. Advanced learners also use the site regularly. You would just think that "everyone knows" and never think of sharing the site on HN.

I’m curious, how does it fit your daily workflow as an engineer? Is it somewhere where you get the “news” for the day? Or do you use it for getting information relevant to your current work projects?

You use it to find the code and data of a paper - since it also lists other implementations - to run additional baselines on Imagenet in order to appease reviewer #3( who has no idea why your paper on convex optimization has nothing to do with this but it's easier to run them than argue with them).

Pre-parenthesis part is dead serious, parenthesis part is slightly hyperbolic due to accumulated trauma with bad reviewers

Re: Papers with Code

#14

Post by criticaltinker »

kettleballroll wrote: post #9
criticaltinker wrote: post #6

Transformer based architectures and unsupervised pre-training are achieving state of the art results across multiple modalities including NLP, CV, speech recognition, genomics, physics etc - so here's my must read list of recent papers on the topics (along with some of my notes). Happy holidays! [1] Attention Is All You Need (2017) https://paperswithcode.com/paper/attention-is-all-you-need Introduced the Transformer…

The ViT paper doesn't make your list?

Good suggestion, it was tough to narrow down the list! Here is a link to the ViT paper in case others are interested [1].

According to the latest ImageNet standings [2], ViT appears to have slipped to second place in Top-1 Accuracy. CoAtNet-7 is the new leader, but only by a slight margin and at the cost of what appears to be a significantly larger model.

[1] Scaling Vision Transformers https://paperswithcode.com/paper/scaling-vision-transformers

[2] https://paperswithcode.com/sota/image-classification-on-imag...

Re: Papers with Code

#15

Post by kettleballroll »

criticaltinker wrote: post #14
kettleballroll wrote: post #9

Earlier quoted context omitted.

The ViT paper doesn't make your list?

Good suggestion, it was tough to narrow down the list! Here is a link to the ViT paper in case others are interested [1]. According to the latest ImageNet standings [2], ViT appears to have slipped to second place in Top-1 Accuracy. CoAtNet-7 is the new leader, but only by a slight margin and at the cost of what appears to be a significantly larger model. [1] Scaling Vision Transformers https://paperswithcode.com/pap…

That isn't the ViT paper, this one is https://paperswithcode.com/paper/an-image-is-worth-16x16-wor...

Re: Papers with Code

#16

Post by zacmps »

2bitencryption wrote: post #12

Question - I get that your run-of-the-mill paper saying "Here we present a novel algorithm for xyz" will usually have the algorithm defined in simple psuedo-code, maybe with an implementation in a "real" language as a proof of concept. But for the many papers describing novel ML models, how does that work? They seem to use images that diagram out the different layers of the model. But is that truly "universal" the wa…

It depends. Usually a paper doesn't have enough room to mention all of the possible choices in preprocessing, architecture, optimiser, etc. You can usually get pretty close with details just in the paper, but it's not always possible.

That's why a large number of journals now have requirements for publishing code and/or pretrained models (if applicable).

An annoying trend I've noticed in a number of SotA ML papers in video classification present multiple models and only publish the exact architecture & weights for the smaller models which are only as-good-as SotA (see tiny video networks, X3D for examples).

Re: Papers with Code

#17

Post by liquidmetal »

2bitencryption wrote: post #12

Question - I get that your run-of-the-mill paper saying "Here we present a novel algorithm for xyz" will usually have the algorithm defined in simple psuedo-code, maybe with an implementation in a "real" language as a proof of concept. But for the many papers describing novel ML models, how does that work? They seem to use images that diagram out the different layers of the model. But is that truly "universal" the wa…

In my experience there are many lesser significant hyperparameters that can impact performance when going from the released code to your personal favorite framework.

Nothing you can't figure out by reading source code of the two frameworks or by reading the documentation closely.

Generally, people don't seem to care about reproducing exact metrics - as long as it is close enough they're happy. You need to dig a bit deeper if you want the full quality.

Re: Papers with Code

#18

Post by motiejus »

I implemented Wang–Müller algorithm, described it, and embedded the code to the pdf, along with tooling how to generate the example diagrams of the paper (and the whole paper). Everything is in the pdf[1].

Arxiv.org won't accept a pdf with attachments though, so only a stripped-down version will come there (once/if I get an endorsement, fingers crossed).

I copied this concept from Joe Armstrong, where he suggested to distribute Erlang modules as PDFs with code files (*.erl) as attachments. "Documentation comes first, and the distribution should prioritize humans".

[1]: See Section A.1 of https://github.com/motiejus/wm/blob/main/mj-msc-full.pdf

Re: Papers with Code

#19

Post by TuringTest »

rg111 wrote: post #3

It is weird to see Papers with Code on the front page of HN. This site is the bread and butter of each Research Engineers and Scientists working in Deep Learning. You use the site almost everyday. Advanced learners also use the site regularly. You would just think that "everyone knows" and never think of sharing the site on HN.

I'm concerned that this "every knows" is increasingly becoming a true social problem, unsolved by current technology - in fact, worsened by it.

Knowledge about a field transfers best by hands-on association with people who practice it. Before widespread IT, communities of practice were local and relatively homogeneous; so it was easy to share the essentials of a field quickly, and get newcomers up and running with best practices.

Nowadays however, communities of practice are widespread, coming around all the world with very different backgrounds, communicating through low-bandwidth channels, and we're flooded with information so it's difficult to ascertain what is essential and what's accessory.

It is much more difficult for an outsider to grasp the essential qualities of a field they want to enter, as there are usually no guides comprehensive enough to detail everything you need to know.

Re: Papers with Code

#20

Post by criticaltinker »

2bitencryption wrote: post #12

Question - I get that your run-of-the-mill paper saying "Here we present a novel algorithm for xyz" will usually have the algorithm defined in simple psuedo-code, maybe with an implementation in a "real" language as a proof of concept. But for the many papers describing novel ML models, how does that work? They seem to use images that diagram out the different layers of the model. But is that truly "universal" the wa…

It's a good question which might yield a very complex answer depending on how far down the rabbit hole of reproducible science/computation/machine learning you're willing to go.

To keep things simple, I'd say the true "primitives" of ML models can be reduced to mathematical formulas. For example, a plain old feed forward network is implemented as matrix multiplication. Sprinkle in a bit of calculus to analytically derive the formula for back-propagating errors (aka training), and you have the basic building blocks of modern deep learning. Convolutions, Transformers, etc are just a bit fancier spins on the same mathematical foundations.

Hyper-parameters are essentially tunable variables in a formula. I'd say your instinct is spot on - they are absolutely necessary to capture for reproducible results.

If you have the code and the data the answer should be yes. You should be able to take that PyTorch code and translate it to MyFavoriteMLToolkit to obtain numerically identical results.

In practice, we face the same universal difficulties as other computer science based research: fighting inconsistencies in software, hardware, all the way down to the physics of the universe with cosmic ray induced bit flips, etc.

Post reply on HN