Live data from Hacker News

Ask HN: What's the best paper you've read in 2020?

news.ycombinator.com

191–197 of 197 posts

Re: Ask HN: What's the best paper you've read in 2020?

#191
post #128

Not a paper, but a fantastic talk by Samy Bengio "Towards better understanding generalisation in deep learning" at ASRU2019. Some pretty mind blowing insights - ex: if you replace one layer's weights in a trained classification network with the initialisation weights for the layer (or some intermediate checkpoint as well), many networks show relatively unaffected performance for certain layers ... which is seen as a…

thats not surprising to me: if we view the weights in the whole network as "individual cells" in a population, and if we pretend that at each update of network weights, before the update each weight undergoes cell division such that one daughter cell / weight is an increase in weight, and the other a decrease, then each component of the gradient descent vector can be viewed as the fitness function for that specific cell or weight: an increase or a decrease. From this perspective each cell forms its own niche in the ecosystem, and its no surprise that replacing a cell with its ancestor is roughly compatible with the final network cells: the symbiosis goes both ways.

The reason for Bengio demonstrating this on a complete layer is obviously to demonstrate that this is NOT due to redundantly routing information to the next layer (think holographically, for robustness). And using non-ancestor random weights illustrates that the ecosystem fitness suffers if redundant / holographic routing is prevented while also using non ancestral cells / weights...

Re: Ask HN: What's the best paper you've read in 2020?

#192
Difficult to choose one this year (plenty of things happened + plenty of time to read things due to lock down).

"Equality of Opportunity in Supervised Learning" (https://arxiv.org/abs/1610.02413)

It explain the basic concept about fairness in ML. Very practical exemple in my domain knowledge that show the trade-off between fairness of an algo and overall performance (money). Really make you see what may go wrong with bias in ML. It shows, in my opinion, why we will have to regulate ML as corporation aren't really incentivized to deal with fairness. It also shows that there is different notions of fairness. So there will always be something that feel unfair and also doing something can always be interpreted as positive discrimination.

Re: Ask HN: What's the best paper you've read in 2020?

#193
This one is from 2008. There's this method from statistics called PCA that let's you reduce high dimensional data into a few (usually meaningless) newly-fabricated dimensions. It's useful to visualize complex data in 2d space.

In this paper, they did that with genes. And the 2d space that was left wasn't meaningless at all. It accurately recreated map of Europe.

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2735096/

Re: Ask HN: What's the best paper you've read in 2020?

#194

What's Wrong with Computational Notebooks? Pain Points, Needs, and Design Opportunities https://web.eecs.utk.edu/~azh/pubs/Chattopadhyay2020CHI_Note...

Aren't you one of the authors of that paper? It was a good read. Have a look at https://iko.ai, it solves not only many of the pain points you wrote about, but the "critical" pain points (difficult and important).

- No-setup collaborative notebooks: near real-time editing on the same notebook. Large Docker images.

- Long-running notebook scheduling: you can schedule a notebook right from the JupyterLab interface and continue to work on your notebook without context switch. The notebook will run and you can view its state even if you close your browser or get disconnected, and you can view it without opening the JupyterLab interface, even on your mobile phone. https://iko.ai/docs/notebook/#long-running-notebooks

- Automatic experiment tracking: iko.ai automatically detects parameters, metrics, and models and saves them. Users don't have to remember or know how to write boilerplate code or experiment tracking code.

- One click parametrization: you can publish an AppBook to enable other people to run your automatically parametrized notebook without being overwhelmed by the interface. You don't have to use cell tags or metadata to specify parameters. You click a button and an application is created from your notebook. The runs of this application are also logged as experiments in case they generate a better model. https://iko.ai/docs/appbook/

- Easy deployment: people can look at the leaderboard, and click on a button to deploy the model they choose into a "REST API" endpoint. They'll be able to invoke it with a POST request or from a form where they simply upload or enter data and get predictions.

We haven't focused on the stylesheets given that in our previous projects with actual paying enterprise customers, it wasn't CSS that held us back.

Here's an invite link that's valid multiple times: https://iko.ai/invite/lEMzE_hKwJ2SUbfLdnK7SbZb1c3zUCOAQexakL...

Re: Ask HN: What's the best paper you've read in 2020?

#195
post #181

A paper that profoundly influenced my language design: “Programming with Polymorphic Variants” https://caml.inria.fr/pub/papers/garrigue-polymorphic_varian... And the earlier paper “A Polymorphic Type System for Extensible Records and Variants” https://web.cecs.pdx.edu/~mpj/pubs/96-3.pdf Row types are magically good: they serve either records or variants (aka sum types aka enums) equally well and both polymorphically…

> (...) my programming language. https://www.unisonweb.org/ ?

Not that, but Unison is one direct inspiration. I wrote a small overview here https://community.inflex.io/t/the-inflex-language/20

Re: Ask HN: What's the best paper you've read in 2020?

#196
post #29

Here's a wonderful one I read a little over a year ago: "Estimating the number of unseen species: A bird in the hand is worth log(n) in the bush" https://arxiv.org/abs/1511.07428 https://www.pnas.org/content/113/47/13283 It deals with the classic, and wonderful, question of "If I go and catch 100 birds, and they're from 20 different species, how many species are left uncaught?" There's more one can say about that tha…

This sounds like a similar problem to an exercise in the famous probability textbook by William Feller about estimating the total number of fish in a lake by catching N of them and tagging them, and then throwing them back in the lake and coming back later to catch another N fish. You check how many of those fish are tagged and derive your estimate from that using the hypergeometric distribution. See the pages here:…

I appreciate the footnote that basically says "Whoa, this is more practically useful than we realized!"
Post reply on HN