21–30 of 174 posts
Not only did we get a whole new type of Chess engine, it was also interesting to see how the engine thought of different openings at various stages in its training. For instance, the Caro-Kann, which is my weapon of choice, was favored quite heavily by it for several hours and then seemingly rejected (perhaps it even refuted it?!) near the end.
"Overview of SHARD: A System for Highly Available Replicated Data" it's the first paper to introduce the concept of database sharding. It was published in 1988 by the Computer Corporation of America. It is referenced hundreds of times in many classic papers. But, here's the thing. It doesn't exist . Everyone cites Sarin, DeWitt & Rosenb[e|u]rg's paper but none have ever seen it. I've emailed dozens of academics, libr…
Now I’m going to be bugged by this too! Great trivia also a heck of a mystery
"Blue Is the New Black (Market): Privacy Leaks and Re-Victimization from Police-Auctioned Cellphones" https://krebsonsecurity.com/2023/05/re-victimization-from-po... Researchers bought up a bunch of seized phones from police auction sites and found about 25% of them were trivially unlockable and still held sensitive data about suspects and victims .
Is this really "mind blowing"? Knowing the competency of the average police department, I'd consider it more par for the course.
"Overview of SHARD: A System for Highly Available Replicated Data" it's the first paper to introduce the concept of database sharding. It was published in 1988 by the Computer Corporation of America. It is referenced hundreds of times in many classic papers. But, here's the thing. It doesn't exist . Everyone cites Sarin, DeWitt & Rosenb[e|u]rg's paper but none have ever seen it. I've emailed dozens of academics, libr…
Wait, you mean people include papers they haven't even opened in their references?!
Someone managed to GPU-accelerate program synthesis, a form of symbolic ML. First time for ML that is not deep learning: https://dl.acm.org/doi/10.1145/3591274 Deep learning took off precisely when the ImageNet paper dropped around 2010. Before nobody believed that backprop can be GPU-accelerated.
When I was doing my master's in 2004-06, I talked to a guy whose MSc thesis was about running NNs with GPUs. My thought was: you're going to spend a TON of time fiddling with hacky systems code like CUDA, to get basically a minor 2x or 4x improvement in training time, for a type of ML algorithm that wasn't even that useful: in that era the SVM was generally considered to be superior to NNs.
So it wasn't that people thought it couldn't be done, it's that nobody saw why this would be worthwhile. Nobody was going around saying, "IF ONLY we could spend 20x more compute training our NNs, then they would be amazingly powerful".
My understanding of this work: A forward pass for a (fully-connected) layer of a neural network is just a dot product of the layer input with the layer weights, followed by some activation function. Both the input and the weights are vectors of the same, fixed size.
Let's imagine that the discrete values that form these vectors happen to be samples of two different continuous univariate functions. Then we can view the dot product as an approximation to the value of integrating the multiplication of the two continuous functions.
Now instead of storing the weights of our network, we store some values from which we can reconstruct a continuous function, and then sample it where we want (in this case some trainable interpolation nodes, which are convoluted with a cubic kernel). This gives us the option to sample different-sized networks, but they are all performing (an approximation to) the same operation. After training with samples at different resolutions, you can freely pick your network size at inference time.
You can also take pretrained networks, reorder the weights to make the functions as smooth as possible, and then compress the network, by downsampling. In their experiments, the networks lose much less accuracy when being downsampled, compared to common pruning approaches.
Paper: https://openaccess.thecvf.com/content/CVPR2023/papers/Solods...
"Overview of SHARD: A System for Highly Available Replicated Data" it's the first paper to introduce the concept of database sharding. It was published in 1988 by the Computer Corporation of America. It is referenced hundreds of times in many classic papers. But, here's the thing. It doesn't exist . Everyone cites Sarin, DeWitt & Rosenb[e|u]rg's paper but none have ever seen it. I've emailed dozens of academics, libr…
> It is referenced hundreds of times in many classic papers. Wait, you mean people include papers they haven't even opened in their references?!
Gene linked to long COVID found in analysis of thousands of patients https://www.nature.com/articles/d41586-023-02269-2
Surfactants safely take down mosquitoes without using insecticides https://newatlas.com/science/surfactants-safely-take-down-mo...
This is what our Milky Way galaxy looks like when viewed with neutrinos https://arstechnica.com/science/2023/06/ghost-particles-have...
"Overview of SHARD: A System for Highly Available Replicated Data" it's the first paper to introduce the concept of database sharding. It was published in 1988 by the Computer Corporation of America. It is referenced hundreds of times in many classic papers. But, here's the thing. It doesn't exist . Everyone cites Sarin, DeWitt & Rosenb[e|u]rg's paper but none have ever seen it. I've emailed dozens of academics, libr…
Open AI’s como paper, A Holistic Approach to Undesired Content Detection in the Real World.
https://arxiv.org/pdf/2208.03274.pdf
Lots of interesting facts are strewn around the paper.
——-
The first paper that squarely talked about the language resource gap in CS/ML. Before this came out, it was hard to explain just how stark the gap between English and other languages was.
Lost in Translation: Large Language Models in Non-English Content Analysis
https://cdt.org/insights/lost-in-translation-large-language-...
——
This paper gets in for the title:
“I run the world’s largest historical outreach project and it’s on a cesspool of a website.” Moderating a public scholarship site on Reddit: A case study of r/AskHistorians
https://drum.lib.umd.edu/bitstream/handle/1903/25576/CSCW_Pa...
——
This was the first paper I ended up saving on online misinformation. The early attempts to find solutions.
The Spreading of Misinformation online, https://www.pnas.org/doi/10.1073/pnas.1517441113
What I liked here was the illustration of how messages cascade differently based on the networks the message is traveling through.