Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Most Cited AI Papers in 2022

zeta-alpha.com

Post reply on HN
Print view View original

21–30 of 42 posts

Re: The Most Cited AI Papers in 2022

#21

Post by dadoomer »

agolio wrote: post #15
mtcrawshaw wrote: post #9

Earlier quoted context omitted.

Of course interesting things happen outside of mainstream deep learning. The problem is that sorting by citation count is a terrible way to look broadly at research, because mainstream deep learning is so hyped up that there are just way more people working in that area than in other areas, and with more people come more citations. It's a shame because I think that other areas offer much more interesting and technica…

Disagree, or can you name some recent AI developments outside of DL that are anywhere near as groundbreaking?

As far as I know, the field of robotics sees plenty of innovation with no deep learning involved.

Whether you count robotics as AI is up to debate, I suppose

Re: The Most Cited AI Papers in 2022

#22

Post by nl »

agolio wrote: post #15
mtcrawshaw wrote: post #9

Earlier quoted context omitted.

Of course interesting things happen outside of mainstream deep learning. The problem is that sorting by citation count is a terrible way to look broadly at research, because mainstream deep learning is so hyped up that there are just way more people working in that area than in other areas, and with more people come more citations. It's a shame because I think that other areas offer much more interesting and technica…

Disagree, or can you name some recent AI developments outside of DL that are anywhere near as groundbreaking?

To be clear: I think the hype around deep learning is 100% justified (in fact I think it is underplayed).

But in terms of groundbreaking I do think "Zero-Knowledge Proofs for Machine Learning"[1] from 2020 has the potential to unlock some really revolutionary applications in the sense of "things that were not really possible without it".

[1] https://dl.acm.org/doi/abs/10.1145/3411501.3418608

Re: The Most Cited AI Papers in 2022

#23

Post by choppaface »

anshumankmr wrote: post #18

>An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale > https://arxiv.org/abs/2010.11929 Very interesting. I didn't think of connecting these two dots. Let's see if it can be applied to other CV tasks, like object detection.

see e.g. DETR https://arxiv.org/abs/2005.12872

Re: The Most Cited AI Papers in 2022

#24

Post by melony »

dadoomer wrote: post #21
agolio wrote: post #15

Earlier quoted context omitted.

Disagree, or can you name some recent AI developments outside of DL that are anywhere near as groundbreaking?

As far as I know, the field of robotics sees plenty of innovation with no deep learning involved. Whether you count robotics as AI is up to debate, I suppose

Any real innovation on the mechatronics side? The computer vision and controls are now increasingly dominated by deep learning.

Re: The Most Cited AI Papers in 2022

#26

Post by TaylorAlexander »

panabee wrote: post #6

very useful but flawed. the list biases against papers published later in the year. this is a problem raised by openai and brain researchers: https://twitter.com/_jasonwei/status/1631935794301771777

Also the post conflates citations with impact. Citations is a measure of popularity, but impact is subjective. Which has more impact: stable diffusion or research that can better diagnose cancer? The answer to that could be argued either way.

Re: The Most Cited AI Papers in 2022

#27

Post by zavrel »

tiedieconderoga wrote: post #12
tomohelix wrote: post #10

Incredible how much benefit alphafold has brought. And all of that from a less than 100million parameters model. I might be dumb but could they scale it up and make an alphafold 3 with maybe like 10bln params? Would it be a lot better assuming the same training effort is put into it? If it does, can't biotech companies just go nuts and make a 100bln params internal model and have all the protein structures they want?

Has there been much research into the idea of distributing these large models across many heterogeneous machines? I'm wondering if there could be a path towards a mix of alphafold and folding@home, with donated idle compute resources being used to train/run the models. Designing for that sort of fragmentation could also make it easier to slowly run oversized models on local machines with swapped memory.

There was great work last year on distributed training and inference:

Petals: Collaborative Inference and Fine-tuning of Large Models Alexander Borzunov, Dmitry Baranchuk, Tim Dettmers, Max Ryabinin, Younes Belkada, Artem Chumachenko, Pavel Samygin, Colin Raffel https://arxiv.org/abs/2209.01188

Would be wonderful to use this host this model LLaMA: Open and Efficient Foundation Language Models https://arxiv.org/abs/2302.13971

Re: The Most Cited AI Papers in 2022

#28

Post by zavrel »

firechickenbird wrote: post #25

Very interesting, although I was expecting the Stable Diffusion paper[1] to be one of the most cited in 2022 [1]: High-Resolution Image Synthesis With Latent Diffusion Models, https://arxiv.org/abs/2112.10752

The arXiv version is from 2021 so it is not counted towards 2022.

Re: The Most Cited AI Papers in 2022

#29

Post by zavrel »

elektor wrote: post #4

I'm getting wildly different citation counts for some of the listed AI papers. For example, the paper "ColabFold: making protein folding accessible to all" is listed as having 1162 citations. I'm seeing that it was cited only by 899 publications on Scite: https://scite.ai/reports/colabfold-making-protein-folding-ac... I'm wondering if Google Scholar is overestimating or Scite is underestimating.

We used the Google Scholar counts - and in particular a snapshot of end of february for this ranking. There is no perfect number, but at least this one is generally accepted as reasonable, and public, so easy to verify for everyone.

Re: The Most Cited AI Papers in 2022

#30

Post by zavrel »

panabee wrote: post #6

very useful but flawed. the list biases against papers published later in the year. this is a problem raised by openai and brain researchers: https://twitter.com/_jasonwei/status/1631935794301771777

Citations are of course only a distant proxy for real world impact. What would you recommend as a better early indicator? We also looked at Tweet count, but that seems more biased towards marketing and hype, and in fact does not even correlate very well with citation count.
Post reply on HN