alphaXiv: Open research discussion on top of arXiv • Hacker News Board

Re: alphaXiv: Open research discussion on top of arXiv

#161

Post by godelski » Sun, Sep 08, 2024, 11:24 PM UTC

Nice idea but I dislike the implementation. Honestly, I very much like OpenReview and question why we don't use that?

OpenReview has: - preprints - versioning - reviewing, with threads and latex support - ability to link websites, repos, datasets, etc - bibtex generation

But it's not as popular as arxiv, though very popular in review (conferences often do not use full features)

One thing I dislike about this is that it is open to all. Arxiv doesn't have a hard filter (you just need someone with an account to vet you, which stakes their reputation), but the existence of a filter is critical.

I don't want a place to engage with the public. We have Reddit, hacker news, Twitter, mastodon, and countless other places. I want a place for academics to talk to academics [0], researchers to researchers. There is a serious lack of spaces where serious low level in-the-weeds discussions can happen. Even fucking GitHub and hugging face are swamped by people asking dumb questions on research projects like how to install pytorch, fine tune a model, or where the source code is.

I'm really happy to include a lot of people, but I think we also need spaces where experts know they're talking to peers. Without that you have to assume you're not talking to peers because they outnumber us a few thousand to one. So that doesn't encourage engaging in research or technical discussions, it encourages talking down to peers and misinterpreting.

[0] a degree is not what makes you "an academic"

Re: alphaXiv: Open research discussion on top of arXiv

#162

Post by godelski » Mon, Sep 09, 2024, 1:00 AM UTC

throw_pm23 wrote: post #65

amadeuspagel wrote: post #47

Great idea. - The frontpage should directly show the list of papers, like with HN. You shouldn't have to click on "trending" first. (When you are logged in, you see a list of featured papers on the homepage, which isn't as engaging as the "trending" page. Again, compare HN: Same homepage whether you're logged in or not.) - Ranking shouldn't be based on comment activity, which ranks controversial papers, rather papers…

Counterpoint: please don't do any of the above and keep arxiv as it is. It is too valuable to mess it up, it is the few things on the internet that have not been ruined yet, and the "comment activity" can happen in the articles themselves at the scale of years, decades, and centuries.

I actually want the above, BUT I don't want it to be open to everyone. Not all gate keeping is bad.

We don't lack places that the public can engage with researchers and experts. What we do lack are places where researchers/experts can communicate with one another __and expect the other person to be a peer__. The bar to arxiv is (absurdly) low, and I think that's fine.

Not everything has to be for everyone.

My longer comment: https://news.ycombinator.com/item?id=41484123

I'm going to go crazy if I get more GitHub issues asking where the source code is or how to fine tune a model. My research project page is not a Google Search engine nor ChatGPT...

Re: alphaXiv: Open research discussion on top of arXiv

#163

Post by godelski » Mon, Sep 09, 2024, 1:04 AM UTC

gradus_ad wrote: post #99

amadeuspagel wrote: post #47

Great idea. - The frontpage should directly show the list of papers, like with HN. You shouldn't have to click on "trending" first. (When you are logged in, you see a list of featured papers on the homepage, which isn't as engaging as the "trending" page. Again, compare HN: Same homepage whether you're logged in or not.) - Ranking shouldn't be based on comment activity, which ranks controversial papers, rather papers…

> Ranking shouldn't be based on comment activity, which ranks controversial papers But don't we want people's attention drawn to controversial/conversation generating papers? The whole point of the platform is to drive conversation

My guess is that they're trying to deal with a problem they're creating: being open to everyone. The problem with places like Twitter, Reddit, Github, HN, etc is that you don't know you're talking to a peer and the idiots asking irrelevant questions or proposing dumb things outnumber researchers a million to one. Even allowing public to upvote or affect the rankings is not beneficial to science.

I'm all for casting wide nets and making things available to everyone, but a little gate keeping is not bad (just don't gate keep by race, class, or those things). But I'm sorry, research is hard. There's a reason people spend decades researching things that at face value look trivial. Rabbit holes are everywhere and just because you don't know about them doesn't mean your opinion has equal weight.

We seriously lack areas where experts can talk to other experts.

Re: alphaXiv: Open research discussion on top of arXiv

#164

Post by jltsiren » Mon, Sep 09, 2024, 1:07 AM UTC

throw10920 wrote: post #105

impendia wrote: post #90

Earlier quoted context omitted.

> Use HTML rather then PDF. The PDF is the original paper, as it appears on arXiv, so using PDF is natural. In general academics prefer PDF to HTML. In part, this is just because our tooling produces PDFs, so this is easiest. But also, we tend to prefer that the formatting be semi-canonical, so that "the bottom of page 7" or "three lines after Theorem 1.2" are meaningful things to say and ask questions about. That sa…

> In general academics prefer PDF to HTML. In part, this is just because our tooling produces PDFs, so this is easiest. The tooling producing PDF by default absolutely makes the preference for PDF justifiable. However, tooling is driven by usage - if more papers come with rendered HTML (e.g. through Pandoc if necessary), and people start preferring to consume HTML, then tooling support for HTML will improve. > But al…

HTML still lacks one key feature: a way of storing the entire document as a single file that remains fully functional offline and can be reasonably expected to be widely supported for decades. Research papers are used both for communicating new results and for archiving them. The long-term stability needed for the latter has never been a strong point of web technology.

Re: alphaXiv: Open research discussion on top of arXiv

#165

Post by SubiculumCode » Mon, Sep 09, 2024, 1:18 AM UTC

This is cool. Seriously, I mean it. In too many field, the discussion of papers is so ephemeral, and this can really do something cool. ...but... I sure hope that no one submits a fake paper made by chatbots, then use chatbots to discuss the paper on alphaxiv.org ad nauseum to get it trending.

Which means they will. Sciencing is hard, especially because gatekeeping is made necessary to keep the spammers (the ones that flood my email with fake journal offers) at bay.

Re: alphaXiv: Open research discussion on top of arXiv

#166

Post by faangguyindia » Mon, Sep 09, 2024, 1:31 AM UTC

jltsiren wrote: post #164

throw10920 wrote: post #105

Earlier quoted context omitted.

> In general academics prefer PDF to HTML. In part, this is just because our tooling produces PDFs, so this is easiest. The tooling producing PDF by default absolutely makes the preference for PDF justifiable. However, tooling is driven by usage - if more papers come with rendered HTML (e.g. through Pandoc if necessary), and people start preferring to consume HTML, then tooling support for HTML will improve. > But al…

HTML still lacks one key feature: a way of storing the entire document as a single file that remains fully functional offline and can be reasonably expected to be widely supported for decades. Research papers are used both for communicating new results and for archiving them. The long-term stability needed for the latter has never been a strong point of web technology.

You can make an HTML file self-contained by embedding CSS in a `` tag and converting images to Base64, embedding them directly in the `` tag as data URLs. This removes the need for external files, making everything contained within a single HTML file.

Re: alphaXiv: Open research discussion on top of arXiv

#167

Post by godelski » Mon, Sep 09, 2024, 1:33 AM UTC

cgshep wrote: post #102

Tenured prof here. Every paper of mine goes on Arxiv with no exceptions, published under CC BY-NC-ND licenses. Some of us are working hard to overcome the system (e.g. look at the IACR's efforts). Unfortunately, academics are still hindered by institutional inertia; in fact, many prefer the status quo, usually those who rely on prestige over actual quality to advance their careers.

  > Unfortunately, academics are still hindered by institutional inertia

As an ABD this has been a real pain point for me. Maybe I came into academia thinking what mattered most was the research. But now I'm the stereotypical PhD who passionately despises academia for its lack of being academic. I'm happy to have competition, but at the end of the day are we all not on the same team?

How the hell did we create a system where it is the norm that an advisor does not read a thesis, to read papers, to mentor? For that to be the norm among a committee? When I've had issues with getting works getting through review (even when they have high citations due to arxiv) I don't understand why it's acceptable for a response to be "keep trying" instead of "here, I read the paper and reviewer responses, let me help"[0]. It seems inefficient that we throw students into the deep end and watch them sink or learn to swim. I think there'd be a lot fewer dejected PhD students if there was a stronger focus on academics, mentorship, and collaboration over churning out ̶w̶i̶d̶g̶e̶t̶s̶ papers.

I think what pisses me off the most is thinking that research significance and success can be measured __purely__ through metrics like citation counts, H-indices, i10's, awards, etc. I'm not saying those are useless, but that we can evaluate without looking at the content? (as you say, actual quality of work) It's like we learned about Goodhart's Law and decided it was a feature not a bug.

(I know this is not always the case and there are many amazing advisors, but I'd be impressed if someone didn't know this is happening at least somewhere within their department.)

[0] If it takes a village to raise a child, it takes a department to mint a PhD. These types of things should come from committees, not just advisors. Our annual meetings and review shouldn't just be going through the motions.

Re: alphaXiv: Open research discussion on top of arXiv

#168

Post by impendia » Mon, Sep 09, 2024, 1:35 AM UTC

throw10920 wrote: post #105

impendia wrote: post #90

Earlier quoted context omitted.

> Use HTML rather then PDF. The PDF is the original paper, as it appears on arXiv, so using PDF is natural. In general academics prefer PDF to HTML. In part, this is just because our tooling produces PDFs, so this is easiest. But also, we tend to prefer that the formatting be semi-canonical, so that "the bottom of page 7" or "three lines after Theorem 1.2" are meaningful things to say and ask questions about. That sa…

> In general academics prefer PDF to HTML. In part, this is just because our tooling produces PDFs, so this is easiest. The tooling producing PDF by default absolutely makes the preference for PDF justifiable. However, tooling is driven by usage - if more papers come with rendered HTML (e.g. through Pandoc if necessary), and people start preferring to consume HTML, then tooling support for HTML will improve. > But al…

> This would also make it easier to rewrite parts of the paper without having to go back and fix all of your layout-dependent references when the layout shifts.

Just thinking about having to change layout-dependent references, every time I add two sentences to the introduction, gives me a migraine.

I never do anything like this in the paper itself, nor does anyone else that I'm aware of. I'm thinking of informal discussions, where I ask another mathematician about something specific in a paper.

Re: alphaXiv: Open research discussion on top of arXiv

#169

Post by godelski » Mon, Sep 09, 2024, 1:37 AM UTC

michaelmior wrote: post #108

cgshep wrote: post #102

Tenured prof here. Every paper of mine goes on Arxiv with no exceptions, published under CC BY-NC-ND licenses. Some of us are working hard to overcome the system (e.g. look at the IACR's efforts). Unfortunately, academics are still hindered by institutional inertia; in fact, many prefer the status quo, usually those who rely on prestige over actual quality to advance their careers.

> usually those who rely on prestige over actual quality to advance their careers Unfortunately for those of us pre-tenure, it's difficult to balance these as I'm sure you aware. We're evaluated by people who may have the best intentions, but don't work directly in our field. They then determine whether we keep our jobs. It's difficult not to consider prestige as a factor when you know those evaluating you will.

I cannot understand how people think that trying to evaluate people who are pushing the bounds of human knowledge is anything short of "a clusterfuck." I understand the appeal of metrics and prestige, as they provide some signal in an insanely noisy system, but to think they are perfectly aligned is ludicrous. Are we unwilling to just admit that the process is noisy? Is it really not okay that it is (and that we'll probably never escape this aspect)? Because I can't see us doing good science if we're not.

Re: alphaXiv: Open research discussion on top of arXiv

#170

Post by impendia » Mon, Sep 09, 2024, 1:42 AM UTC

jltsiren wrote: post #164

throw10920 wrote: post #105

Earlier quoted context omitted.

> In general academics prefer PDF to HTML. In part, this is just because our tooling produces PDFs, so this is easiest. The tooling producing PDF by default absolutely makes the preference for PDF justifiable. However, tooling is driven by usage - if more papers come with rendered HTML (e.g. through Pandoc if necessary), and people start preferring to consume HTML, then tooling support for HTML will improve. > But al…

HTML still lacks one key feature: a way of storing the entire document as a single file that remains fully functional offline and can be reasonably expected to be widely supported for decades. Research papers are used both for communicating new results and for archiving them. The long-term stability needed for the latter has never been a strong point of web technology.

Indeed, I posted my first paper in 2006. It is still live on the internet in exactly the same format, and I've done absolutely nothing to maintain it.

I'm guessing there are few web pages of any significance which need to stay exactly the same for a long time. Here is one example which I've seen trotted out from time to time on HN:

https://www.dolekemp96.org/main.htm

This is clearly the exception. It seems that maintainers of web pages usually expect that they'll need to maintain and update them for as long as they want them to be accessible, and that's definitely not something I'd care to do for research papers.