Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Distill: a modern machine learning journal

distill.pub

Post reply on HN
Print view View original

91–100 of 109 posts

Re: Distill: a modern machine learning journal

#91

Post by kriro »

_delirium wrote: post #17
curuinor wrote: post #4

Earlier quoted context omitted.

As I said in Rob's thingy, I hope you get the tenure committees and job committees, because they don't have to respect it but they're the ones you have to get to respect

It varies heavily by institution and country, but CS is moving increasingly towards caring about citation metrics above anything else (with "selectivity", i.e. a high bar for peer review and low acceptance rate, being the main other factor). Unlike in most other fields, conference papers therefore hold weight, not only journal articles. This does sometimes cause trouble at higher levels of large institutions, where a…

Agree with a lot of this. If Google wanted they could probably even give some extra boost to forms of publication they endorse. I'd love to see Open Access weighted higher in Scholar and they could add an extra boost for "interactive examples" or "available data sets". I think you're spot on that they hold quite a bit of power (high GS ranking is also an incredible citation boost for the typical "tack on citations").

Re: Distill: a modern machine learning journal

#92

Post by reachtarunhere »

j2kun wrote: post #51
colah3 wrote: post #22

Earlier quoted context omitted.

You're absolutely right that this is a lot of work, and not many ML researchers have all the skills needed for it. In the short term, Distill's editorial assistance will help authors produce outstanding papers, although they need to be willing to work as well. In the longer-term, I'd like to explore match making between data visualization people who would like to get into machine learning and machine learning researc…

I left a comment on your blog announcement to this effect, but I'd love to be a "research distiller" :)

You already are. :) Love your blog.

Re: Distill: a modern machine learning journal

#93

Post by IanCal »

bpicolo wrote: post #73
colah3 wrote: post #26

Earlier quoted context omitted.

I think that having a venue that can publish non-traditional academic artifacts is an important step for reproducibility, even if it isn't our focus. It seems clear to me that the future will involve some kind of linking reproducibility to papers. If we want to find that future, we need a way for people to experiment with what a publication is.

Jupyter notebooks are a big piece of solving ML reproducability, it feels like.

I see this a lot, but I disagree, at least in their current form. They miss a variety of very key parts for reproducibility (which, to be fair, was not their original goal).

* Dependencies like libraries are not specified anywhere.

* Dependencies on local code are not bundled.

* Dependencies on local data are not bundled.

* Underlying requirements like LLVM (which needs to be specifically 3.9.X for llvmlite in python as I discovered recently).

* Perhaps most dangerously, you can run the code sections out of order, and deleted sections will leave their variables around which can interfere with the run. I've been caught out by this in my own notebooks.

I really like jupyter notebooks, but I think some of the design decisions (correct for some ways of working) actively work against reproducible reports.

There was a recent writeup here:

> we were able to successfully execute only one of the ~25 notebooks that we downloaded.

https://markwoodbridge.com/2017/03/05/jupyter-reproducible-s...

Re: Distill: a modern machine learning journal

#94

Post by fnl »

colah3 wrote: post #78
fnl wrote: post #39

How does this provide IF ratings? Probably irrelevant for industry, but publishing in academia is all about IF, no matter how bad and corrupt one might think it is. And what about long-term stability/presence. Most top journals and their publishing houses (NPG, Elsevier, Springer) are likely to hang around for another decade (or two...), while I don't feel so sure about that for a product like GitHub. Maybe Distill i…

Our present JIF is undefined because we haven't existed for two years yet. If you just apply the formulas anyways, you'll get an JIF of (6 citations)/(4 publications) = 1.5. Again, this number is really pessimistic because those publications are only a few months old and haven't had time to accumulate citations. > And what about long-term stability/presence. We aren't particularly tied to github besides it being conv…

I for one am not so convinced GitHub is likely to be around for another decade or two. But whatever, let's just pretend that Distill can always find a free hosting solution, that is not so unlikely. Maybe that's good enough?

Re. IF, sorry if my first post wasn't as as obvious as I thought it would be. I wasn't referring to how IF is calculated, much less to Distill's current IF. Rather, there are two big problems related to IF that Distill needs to "solve"; Not the how, but rather then when and who of IF:

Ad when: The egg and the hen problem. As colah3 wrote, Distill's IF will only become meaningful in two years. But if you have exciting research, you want that to be in an high-impact journal/venue now. So attracting good research as a new journal/venue is extremely difficult, and probably the one main reason why new journals fail (c.f. the number of new journals/venues and the mostly non-existent change in impact rankings of the "best" places to publish). However, if you can get private researchers in industry to publish in Distill, because they are not [so] "dependent" on IF, you might accumulate sufficient impact in the first two years to get to a nice score, that later makes Distill competitive to the various IEEE journals or JMLR.

Ad who: The even worse problem that (at least European, not sure about US) universities evaluate their researchers by looking up their Web of Science ranking/score. WoS in turn is controlled by Thomson Reuters (TR), who also decide which journals get ranked in WoS (and sell access to WoS to universities and governments - n/c...). If a journal is not "recognized" by WoS, the publication or its citations do not get counted by TR. Ergo, as a public researcher, your funding dries up and/or you don't get the promotions you need. For that reason alone, no researcher in public research will allow her/his students and postdocs to publish in a journal that is not indexed by TR/WoS. But again, you might get around that by behaving "like" arXiv at first, at least: Most journals now grudgingly accept that the work was first on arXiv before it got published in some high-impact journal or venue. And maybe there is even a chance that the publishing industry will have to accept Distill in their midst (i.e., index it in WoS) if some other industrial backers create enough pressure...

As might be clear from the above, I (and many researchers) am (are) fed up with the current publishing system, so I certainly hope a "self-hosted", free solution controlled by the public [researchers] one day will break the iron first the current (private) publishing houses exert over how research is managed and evaluated today. If Distill manages to keep itself independent from industry, but at the same time can use the political weight its current backing could bring, maybe this is a way to break this vicious cycle?

Re: Distill: a modern machine learning journal

#95

Post by IanCal »

shancarter wrote: post #21
cing wrote: post #19

Is there any concern about a web-native journal being less "future-proof"? I've come across quite a few interactive learning demonstrations in Flash/Java that no longer work.

This is a high-priority for us. By focusing on web-standards and avoiding proprietary plugins we're pretty confident that the content will be future-proof.

Something that could help is perhaps a choice that examples should work in (e.g.) Firefox recent.x on ubuntu, then provide a VM and archived version of firefox. Put it on a platform that archives things with C/LOCKSS and get a doi, then although you're not expecting people to use it on a daily basis, it'd cover several "worst case" kind of scenarios.

Of course that's not completely permanent, but would perhaps provide some more safety.

Re: Distill: a modern machine learning journal

#97

Post by eb0la »

j2kun wrote: post #18

I sure hope this catches on, but we should all be aware of the hurdles: - Little incentive for researchers to do this beyond their own good will. - Most ML researchers are bad writers, and it's unlikely that the editing team will do the work needed (which is often a larger reorganization of a paper and ideas) to improve clarity. - Producing great writing and clear, interactive figures, and managing an ongoing github…

I'm not in academia, but I guess the impact (citations) you could get with a distill-like paper will be higher than the ones you get on a traditional paper-based journal.

So, I guess this will get distill get traction.

Re: Distill: a modern machine learning journal

#98

Post by lhnz »

colah3 wrote: post #22
j2kun wrote: post #18

I sure hope this catches on, but we should all be aware of the hurdles: - Little incentive for researchers to do this beyond their own good will. - Most ML researchers are bad writers, and it's unlikely that the editing team will do the work needed (which is often a larger reorganization of a paper and ideas) to improve clarity. - Producing great writing and clear, interactive figures, and managing an ongoing github…

You're absolutely right that this is a lot of work, and not many ML researchers have all the skills needed for it. In the short term, Distill's editorial assistance will help authors produce outstanding papers, although they need to be willing to work as well. In the longer-term, I'd like to explore match making between data visualization people who would like to get into machine learning and machine learning researc…

  > In the longer-term, I'd like to explore match making
  > between data visualization people who would like to
  > get into machine learning and machine learning
  > researchers publishing papers.
I'm into data viz and interested in doing this. I'm currently plowing through the Fast.AI course, and was actually already considering creating visualisations to help test my thinking.

Re: Distill: a modern machine learning journal

#100

Post by ycHammer »

Would saving jupyter notebooks as .html work? PS: I have published in all of top-4 tier ML conferences but sk at html/css/js. What is my pathway to distill now? I, like every other researcher worth her/his name in salt is always running behind clock when it comes to deadlines and lit to review. So, yeah? Coaxing myself into investing time for css/html/js in lieu of picking up more math tools seems criminal to me. Am I alone in this ?
Post reply on HN