71–80 of 363 posts
Also, include basic instructions for running your code.
I helped my wife with a replication study that should have been straightforward, and I was unable to get the code running after about a week. I don’t necessarily believe the research was suspect, but broken code does draw more suspicion.
I've posted a huge amount of academic code (I've linked to a small number at the end). I think you should, but it won't help advance your career immediately. However, I still think it's better for science. What is useful is if you can produce code people can build on and do their own cool stuff with -- then they will cite you. However, getting something to a state where it is tested for all reasonable inputs, has som…
Thanks, agreed. Small note: it is not clear what Minion is doing, from just visiting the github repo. Perhaps add "C++ constraint solver" in the github description, but it is still unclear: it could be a rigid body constraint solver for games? Maybe add a link to a paper?
In practice Minion is generally used as a backend to Conjure ( https://conjure.readthedocs.io/en/latest/ ), which provides a much nicer input language.
Agree with other comments on CRAPL, but you should release it.
Earlier quoted context omitted.
Not sure if posting the paper only is even the best move. I personally never work with papers with no code published. Just not worth the effort to reproduce them, when I can use the second SOTA for nearly no performance penalty and much less effort. All the groundbreaking papers in deep learning in the last decade had code published. So if you're aiming for thousands of citations, you need code.
Can confirm. I personally look at any paper without code with great suspicion. The reviewers certainly did not try to reproduce your results, and I have no guarantee that a paper without code has enough information for me to reproduce. I always go for the papers with code provided.
Is your goal to help advance the science and our general knowledge? Publish the code. You don’t even need to clean it up. Just publish. Don’t worry about coding style nitpicks. Having the code and data available actually protects you from claims of fabrication or unseen errors in hidden parts of your research. On the other hand, if your goal is only to advance your own career and you want to inhibit others from opera…
Not sure if posting the paper only is even the best move. I personally never work with papers with no code published. Just not worth the effort to reproduce them, when I can use the second SOTA for nearly no performance penalty and much less effort. All the groundbreaking papers in deep learning in the last decade had code published. So if you're aiming for thousands of citations, you need code.
I am in this field and I would say less than 10% of the top papers have code published by the author, and those are most of the time another 0.1% improvement in imagenet. All the libraries that you generally use are likely to be recreated by others in this field. Lot of most interesting work's code never come out like alphazero/muzero, GPT-3 etc.
I suggest publishing the code as is on something such as Github, Gitlab, etc. I suspect you have ideas on how you can improve the code, perhaps there's even a way of improving your research methodology by doing so, enabling new insights with further research. If you did a follow up experiment with improved analysis enabled by your improved code, then that's another paper, and another (more cleaned up) version of the code to push to the repository.
The above is all supposition though, as I don't know your field. If deep learning then the above seems more likely. If your field is geology, then improvements in the software might not enable better insights.
Frankly any paper which can't offer the basics of reproducibility is adding to the current problems across many fields.
Real-World data may restricted by copyright so be careful if this applies. If it does consider publishing with some MC data demonstrating how things are supposed to work. (You did verify your code behaviour didn't you?)
Don't clean and organise code for publishing. It is a tool, it is not 100% perfect, but it is supposed to work. Unfortunately after years in the field sometimes the correct response to nit-picking is "I don't care".
This is the trap between "writing code that was intended to give an answer" and "code that was intended to be re-used by others". Scientists often write code that fits into the former and this code should be published (in-case of mistakes and in the interests of reproducibility). But this code should never be taken to be of the quality that it should be built on by others unless this was the express intent. People who mistake it for that haven't understood the point of the work the author is engaging in.
With regard to what license, I tend to use DWTFYWWI, or just GPL, but frankly you can pick some wonderfully closed thing if you think your code might revolutionise something which in principle stops commercial entities ripping it off directly.