Data science tips and tricks from the developer community
blog.algorithmia.com
Data science tips and tricks from the developer community
1–10 of 69 posts
Re: Data science tips and tricks from the developer community
#2It's worth nothing that the article does not discuss the reproducibility of results (e.g. with a Jupyter Notebook) and the implementation of said results (e.g. deploying/validating models), both of which matter much more than any code style conventions for data-related projects.
Re: Data science tips and tricks from the developer community
#3The advice doesn't look bad but the click-bait title really inclined me to skip it, especially since it meant opening with a digression about whether 10x developers even exist rather than the actual content.
Re: Data science tips and tricks from the developer community
#4Is writing docstrings with argument types a thing in Python? If so, wouldn't these developers benefit from using actual type annotations (or a language with static types)? This is one area where types actually help a great deal with rapid prototyping!
Also, I disagree with the Scala examples and the argument against brevity, but I guess this is the stuff of flamewars. Not only do I not find his more verbose examples any clearer, they also lack context: presumably the full snippet looks like this:
allClothesCount.sortBy(-_._2)
Re: Data science tips and tricks from the developer community
#5It's worth nothing that the article does not discuss the reproducibility of results (e.g. with a Jupyter Notebook) and the implementation of said results (e.g. deploying/validating models), both of which matter much more than any code style conventions for data-related projects.
This 100%. I have read postmortems of some "significant discoveries" which have turned out to only be reproducible on a particular build or software on a single analyst's machine. Or not at all. One "result" turned out to hinge on the iteration order of python dictionaries.
And this definitely didn't happen after those working on tools to help said analysts make reproducible results encouraged the analysts to use said tools... No, that would be crazy.
Re: Data science tips and tricks from the developer community
#6One big takeaway from the article is how easy Algorithmia makes Data Scientists and Teams in productionising Data Science Models which is still a bigger challenge with DevOps and Data Engineers scratching their heads with the model output from Data Scientists.
Re: Data science tips and tricks from the developer community
#7Is writing docstrings with argument types a thing in Python? If so, wouldn't these developers benefit from using actual type annotations (or a language with static types)? This is one area where types actually help a great deal with rapid prototyping! Also, I disagree with the Scala examples and the argument against brevity, but I guess this is the stuff of flamewars. Not only do I not find his more verbose examples…
Argument types in Python docstrings predate type hinting. They're part of a de facto standard started by NumPy.
Re: Data science tips and tricks from the developer community
#8It's worth nothing that the article does not discuss the reproducibility of results (e.g. with a Jupyter Notebook) and the implementation of said results (e.g. deploying/validating models), both of which matter much more than any code style conventions for data-related projects.
I completely agree. Almost all of this article appears to have little to do with being a Data Scientist in particular and more to do with some good practices for writing code in general. So the advice itself is fine, just not what I was hoping for based on the title.
Reproducibility, like you say, however, is something that is an issue far more particular to data science, and worth more serious consideration and discussion. Hand-in-hand with that is shareability. I'm a fan of what airbnb has open sourced to address some of those issues in their knowledge repo project: https://github.com/airbnb/knowledge-repo
Re: Data science tips and tricks from the developer community
#9Is writing docstrings with argument types a thing in Python? If so, wouldn't these developers benefit from using actual type annotations (or a language with static types)? This is one area where types actually help a great deal with rapid prototyping! Also, I disagree with the Scala examples and the argument against brevity, but I guess this is the stuff of flamewars. Not only do I not find his more verbose examples…
Argument types in Python docstrings predate type hinting. They're part of a de facto standard started by NumPy.
Fair enough. Shouldn't the advice be updated to use type hinting then? It seems like a terrible standard these days.
Re: Data science tips and tricks from the developer community
#10It's worth nothing that the article does not discuss the reproducibility of results (e.g. with a Jupyter Notebook) and the implementation of said results (e.g. deploying/validating models), both of which matter much more than any code style conventions for data-related projects.
^ this.
I cannot describe how many times I've been shown results and when asking how to reproduce them, after several notes (and sometimes complaints to higher ups) I eventually get a series of command line arguments or a barely functioning R-script.
These conclusions are too important to be so sloppily produced. We need verification, validation and uncertainty quantification for any result provided to decision makers.