Data analysis is alas a big field. I would say you should assess a few factors:
1) how much basic-ish learning in the data science / statistics / ML areas you expect to be undergoing yourself; the Python ecosystem will probably make this much faster
2) how you expect to scale & productionize your analysis tasks (if at all); in my experience Python is a second-class citizen in the Spark world, not far above Clojure's third-class status, and throughput gains from Clojure's native JVM output may outweigh the relative convenience of the PySpark interface. TensorFlow & TFX's interfaces are basically designed from the ground-up for Python.
3) Which major techniques & corresponding libraries you expect to use (e.g. MCMC/STAN, Pyro, TensorFlow, scipy, scikit-learn). Some of these might rule out one language (more likely eliminating Clojure) or the other.
4) How important data visualization will be for you. This aspect of the work will be much easier & richer in Python than in Clojure.
5) What kind of data transformation & validation you expect to do. If this is largely statistical in nature (e.g. rescaling distributions) it's probably a wash. If this is viz-heavy it'd favor Python. If this involves complicated structured data, I'd recommend Clojure.
Python or Clojure for Data Analysis?
11–20 of 21 posts
Re: Python or Clojure for Data Analysis?
#12Best wishes for the new role! Eventually, it may be a good idea to try both Clojure and Python. Personally I find Clojure's approach towards data very refreshing. It does require an open mind and a mindset different than usual. Eventually, this can bring joy, simplicity and power. This article by Chris Nuernberger nicely explains what it is about: https://cljdoc.org/d/cnuernber/libpython-clj/1.2/doc/so-many... Clojur…
Thanks for the links! I like the idea of a changed mindset. That sort of thing can be useful in and of itself as an educational exercise.
By the way, we are having a Clojure data science public meeting at the end of this month:
https://twitter.com/scicloj/status/1291845872884625408
We will assume some basic knowledge of Clojure, but I guess it may be interesting to join anyway.
Re: Python or Clojure for Data Analysis?
#13Why Clojure? I honestly never heard of anyone using Clojure as data tools. I use Python and Scala. I use Python for mostly small tasks. When I hit large data, I normally use Spark on EMR (PySpark or Scala).
I tried ... I hadn’t used a Lisp before and writing Clojure felt like some kind of mind masturbation. Everything was just sooo much more elegant. Having said that it’s awful for data work since it has close to no supported libraries you might be used to in the python eco system.
Do it just to learn a Lisp ; there is something magical there.
Re: Python or Clojure for Data Analysis?
#14I would 100% go with Python, it has a much bigger tooling and community which makes easier to ask question and to collaborate with people.
Once you're comfortable with it, then it might be worth exploring other languages that are less known to have a (subjective) better software design.
Re: Python or Clojure for Data Analysis?
#15Earlier quoted context omitted.
Thanks for the links! I like the idea of a changed mindset. That sort of thing can be useful in and of itself as an educational exercise.
By the way, we are having a Clojure data science public meeting at the end of this month: https://twitter.com/scicloj/status/1291845872884625408 We will assume some basic knowledge of Clojure, but I guess it may be interesting to join anyway.
Interesting! Appreciate the info
Re: Python or Clojure for Data Analysis?
#16Why Clojure? I honestly never heard of anyone using Clojure as data tools. I use Python and Scala. I use Python for mostly small tasks. When I hit large data, I normally use Spark on EMR (PySpark or Scala).
> I honestly never heard of anyone using Clojure as data tools.
Clojure is one of the most actively-used data analysis languages actually. It is used by many industry for that purpose. Heck even the widely-used Metabase is written in Clojure.
Re: Python or Clojure for Data Analysis?
#17Best wishes for the new role! Eventually, it may be a good idea to try both Clojure and Python. Personally I find Clojure's approach towards data very refreshing. It does require an open mind and a mindset different than usual. Eventually, this can bring joy, simplicity and power. This article by Chris Nuernberger nicely explains what it is about: https://cljdoc.org/d/cnuernber/libpython-clj/1.2/doc/so-many... Clojur…
Also SciCloj meetups: https://www.youtube.com/channel/UCaoZzhNzq-H7YiQczXKuXuw
Re: Python or Clojure for Data Analysis?
#18Why Clojure? I honestly never heard of anyone using Clojure as data tools. I use Python and Scala. I use Python for mostly small tasks. When I hit large data, I normally use Spark on EMR (PySpark or Scala).
> I honestly never heard of anyone using Clojure as data tools. Clojure is one of the most actively-used data analysis languages actually. It is used by many industry for that purpose. Heck even the widely-used Metabase is written in Clojure.
I don’t believe that. What do you have to back that up? I would have thought Python or Scala - been in the data analysis game for a few years now and have never heard Clojure mentioned once except on Hackernews
Re: Python or Clojure for Data Analysis?
#19As for Jupyter notebooks, REPL-driven development in Clojure gives you the same ease (arguably better) of messing around with code while also scaling to serious software dev. Though it's not as nice for sharing with others.
If you're working in an environment where there's a lot of collaboration Clojure might be tough. But if you're actually going to be developing software that relies on data analysis (rather than just doing it as a one off) I think Clojure might be worth considering.
Re: Python or Clojure for Data Analysis?
#20As for Jupyter notebooks, REPL-driven development in Clojure gives you the same ease (arguably better) of messing around with code while also scaling to serious software dev. Though it's not as nice for sharing with others. If you're working in an environment where there's a lot of collaboration Clojure might be tough. But if you're actually going to be developing software that relies on data analysis (rather than ju…
REPL-driven development is wonderful.
Clojure does have notebook solutions which are worth looking into: * https://github.com/clojupyter/clojupyter * https://github.com/jsa-aerial/saite * https://pink-gorilla.github.io (WIP, but is growing fast and going to be magnificent)
Some other projects are trying to connect the notebook idea with the REPL+editor experience. For example: * https://github.com/metasoarous/oz