I’m starting a new role that will be heavy on data analysis. In the past I’ve focused on web dev/CRUD apps using Go, JS, PHP, other web stuff. All that to say, this will be new territory for me. Conventional wisdom seems to be that Python is the tool of choice for data science type roles, along with pandas/numpy and Jupyter notebooks. That or maybe R or Julia. On the other hand I hear about how Clojure is great for data munging type tasks. I have no experience with lisps but I’m interested in dipping my toe in the water if it will be a good fit for what I’m going to be doing. Which one in your experience will pay off more in the long run? Python with its tooling and community or Clojure’s language design?
Python or Clojure for Data Analysis?
1–10 of 21 posts
Re: Python or Clojure for Data Analysis?
#2Hi,
Congratulations on your new role. Are you joining a team, or are you the team? If you're joining a team, then you'll probably use what they're using and learn their tooling before you could endeavor to improve it.
You're doing it in a professional context, so it will be Python. Many blog posts and articles on popular medium websites address shiny new things, but most of these posts address one of two scenarios: portfolio/toy projects, a project with one individual working on it, a project with data that fits on disk and RAM, and/or a Kaggle project where a good part of the heavy lifting has been done for you (data acquisition, cleaning, feature engineering, metric identification) which never happens in real life because that's what you're hired for in the first place.
A big problem in this field is the fragmented tooling and experience, which means you have to weave tools together, unless the team you're joining has it figured out and have internal tooling dialed in. Python dominates. I'm sure other languages are used at other ML shops (we have used Scala in some of our projects) but I think in your situation, there's no need to complicate things.
Then again, that is just an opinion. It is not the right answer. The goal is to deliver value.
All the best,
Re: Python or Clojure for Data Analysis?
#3Hi, Congratulations on your new role. Are you joining a team, or are you the team? If you're joining a team, then you'll probably use what they're using and learn their tooling before you could endeavor to improve it. You're doing it in a professional context, so it will be Python. Many blog posts and articles on popular medium websites address shiny new things, but most of these posts address one of two scenarios: p…
Thanks for the insight! I am the team--everything before was done quite manually with spreadsheets. I already work in the organization but have been promoted out of development into this new role so I'm blessed to have a large amount of flexibility. It does seem like Python is the right tool for the job. Your perspective confirms my hunch.
Re: Python or Clojure for Data Analysis?
#4Best wishes for the new role!
Eventually, it may be a good idea to try both Clojure and Python.
Personally I find Clojure's approach towards data very refreshing. It does require an open mind and a mindset different than usual. Eventually, this can bring joy, simplicity and power.
This article by Chris Nuernberger nicely explains what it is about: https://cljdoc.org/d/cnuernber/libpython-clj/1.2/doc/so-many...
Clojure's community is certainly smaller than Python's, but some say it is very friendly.
Below are some beginner-friendly places to chat about it. If you wish, let us chat there, dive into the details, and think how you could begin exploring.
Clojurians Zulip https://clojurians.zulipchat.com and especially the data-science stream: https://clojurians.zulipchat.com/#narrow/stream/151924-data-...
Clojureverse https://clojureverse.org
Re: Python or Clojure for Data Analysis?
#5Why Clojure? I honestly never heard of anyone using Clojure as data tools.
I use Python and Scala. I use Python for mostly small tasks. When I hit large data, I normally use Spark on EMR (PySpark or Scala).
Re: Python or Clojure for Data Analysis?
#6Why Clojure? I honestly never heard of anyone using Clojure as data tools. I use Python and Scala. I use Python for mostly small tasks. When I hit large data, I normally use Spark on EMR (PySpark or Scala).
Honestly? I’m interested in it and wanted an excuse to give it a try. No more compelling reason than that.
Re: Python or Clojure for Data Analysis?
#7Best wishes for the new role! Eventually, it may be a good idea to try both Clojure and Python. Personally I find Clojure's approach towards data very refreshing. It does require an open mind and a mindset different than usual. Eventually, this can bring joy, simplicity and power. This article by Chris Nuernberger nicely explains what it is about: https://cljdoc.org/d/cnuernber/libpython-clj/1.2/doc/so-many... Clojur…
Thanks for the links! I like the idea of a changed mindset. That sort of thing can be useful in and of itself as an educational exercise.
Re: Python or Clojure for Data Analysis?
#8Hi, Congratulations on your new role. Are you joining a team, or are you the team? If you're joining a team, then you'll probably use what they're using and learn their tooling before you could endeavor to improve it. You're doing it in a professional context, so it will be Python. Many blog posts and articles on popular medium websites address shiny new things, but most of these posts address one of two scenarios: p…
Thanks for the insight! I am the team--everything before was done quite manually with spreadsheets. I already work in the organization but have been promoted out of development into this new role so I'm blessed to have a large amount of flexibility. It does seem like Python is the right tool for the job. Your perspective confirms my hunch.
> I am the team--everything before was done quite manually with spreadsheets.
In that case I would pick between Python and R. R might even win out slightly over Python for your use case. Definitely not Clojure, Scala or even Julia.
Re: Python or Clojure for Data Analysis?
#9Why Clojure? I honestly never heard of anyone using Clojure as data tools. I use Python and Scala. I use Python for mostly small tasks. When I hit large data, I normally use Spark on EMR (PySpark or Scala).
Honestly? I’m interested in it and wanted an excuse to give it a try. No more compelling reason than that.
I think that's the best reason for doing things. Find something interesting and give it a try.
Using Scala as a functional language has been a fun journey for me. I still code imperatively when solving problems, but when I get a chance to refactor them into functional programming, boy, my mind goes Boom!
I do hope I get to use Clojure at some point.
Re: Python or Clojure for Data Analysis?
#10If you want to learn something new and have cycles to burn on that: Clojure. It’s a great language, but learning it is going to be a slower and more scenic route.
If you want to get things done: Python. You’ll have no problem getting up to speed based on your past experience, and the ecosystem is orders of magnitude larger than Clojure.