Zeppelin[1] is another great tool of a similar nature. It leans a little bit more towards the Scala / Spark world, for people who like that stack. That said, you can use Python, R, etc. with Zeppelin as well.
Why Jupyter is data scientists’ computational notebook of choice
31–40 of 308 posts
Re: Why Jupyter is data scientists’ computational notebook of choice
#32The only thing that stops me from being able to use notebooks full time is their intellisense compared to IDEs is horrible. I like being able to use them for demos/presentations, but I can't imagine trying to code within one primarily. Especially when it comes to tracking results. How do people cope with this? Do you supplement it with other tools? I spend a lot of my time in an IDE and then just paste some of the co…
It's not about writing code as much as it is about exploring the data. If you're writing a lot of code in them, it's probably better to put that code into libraries that get imported and reused. And I do agree that default code environment is unbearable. Particularly the auto insertion of completing quotation marks, which has me continually fighting with the editor to get correct code into a tiny web text box.
Oh I won't argue you with you there. I just find myself rotating quite a bit because I have to do both deployment as well as writing code for experimentations.
What I'm specifically talking about is even that kinda hacky experiment code you end up writing. I don't try to implement whole projects in there, but even just "train this model" type code ends up being a hassle because of how bad the editors are.
My above comment was more referencing wishing I could spend more time writing experiment code in jupyter without copying and pasting all the time.
Re: Why Jupyter is data scientists’ computational notebook of choice
#33I recently got a Jupyter Notebook, and found it's a large JSON document, with some sections in markdown and some in Python. A browser could omit the Python, and an interpreter could omit the markdown. Would this work for other languages? Maybe JavaScript or Powershell instead of Python?
You may be interested in jupytext
https://github.com/mwouts/jupytext
Re: Why Jupyter is data scientists’ computational notebook of choice
#34The only thing that stops me from being able to use notebooks full time is their intellisense compared to IDEs is horrible. I like being able to use them for demos/presentations, but I can't imagine trying to code within one primarily. Especially when it comes to tracking results. How do people cope with this? Do you supplement it with other tools? I spend a lot of my time in an IDE and then just paste some of the co…
If you miss intellisense, you can try datalore ( https://datalore.io/ ). P.S. Disclaimer: I lead this project at JetBrains, Inc.
Already a customer, you have nothing you can sell me :).
Re: Why Jupyter is data scientists’ computational notebook of choice
#35Here are the issues with Jupyter, and most other flavor, of notebook:
1. variables have to be explicitly output
The most important tool for programming, for me, is that window that shows you the current state of all the variables. When I step through a program, I look at the state. 90% of my debugging solutions come from seeing that variable doesn't have the right state.
2. Intellisense
For the love of god, I do not want to remember if it is len(), length(), .len(), .length(), .size(), size(1) or whatever.
That's it. But those two are so big that I have to code and debug in Spyder and then paste the code into notebook. I feel sorry for people who are new who think that all the debugging is happening in the notebook.
Re: Why Jupyter is data scientists’ computational notebook of choice
#36The only thing that stops me from being able to use notebooks full time is their intellisense compared to IDEs is horrible. I like being able to use them for demos/presentations, but I can't imagine trying to code within one primarily. Especially when it comes to tracking results. How do people cope with this? Do you supplement it with other tools? I spend a lot of my time in an IDE and then just paste some of the co…
I do the opposite, my job is kind of bad data engineer/scientist/etl minion so it's a lot of dataframes. Work (and often debug) in jupyter -> open the notebook from pycharm when it's got some completed thoughts and write into a python module + test module, tidying up and adding type annotations. Sometimes doing that multiple times so that the notebook is importing from modules which were originally pulled out of the…
Data frame rendering in the various notebooks (beaker,jupyter,zeppelin,..) is wonderful.
Your workflow sounds closest to what I do. If I want to visualize something I tend to compile my thoughts/imports and organize things in an editor first and put it in a notebook in parallel. It helps with version control as well.
Re: Why Jupyter is data scientists’ computational notebook of choice
#37The only thing that stops me from being able to use notebooks full time is their intellisense compared to IDEs is horrible. I like being able to use them for demos/presentations, but I can't imagine trying to code within one primarily. Especially when it comes to tracking results. How do people cope with this? Do you supplement it with other tools? I spend a lot of my time in an IDE and then just paste some of the co…
diving code between models/data-pipelines and experiments. Notebook are used for visualization and telling a story why you tried what for the other team-members.
Yeah but the whole point is "interactive coding". It doesn't feel very interactive when I have to context switch all the time :). I'd prefer something closer to what the lisp folks get to do with the repl where you can scratch out an idea and see it working without leaving your environment.
Re: Why Jupyter is data scientists’ computational notebook of choice
#38For people who prefer to code in JS. There is a similar application called observable notebooks that has recently come out: https://beta.observablehq.com/ It offers some nifty things including, well , observables where cells of the scratchpad can automatically update by observing changes from other cells.
Thx for the pointer. Seems this is related to Mike Bostock the guy behind d3.js, will definitely look into it.
Re: Why Jupyter is data scientists’ computational notebook of choice
#39I like R for many things, but Python just keeps getting more compelling, particularly given the excellent machine learning packages. As these sorts of toolchain elements get better and better, and as more people realize that there's a benefit to simultaneously training researchers to run code as well as stats, I suspect we'll start to see an exodus from pure R solutions. The real question is when (and whether) new so…
The migration between languages is also industry specific. They are still teaching SAS to finance and healthcare analysts, for instance, and R and Python are still rising in healthcare specifically. Keep in mind all the legacy code and all the coders who just know SAS and don't need to change. It'll take longer for the transition than you think.
Re: Why Jupyter is data scientists’ computational notebook of choice
#40For people who prefer to code in JS. There is a similar application called observable notebooks that has recently come out: https://beta.observablehq.com/ It offers some nifty things including, well , observables where cells of the scratchpad can automatically update by observing changes from other cells.
More of an online service, though.