Live data from Hacker News

Jupyter notebooks in the IDE: VS Code vs. PyCharm

towardsdatascience.com

51–60 of 62 posts

Re: Jupyter notebooks in the IDE: VS Code vs. PyCharm

#51
post #17

Earlier quoted context omitted.

Frankly I HATE web based notebooks. I do deep learning research, and I write a lot of complex functions to handle pre-processing. And with that there's a lot of debugging I need to do, particularly visual debugging of images. The interactivity of notebooks and the features of an IDE has made my programming experience much better.

I'm curious, what interactivity do you enjoy about notebooks that you find missing in IDEs? I find my situation to be quite the opposite. I love developing/writing code in Pycharm because apart from all the things it excels at (introspection, auto complete, linting, documentation, debugging, etc), it lets me easily prototype things in the interactive python interpreter (usually set to IPython), has an excellent varia…

If I'm being honest, I haven't really mucked around pycharm. My primary hate really is using a chome/firefox/safari browser window for the notebook interactivity.

Personally, I use Atom with Hydrogen backend, but I develop the files in sch a way that they tend to end up as working for both interactivity and for scripts. So say my train_keras_script.py works both to train when it's deployed on our GPU server, and works as my primary debugging script.

I tend to throw most of my results into a .png output files and .csv's, the latter of which I use for R analysis scripts (and yes I know R Studio has markdown, but I'm just stubborn I suppose ).

In terms of IDE bells and whistles, I think #1 is certainly autocomplete. I will admit that I can be somewhat of a crummy speller. When first learning to program, it caused me a lot of headache trying to debug. Simple syntax checking and autocomplete of variables (especially long variable names that are meant to be descriptive) made me really embrace programming professionally.

Re: Jupyter notebooks in the IDE: VS Code vs. PyCharm

#52
post #50

I'm trying to implement these same features in my app, Repla, but without requiring an editor plugin, and instead having a view in a separate app that automatically updates via file-system events when you save. There's some more information here https://repla.app/live-coding.html It's an approach that I think has somes pros and cons relative to integrating these features into an IDE. The main argument for moving them…

I'm not on macOS, so I can't try it right now, but I've bookmarked this and will keep an eye on it. Basically the report/output view of Iodide/Jupyter without the "editor" view? I was really hoping something like this would come along.

Yes, that's exactly right. Great to hear other people are also interested in this approach! The technology I developed has ended up being pretty flexible, you can read about it here (https://blog.repla.app/2020/01/13/what-is-repla/) and here (https://blog.repla.app/2020/01/13/repla-use-cases/) if you're curious. But yeah, that's exactly the use case I developed the technology for. I wanted a browser engine with a plugin system so that I could use to display output of systems like Jupyter and Observable HQ, while still editing the source code in my exiting text editor, the same way I edit other source code. I sort of discovered along the way that this system is also a nice way to run a local development web server, and that was a bit easier to implement, so that's what I've launched with first, the homepage has more info on that (https://repla.app/).

I'd like to support Linux and Windows eventually too, but it's easier to start with supporting just one platform while I'm just testing out whether there's other people interested in the idea.

Re: Jupyter notebooks in the IDE: VS Code vs. PyCharm

#53
post #7
post #3

Anyone else noticed that when editing a notebook in VS Code, one could lose unexecuted edits to a cell with accidental arrow key input (that defocuses the current cell and erases changes)? Too many times I have lost a paragraph worth of Markdown text midst typing, it’s kind of maddening.

Yep - it's very annoying but for me it only happens with markdown cells? So now I generally select all and copy now before leaving a markdown cell if I've edited it. The other issue is the language server seems to get into a funk regularly - the completion stops working as does the display of docstrings so I regularly use the "Developer: reload window" feature to reset everything. Also the plots view is poor - it's b…

Disclaimer: I am the engineering manager for the jupyter support supplied in the Python extension.

I LOVE this feedback! Keep it coming (though entering issues https://github.com/Microsoft/vscode-python would be ideal)

Yes, as conner4312 VS Code is investigating how to better incorporate a juypter notebook experience within it, natively. As has been pointed out here, there are a number of limitations that the python extension notebook support has run into. We have been working closely with the VS Code team to find solutions to these limitations. The custom editor work (https://github.com/microsoft/vscode/issues/77131) is one such mechanism. We plan on rolling out our use of that work soon after it is officially released (sometime in February). However, this will not address all of the problems mentioned in this thread. Those issues will be addressed as native jupyter support arrives in VS Code.

Here's some additional info: - Find/replace. (Personally, this drove me NUTS! Just ask my team! :) ) That being said a simple find (ctrl-f) is avaiable in the current official release. Unfortunately, it does not include replace. While we recognize that this is a severe limitation for some, implementing a full find and replace solution would end up requiring the extension to duplicate most of the functionality provided by VS Code itself. Therefore, we currently planning to piggy back off of that work when it becomes available. If that winds up taking an inordinate amount of time, we will consider further stop gap work.

- The markdown loss issue has been addressed and will be in the February release.

- For the language server problem discussed above, we need more info. I would encourage you to enter a new issue at https://github.com/Microsoft/vscode-python. In the meantime, there are known related issues that will be fixed the Feb release. One in particular, was caused by the backend jupyter server inexplicably taking a long time to return us a full completion list. So when you don't see completion pop up, it may not be that it's broken, it may just be taking a long time. We are working around this issue for the February release. Also, if you're having docstring issues, you may find that turning off the python jedi language service in settings will help.

- Plots view. Yes, it is limited and honestly can cause rendering slowdowns. We are working through ways of addressing both problems. A good solution for supporting "proper" zoom will require adding ipywidget support overall, which is coming soon.

- "Smart environment stuff". I'm not following completely. Is this a general python extension issue or does it have to do with the jupyter kernel selection support we have for notebooks? Again, feel free to enter additional details at: https://github.com/Microsoft/vscode-python

- Supporting global editor extensions like "VI mode", isn't something that we can support without help from VS Code itself. One of the main reason's VS Code is pursuing native notebook support is to be able to handle issues exactly like this.

- Saving the notebook weirdness will be addressed in the February release. We were waiting for the custom editor support I referred to above to arrive.

Hope this is useful information!

Re: Jupyter notebooks in the IDE: VS Code vs. PyCharm

#54

Microsoft have released another IDE with notebook support called Azure Data Studio. It uses the same underlying code base as VS Code but is targeted at data driven tasks such as connecting to databases. Strangely, it uses a different jupyter notebook implementation to VS Code. I found it avoids some of the annoying bugs mentioned in another comment. It also introduces a SQL kernel which I’ve found useful for organisi…

Azure Data Studio is a proper fork of VS Code all up. This is why it was able to work around some of the limitations the Python extension is currently dealing with.

Re: Jupyter notebooks in the IDE: VS Code vs. PyCharm

#56
post #53
post #7

Earlier quoted context omitted.

Yep - it's very annoying but for me it only happens with markdown cells? So now I generally select all and copy now before leaving a markdown cell if I've edited it. The other issue is the language server seems to get into a funk regularly - the completion stops working as does the display of docstrings so I regularly use the "Developer: reload window" feature to reset everything. Also the plots view is poor - it's b…

Disclaimer: I am the engineering manager for the jupyter support supplied in the Python extension. I LOVE this feedback! Keep it coming (though entering issues https://github.com/Microsoft/vscode-python would be ideal) Yes, as conner4312 VS Code is investigating how to better incorporate a juypter notebook experience within it, natively. As has been pointed out here, there are a number of limitations that the python…

> - The markdown loss issue has been addressed and will be in the February release.

Awesome!

> - For the language server problem discussed above...

I don’t think that’s Python-specific, I’ve encountered language server seemingly getting “stuck” in some state for almost any language I’ve used for more than a few minutes in VS Code, including Python, C/C++, Go, Rust, etc. I usually need to reload the window more than once per day.

> - "Smart environment stuff".

Probably talking about the general Python-mode Python selector. One can only choose from a list of detected Pythons; no file system dialogue available, and one can’t even paste in a full path unless it’s in the “sanctioned” list. I’ve cursed on this one too.

Re: Jupyter notebooks in the IDE: VS Code vs. PyCharm

#57
post #53
post #7

Earlier quoted context omitted.

Yep - it's very annoying but for me it only happens with markdown cells? So now I generally select all and copy now before leaving a markdown cell if I've edited it. The other issue is the language server seems to get into a funk regularly - the completion stops working as does the display of docstrings so I regularly use the "Developer: reload window" feature to reset everything. Also the plots view is poor - it's b…

Disclaimer: I am the engineering manager for the jupyter support supplied in the Python extension. I LOVE this feedback! Keep it coming (though entering issues https://github.com/Microsoft/vscode-python would be ideal) Yes, as conner4312 VS Code is investigating how to better incorporate a juypter notebook experience within it, natively. As has been pointed out here, there are a number of limitations that the python…

Great response - I really appreciate the detail and also commend the tone you use. It makes me a little worried that I came across as too negative earlier.

To balance that: I haven't forgotten the "WOW WHAT IS THIS" feeling I experienced the first time opened an ipynb file from vscode when the notebook support kicked in and asking my colleagues "did you know about this?" as I excitedly started exploring the feature. Since then (and I'm a relative vscode noob), I've never been tempted to go back to working with notebooks through a browser. Despite my gripes, the overall experience is still miles better (in my opinion) than accessing notebooks through a browser. It's the future for sure.

And thanks for all those updates - I can't wait for the Feb release - and it's very reassuring that these issues are not only well recogonized but also are being aggressively addressed.

Regarding some of your feedback to the feedback:

- Re. the language server issue - it could well be caused by a slow down. I'll test that hypothesis the next time I experience it.

- The "smart environments stuff" is, as you say, actually a general python extension issue. But actually, I withdraw that complaint. I've just checked and I can point the extension at any python interpreter by editing settings.json - it's just the GUI which restricts you to selecting from a list of discovered interpreters.

Thanks again for your response and your engineering efforts in general.

Re: Jupyter notebooks in the IDE: VS Code vs. PyCharm

#58
post #57
post #53

Earlier quoted context omitted.

Disclaimer: I am the engineering manager for the jupyter support supplied in the Python extension. I LOVE this feedback! Keep it coming (though entering issues https://github.com/Microsoft/vscode-python would be ideal) Yes, as conner4312 VS Code is investigating how to better incorporate a juypter notebook experience within it, natively. As has been pointed out here, there are a number of limitations that the python…

Great response - I really appreciate the detail and also commend the tone you use. It makes me a little worried that I came across as too negative earlier. To balance that: I haven't forgotten the "WOW WHAT IS THIS" feeling I experienced the first time opened an ipynb file from vscode when the notebook support kicked in and asking my colleagues "did you know about this?" as I excitedly started exploring the feature.…

Glad to hear, derriz!

I think environment selection would be considered a bug. I theory any python environment that's on your machine should be selectable in the UI. Could you enter an issue at that same location? I can for you, but I'd prefer that the issue comes directly from an outside user (sometimes we filter bugs based on from where they came).

Re: Jupyter notebooks in the IDE: VS Code vs. PyCharm

#59
post #9

Needs more mention of emacs and org mode... :) I am curious to see the general workflow expanded on. I hesitate to think there is a single correct way to keep a notebook. Seems flexibility is the key feature. And is why text is so well used.

I haven't even gotten into org mode, I just edit Python using emacs and it works great. I can evaluate parts of the code and jump to the interpreter when needed. I keep a matplotlib window open with plt.ion() when running locally, or use plt.savefig() when running remotely and have a qiv window open that automatically detects new images. Works great. The only thing I would like is to be able to use the Python interpr…

To interactively inspect within a function, you could use the debugger with `breakpoint()`. But it's not a regular Python interpreter, which annoys me having to learn the different syntax. You can also embed IPython interactively with:

    import IPython; IPython.embed()
though IPython kinda doesn't clean up after itself, so it's a bit messier (your prompt is messed up afterwards).

Finally, you can:

    import code; code.interact()
The downside of this is there's no history or autocomplete by default.

I have a custom function `embed()` [1] that calls `code.interact()`, but with history and autocomplete configured. This is the closest I got it to looking like a regular Python interpreter at an arbitrary place in the code.

[1] https://pastebin.com/WDuheBfV

Re: Jupyter notebooks in the IDE: VS Code vs. PyCharm

#60

Earlier quoted context omitted.

I haven't even gotten into org mode, I just edit Python using emacs and it works great. I can evaluate parts of the code and jump to the interpreter when needed. I keep a matplotlib window open with plt.ion() when running locally, or use plt.savefig() when running remotely and have a qiv window open that automatically detects new images. Works great. The only thing I would like is to be able to use the Python interpr…

To interactively inspect within a function, you could use the debugger with `breakpoint()`. But it's not a regular Python interpreter, which annoys me having to learn the different syntax. You can also embed IPython interactively with: import IPython; IPython.embed() though IPython kinda doesn't clean up after itself, so it's a bit messier (your prompt is messed up afterwards). Finally, you can: import code; code.int…

Thank you! I didn't know about code.interact, but IPython.embed I have used in the past. My point was more that I would like a way to drop to an IPython prompt when my program crashes, I really don't like the solution of having to modify the code to insert a statement where I think my program is going to crash.

In any case I think there was a way to examine variables from up the stack of a stack trace or something like that.. bit too lazy to look it up right now. Regardless, working with a REPL in emacs is pretty great ;)

I think there is a way to do jupyter from emacs (or at least there was a way to do ipython notebooks), but I haven't used it extensively, just tested it once I think. I guess I'm pretty satisfied with the REPL and haven't found a need to have intermediate output during the running of a program.

What I do like about the matpotlib interactive approach is being able to watch the progress of a loop very easily, which is hard to do from a notebook, although there are some ways, but they are either hacky or require some pretty sophisticated things like custom widgets.

Post reply on HN