Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

JupyterLab 4.0

blog.jupyter.org

Post reply on HN
Print view View original

171–180 of 199 posts

Re: JupyterLab 4.0

#171

Post by Camillo »

mikepurvis wrote: post #13

I keep experimenting with Jupyter in the context of telemetry/fault analysis and then hitting a wall with it where: - I get an analysis that I like, but there isn't a good way to share it with others, so I end up just taking screenshots. - There isn't a good way to take the same analysis and plug new data into it, other than to copy-paste the entire notebook. - The process to "promote" fragments of a notebook into be…

> - The process to "promote" fragments of a notebook into being reusable functions seemed very high-friction: basically you're rewriting it as a normal Python package and then adding that to Jupyter's environment.

> - There aren't good boundaries between Jupyter's own Python environment, and that of your notebooks— if you have a dependency which conflicts with one of Jupyter's dependencies, then good luck.

The best Jupyter UX for me now is VSCode. Just put an .ipynb file in your workspace and you get the notebook interface inside VSCode. Put `%load_ext autoreload` and `%autoreload 2` in the first cell, and use the same python environment you're using in your workspace for the Jupyter kernel. Then you can import libraries from your project, use them, and it's very easy to promote code from the notebook into a library. You can just cut a function from the notebook, paste it into a library, add an import, and rerun the subsequent cells to verify it still works as expected.

Re: JupyterLab 4.0

#172

Post by prometheon1 »

henrikgs wrote: post #145
prometheon1 wrote: post #142

Earlier quoted context omitted.

> I get an analysis that I like, but there isn't a good way to share it with others, so I end up just taking screenshots. What would be your preferable way of sharing your analysis with others? - You can turn jupyter notebooks into pdfs directly in the jupyter UI. - You can upload them to Gitlab/Github and share the link to the rendered result. - You can upload them to Colab/Binder/Kaggle and let people play with the…

I'm guessing the las one should link here https://rise.readthedocs.io/en/stable/ and not to voila two times?

You're right, thanks!

Re: JupyterLab 4.0

#173

Post by prometheon1 »

bippingchip wrote: post #156
prometheon1 wrote: post #142

Earlier quoted context omitted.

> I get an analysis that I like, but there isn't a good way to share it with others, so I end up just taking screenshots. What would be your preferable way of sharing your analysis with others? - You can turn jupyter notebooks into pdfs directly in the jupyter UI. - You can upload them to Gitlab/Github and share the link to the rendered result. - You can upload them to Colab/Binder/Kaggle and let people play with the…

While I have no need for its online functionality and the SAAS part of plotly, I really do like plotly python + cufflinks [1]. It lets you make interactive plots in html/js format. Which means you can save the notebook as html, and while people won't be able to rerun the code, they can still zoom in on graphs, hover to see annotations etc, which is a really nice way to share the outcome of your work in a more accessi…

Cufflinks seems to be stale, maybe it is not needed anymore to bind plotly and pandas? I don't think these options existed in 2021 when cufflinks was last updated:

- Plotly can be used directly as pandas backend: https://plotly.com/python/pandas-backend/

- The plotly.express module makes it easy to create interactive plots in html/js format from pandas dataframes: https://plotly.com/python/plotly-express/#gallery

Re: JupyterLab 4.0

#174

Post by prometheon1 »

mikepurvis wrote: post #170
prometheon1 wrote: post #142

Earlier quoted context omitted.

> I get an analysis that I like, but there isn't a good way to share it with others, so I end up just taking screenshots. What would be your preferable way of sharing your analysis with others? - You can turn jupyter notebooks into pdfs directly in the jupyter UI. - You can upload them to Gitlab/Github and share the link to the rendered result. - You can upload them to Colab/Binder/Kaggle and let people play with the…

I think the ideal would be a link directly to the notebook which allows user-local editing/fiddling, with an option to fork the master copy and save it into their own workspace— basically the Github model. But it sounds from both your comment and the many sibling replies that there are a number of tools now directly addressing this space, so I should definitely re-evaluate what is available.

Do you have an example of how this works with another tool/language?

I don't know if I understood it correctly but maybe you could:

- Upload your notebook to Github, then create a url with Binder (part of the jupyter ecosystem) directly to an editing/fiddling playground: https://mybinder.org/

- If by user-local you mean on their own machine, they can clone your repo and run their own jupyterlab to fiddle

- If everything should stay on your own computer/server, you could share a link to your own jupyterlab and collaborate with others in real-time: https://jupyterlab-realtime-collaboration.readthedocs.io/en/... (doing this securely might be a bit of a hassle)

Re: JupyterLab 4.0

#175

Post by wenc »

bramathon wrote: post #155
wenc wrote: post #55

Earlier quoted context omitted.

Would be curious how long ago you tried this and what your setup is? The lowest spec machine I have is a Windows 10 machine with an i7 and 8 GB RAM and it is super responsive on the latest version of VS Code.

A few weeks ago with Ubuntu 22.04 on i7 with 16GB RAM. Perhaps it's related to using the devcontainer feature of VSCode, although I run jupyterlab in the same container

That might be it. I’m not sure about devcontainers (I don’t use them) but as a data point I dual boot into kubuntu on an 7 year old i7 with 8 gigs of RAM and there are no responsive issues.

Re: JupyterLab 4.0

#176

Post by [deleted] »

Jgrubb wrote: post #164
wenc wrote: post #77

Earlier quoted context omitted.

I’ve no idea what the cost is since the company pays for it — I do need the horsepower to run some really large models and I suspect most people don’t need this kind of spec. But for my company it’s just part of the cost of doing business. There’s no IT and I can provision instances of any type (subject to limits) at any time.

Do you turn that lightbulb off at night when you clock out? Just curious.

[deleted]

Re: JupyterLab 4.0

#177

Post by wenc »

zokier wrote: post #149
digdugdirk wrote: post #72

Earlier quoted context omitted.

Really interesting setup. What kind of monthly expense does this run? And on a separate but related note, does it change the way you think about how you spend your time coding? (Assuming the costs do ramp up with usage such that time literally does equal money?)

32 vCPU / 256 GB instances like r6a.8xlarge is about $900/month (r6ad which has local disk is about $100/month more), I don't see there being much other major costs with such setup?

Unlike individuals, large enterprises rarely pay sticker price but a heavily discounted negotiated rate for software and services. I can’t say how much exactly but it’s less than that.

Re: JupyterLab 4.0

#178

Post by fluidcruft »

spywaregorilla wrote: post #167
jey wrote: post #138

Earlier quoted context omitted.

Ok, but let's also acknowledge that the mental model and levels of abstraction increased significantly, which presents a barrier to people without any prior software engineering experience, who are just getting into Python for "data science".

I don't think it's a huge leap to say your functions are stored in this .py file vs. in a cell above. the magic commands simply allow you to reload the functions automatically without needing to reimport / reload the kernel.

It is a huge gap when the notebook is documentation. Moving functions out either duplicates documentation or separates them. Notebooks are vastly superior for documentation. Extracting functions from notebooks into a py file isn't difficult (see nbdev for example)

Re: JupyterLab 4.0

#179

Post by garbagecoder »

Y_Y wrote: post #104
garbagecoder wrote: post #29

Earlier quoted context omitted.

I just do a venv inside the venv so I can target several Python versions. I also use Jupiter for C#, Clojure, and Julia. I wish the Clojure integration was as good as the others I listed. I’m to the point now where if anything other than venv/pip is required I won’t use it. Unfortunately there are many things that insist on conda.

Conda is a fucking nightmare. Luckily you can usually just use pip to get the same packages.

One that you can’t is the Clojure kernel for Jupiter. I installed it from source. That’s no big deal, but they aren’t 1-to-1 with packages.

Re: JupyterLab 4.0

#180

Post by henrydark »

tetris11 wrote: post #111
anvuong wrote: post #23

Earlier quoted context omitted.

This is the way. I use miniconda to create env for Jupyter and install only Jupyter and its dependency on that, then I just config it to point to all the other Python envs I use for my different projects. This is only one-time setup and it is absolutely worth it given how messy the ecosystem is.

> miniconda Try micromamba, it will shave years off dependency resolution

I used to use micromamba, recently moved to mach-nix with nix. It's also really fast, but with the reproducibility of nix.
Post reply on HN