81–90 of 110 posts
But state is not tracked perfectly. Sometimes you have to manually re-run the cell. For example if one cell defines a dataclass d and another cell changes d.x = "new value". Then other cells using d.x will not know that it has changed.
Both of these solve the most important part of this problems in iPython - horrible git interaction, horrible programming practice to discouraging writing library files, though Jupyter fixes most of the weird non-deterministic behaviour by forcing you to rerun the script every time you load it (rather than reactive techniques). State is OK for power users but it's known to be a massive pain for people who are just learning programming, and an issue in large projects or with interaction.
With this new project having reactive updates I think it's definitely going to be great for beginners, or in gnarly projects.
I wonder if it runs on pyodide (a cPython compiled to run in the browser, with matplotlib and scipy bundled).
This is currently being worked on in Jupyter: https://github.com/jupyterlab/jupyterlab/pull/15448
That's one interesting project. As someone who relies heavily on collaboration with people using Jupyter Notebook. The most annoying points about reproducing their work are the environment and the hidden state of Jupyter Notebooks. This does to address directly the second problem. It does however by sacrificing flexibility. I might need to change a cell just to test a new thing (without affecting the other cells) but…
Yes, the second half of reproducibility is for sure packages. A solution for reproducible environments is on our roadmap ( https://marimo-team.notion.site/The-marimo-roadmap-e5460b9f2... ), but we haven't quite figured it out yet. It's a bit challenging because Python has so many different solutions for package management. If you have any ideas we'd love to hear them.
Earlier quoted context omitted.
Hi Simon, slightly unrelated question. I'm a big fan of your work, and as I've learnt a lot from reading your blog posts over the years, I'd be curious to know a bit more about typical use cases for wanting to work with Observable notebooks. The only reason why I'm using A JavaScript notebook tool (Starboard.gg) is to be able to access cool visualisation packages like Anychart or Highcharts. Given the hype around Obs…
I primarily use Observable to build interactive tools, as opposed to Jupyter which I use more for exploratory development and analysis. Here are some of my Observable notebooks which illustrate the kind of things I use it for: https://observablehq.com/@simonw/search-for-faucets-with-cli... https://observablehq.com/@simonw/openai-clip-in-a-browser Those are both from https://simonwillison.net/2023/Oct/23/embeddings/ h…
That's one interesting project. As someone who relies heavily on collaboration with people using Jupyter Notebook. The most annoying points about reproducing their work are the environment and the hidden state of Jupyter Notebooks. This does to address directly the second problem. It does however by sacrificing flexibility. I might need to change a cell just to test a new thing (without affecting the other cells) but…
It comes with costs and the gpu-related stuff is especially tricky e.g. https://www.canva.dev/blog/engineering/supporting-gpu-accele...
Earlier quoted context omitted.
People always complain about pip and python packaging but it’s never been an issue for me. I create a requirements.base.txt that has the versions of things I want installed. I then: pip freeze -r requirements.base.txt > requirements.txt Install is then simply: pip install -r requirements.txt Updating / installing something new is a matter of adding to the base file and then refreezing.
Poetry handles all of this properly.
Earlier quoted context omitted.
I don’t understand the platform thing, is that something to do with running on Windows? Why wouldn’t you just pip install? Why bring conda etc into the mix? If you have conflicts then you have to reconcile those at point of initial install - pip deals with that for you. I’ve never had a situation in 15 years of Python packages where there wasn’t a working combination of versions. These are genuine questions btw. I se…
I will try to summarize the complaints (mine at least) in obvious simple points 1- pip freeze will miss packages not installed by pip (i.e. Conda). 2- It does include all packages, even not used in the project. 3- It just dumps all packages, their dependencies and sub-dependencies. Even without conflicts, if you happen to change a package, then it is very hard to keep track of dependencies and sub-dependencies that n…
2/3- you need to track only the direct dependencies _manually_ but for reprodicible deployments you need fixed versions for all dependencies. The latter is easy to generate _automatically_ (`pip freeze`, pip-tools, pipenv/poetry/etc).
That's one interesting project. As someone who relies heavily on collaboration with people using Jupyter Notebook. The most annoying points about reproducing their work are the environment and the hidden state of Jupyter Notebooks. This does to address directly the second problem. It does however by sacrificing flexibility. I might need to change a cell just to test a new thing (without affecting the other cells) but…
Yes, the second half of reproducibility is for sure packages. A solution for reproducible environments is on our roadmap ( https://marimo-team.notion.site/The-marimo-roadmap-e5460b9f2... ), but we haven't quite figured it out yet. It's a bit challenging because Python has so many different solutions for package management. If you have any ideas we'd love to hear them.
And even if it’s on some shared storage, it may have been generated by another unreproducible notebook or worse, manually.