Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Lessons learned reinventing the Python notebook

marimo.io

Post reply on HN
Print view View original

31–40 of 44 posts

Re: Lessons learned reinventing the Python notebook

#31

Post by biorach »

tiledjinn wrote: post #29
dan-g wrote: post #11

Earlier quoted context omitted.

I’m not sure what your definition of “failed” is, but notebooks of various kinds have been bringing value and accessibility to various different programming languages for a while now. If it was a “failed concept” I don’t think we’d see as much adoption, let alone new iterations and advancements in the space. Just because you don’t use something doesn’t mean it’s not worth something to someone else (or in this case, a…

inferior mechanisms frequently gain popularity, despite whether people who adopt them would be better off if they didn't. people shouldn't use measuring cups either; since scales are more accurate, faster, and involve less cleanup. and yet.

> inferior mechanisms frequently gain popularity

then they haven't "failed"

> people shouldn't use measuring cups either; since scales are more accurate

low accuracy is fine in many cooking applications. you're being pedantic

Re: Lessons learned reinventing the Python notebook

#35

Post by taeric »

akshayka wrote: post #24
taeric wrote: post #16

I find it so weird to see Jupyter being the "traditional" option in this discussion. Kudos for the shout out to excel being the main "notebook" most people have experience with. I question the choice of Python as the format. Agreed that Json is terrible. But Python will have similar problems. And a notebook is a marked up document. Using a markup language is a natural choice. Delimit areas of particular syntax and yo…

We're adding support markdown with fenced code blocks, which maybe gets you closer to "delimiting areas of particular syntax": https://github.com/marimo-team/marimo/pull/1332 . Python is nice because it enables composition and reuse as vanilla scripts. But there are definitely tradeoffs.

Yes, fenced blocks is what I meant. Recreating parts of xml and cdata seems the common end goal. I think we have enough modern processing that you can do looser grammars.

Composition is an odd goal to me. Most notebook usage is exploratory. As such, taking effort to reuse notebooks feels counter to the benefits of doing it again that you can get.

Though, I'm a fan of seeing different workflows. And excited to be shown places I'm wrong! Kudos!

Re: Lessons learned reinventing the Python notebook

#36

Post by [deleted] »

Kalanos wrote: post #18

as a daily jupyter user, i like this .py DAG approach a lot - more so for visualizing complex workflows than addressing a potentially corrupt hidden state. there isn't much *data science functionality* integrated into jupyter core apart from displaying static dataframes (nor is their internal desire to add it), and jupyter *extensions/widgets* should really be built with dash/streamlit, so i'd encourage you to make a…

[deleted]

Re: Lessons learned reinventing the Python notebook

#37

Post by Kalanos »

similar to how you parse the cell's input variables, it would be cool if you parsed the files being read (e.g. `pd.read_parquet` and `file.open`) or databases/sources. you could show metadata about the sources next to it.

this UI setting could be turned off if every cell i/o from the same file

Re: Lessons learned reinventing the Python notebook

#38

Post by aaplok »

In principle every notebook is a DAG, since the cells are evaluated in a certain order. You could even infer the DAG from Jupyter notebooks, at least when the execution number is stored (the In[...] part).

There is a lot of value in imposing restrictions on the cells (like no reallocation of global variables) to be able to infer the DAG more directly like they do here (or Pluto.jl does). It is simpler, it can serve a dual purpose by being a normal runnable python/Julia/whatever program. Another advantage that I didn't see mentioned in TFA is parallelism. There is some interesting potential there I think.

Re: Lessons learned reinventing the Python notebook

#39

Post by akshayka »

aaplok wrote: post #38

In principle every notebook is a DAG, since the cells are evaluated in a certain order. You could even infer the DAG from Jupyter notebooks, at least when the execution number is stored (the In[...] part). There is a lot of value in imposing restrictions on the cells (like no reallocation of global variables) to be able to infer the DAG more directly like they do here (or Pluto.jl does). It is simpler, it can serve a…

> Another advantage that I didn't see mentioned in TFA is parallelism

That's something we're thinking about and designing:

https://github.com/marimo-team/marimo/issues/1103

Feel free to join the conversation there if you have thoughts!

Re: Lessons learned reinventing the Python notebook

#40

Post by akshayka »

dleeftink wrote: post #4

How about lessons learned from other reactive notebook environments that have been around for a while? If you are 'reinventing' something, I wonder why no mentions are made of the paths that have been laid before (Observable, Quarto, Enso, etc.).

Pluto.jl is our biggest inspiration. We also took some inspiration from streamlit, and of course a lot of inspiration from the IPython kernel and Jupyter. We didn't look closely at Observable (though Fons cites Observable as his biggest inspiration), and didn't look at Quarto or Enso at all.

As others have mentioned, we're very open about our inspiration and document it in many places, such as on GitHub, though you're totally right that we could have mentioned it here too.

We could fill another blog post talking about inspiration and lessons learned from other projects — perhaps we will!

Post reply on HN