Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mito – Excel-like interface for Pandas dataframes in Jupyter notebook

trymito.io

Post reply on HN
Print view View original

31–40 of 101 posts

Re: Mito – Excel-like interface for Pandas dataframes in Jupyter notebook

#31

Post by aarondia »

sodimel wrote: post #10

Looks like a Datasette[0] clone which runs on top of something (jupyter) which runs on top of Python (ipython). I think I would like to see how much time it takes to open a massive dataset in Mito & in Datasette :P [0]: https://datasette.io/

Heyo, one of the Mito creators here. Thanks for sharing Datasette. I haven't seen that one before. It looks neat!

You're right though, there are several tools that fit the general shape of: GUI on top of Jupyter on top of Python. There's a few general vectors to understand these tools by:

1. Excel-ness: Although most (if not all) of these tools incorporate some type of spreadsheet, the interface for interacting with the data in that spreadsheet differs greatly. Some tools, like Bamboolib [1] and Datasette [2] resemble Excel only in the spreadsheet. Other tools, like Mito [3], stick to a lot of the other Excel design decisions -- things like having a toolbar with buttons and menu items to access functionality, the ability to write spreadsheet formulas inside of the cell & formula bar, etc. In many ways, this Excel-ness design vector is a proxy for how easy it is to get started with the tool. What we see, is that users are able to download Mito and get something useful out their first analysis because the interface is one that they are used to!

2. Ownership of your analysis / lack of lockin: We believe that the most powerful low-code spreadsheet tools allow spreadsheet users to easily transition to full programming languages, if they want to. Instead oflocking users into a limited and proprietary product, it's better if users can transition to a full programming language (like Python) very naturally. This transition is super natural in Mito because we generate Python code for every edit that a user makes. So if Mito doesn't support the exact transformation that you want, you can use Mito as a starting point for your analysis and customize the script that Mito generates.

[1] https://bamboolib.8080labs.com/ [2] https://datasette.io/ [3] https://www.trymito.io/

Re: Mito – Excel-like interface for Pandas dataframes in Jupyter notebook

#32

Post by CJefferson »

santiagobasulto wrote: post #12

I like this. Is a "friendlier" way to browse data. Said that, I have to add: Exploring large datasets requires a COMPLETELY different mindset. When your data starts growing, it's impossible to keep it all in a visual format (for 2 reasons[0]) and you have to start thinking analytically. You have to start looking at the statistical values of your data to understand what's its shape. That's why the `.describe()` and `.…

I find the world is full of datasets with Also, even with huge datasets I tend to always look at a random sample, and the "most extreme" datapoints -- mainly because in my experience there is a good chance some parts of the data are malformed, and need to be recollected/fixed. Of course, if you trust your data collection you don't need this!

Re: Mito – Excel-like interface for Pandas dataframes in Jupyter notebook

#33

Post by wenc »

santiagobasulto wrote: post #12

I like this. Is a "friendlier" way to browse data. Said that, I have to add: Exploring large datasets requires a COMPLETELY different mindset. When your data starts growing, it's impossible to keep it all in a visual format (for 2 reasons[0]) and you have to start thinking analytically. You have to start looking at the statistical values of your data to understand what's its shape. That's why the `.describe()` and `.…

I would caution against this approach in general (unless you’re working with unusually uniform data from a deterministic source — in my world that is rarely the case). Summary statistics are useful but taken in isolation they can mislead. One loses the ability to get a feel for interesting non-aggregated phenomenon.

I find it’s important to actually “touch” the raw data even if only in a buffered, random sampling sort of way to get a feel for it. Sometimes with big datasets, looking through rows of data feels tedious and meaningless but I’ve found that I’ve often picked up on things I wouldn’t have without actually looking at the raw data. Raw data is often flawed, but there’s often some signal in it that tells a story hence it’s important not to overlook these through a lens of aggregate statistics.

The next step is to visualize the data multidimensionally in something like Tableau. Tableau works on very large datasets (it has an internal columnstore format called Hyper) and can dynamically disaggregate and drill down. Insights are usually obtained by looking at details, not aggregates.

Re: Mito – Excel-like interface for Pandas dataframes in Jupyter notebook

#35

Post by pbronez »

aarondia wrote: post #24
santiagobasulto wrote: post #12

I like this. Is a "friendlier" way to browse data. Said that, I have to add: Exploring large datasets requires a COMPLETELY different mindset. When your data starts growing, it's impossible to keep it all in a visual format (for 2 reasons[0]) and you have to start thinking analytically. You have to start looking at the statistical values of your data to understand what's its shape. That's why the `.describe()` and `.…

This is a great point and something that we're actively working on improving in Mito. If you have millions of rows of data, its not enough to just scroll through your data, you need tools to build your understanding. Some of the tools that you mentioned exist in Mito today. For example, Mito generates summary information about each column (all of the .describe() info along with a histogram of the data). And we're cre…

It would be super fun to implement an intelligent head() function that shows a representative sample rather than the first X rows. Do the profiling & identify a collection of rows that represent the overall distribution.

You could develop some IP around efficient and effective ways to do this. Probably would require an ensemble of unsupervised methods.

Re: Mito – Excel-like interface for Pandas dataframes in Jupyter notebook

#36

Post by mejutoco »

wenc wrote: post #33
santiagobasulto wrote: post #12

I like this. Is a "friendlier" way to browse data. Said that, I have to add: Exploring large datasets requires a COMPLETELY different mindset. When your data starts growing, it's impossible to keep it all in a visual format (for 2 reasons[0]) and you have to start thinking analytically. You have to start looking at the statistical values of your data to understand what's its shape. That's why the `.describe()` and `.…

I would caution against this approach in general (unless you’re working with unusually uniform data from a deterministic source — in my world that is rarely the case). Summary statistics are useful but taken in isolation they can mislead. One loses the ability to get a feel for interesting non-aggregated phenomenon. I find it’s important to actually “touch” the raw data even if only in a buffered, random sampling sor…

A good example of what you are warning against is Anscombe’s quartet

https://en.wikipedia.org/wiki/Anscombe's_quartet

Re: Mito – Excel-like interface for Pandas dataframes in Jupyter notebook

#37

Post by santiagobasulto »

wenc wrote: post #33
santiagobasulto wrote: post #12

I like this. Is a "friendlier" way to browse data. Said that, I have to add: Exploring large datasets requires a COMPLETELY different mindset. When your data starts growing, it's impossible to keep it all in a visual format (for 2 reasons[0]) and you have to start thinking analytically. You have to start looking at the statistical values of your data to understand what's its shape. That's why the `.describe()` and `.…

I would caution against this approach in general (unless you’re working with unusually uniform data from a deterministic source — in my world that is rarely the case). Summary statistics are useful but taken in isolation they can mislead. One loses the ability to get a feel for interesting non-aggregated phenomenon. I find it’s important to actually “touch” the raw data even if only in a buffered, random sampling sor…

Of course that `.head()`, `.tail()`, `iloc` and other mechanisms to visualize the data of subsets is always important. But would you really caution AGAINST this? Like, literally telling someone NOT to use summary statistics to explore a dataset?

Re: Mito – Excel-like interface for Pandas dataframes in Jupyter notebook

#38

Post by santiagobasulto »

mejutoco wrote: post #36
wenc wrote: post #33

Earlier quoted context omitted.

I would caution against this approach in general (unless you’re working with unusually uniform data from a deterministic source — in my world that is rarely the case). Summary statistics are useful but taken in isolation they can mislead. One loses the ability to get a feel for interesting non-aggregated phenomenon. I find it’s important to actually “touch” the raw data even if only in a buffered, random sampling sor…

A good example of what you are warning against is Anscombe’s quartet https://en.wikipedia.org/wiki/Anscombe's_quartet

Histograms and Boxplots (and IQRs) don't lie tho...

Re: Mito – Excel-like interface for Pandas dataframes in Jupyter notebook

#40

Post by aarondia »

pen2l wrote: post #27
aarondia wrote: post #22

Earlier quoted context omitted.

Yeah, Mito is limited to the Jupyter ecosystem (for now). We want to expand to VSCode, Google Collab, and Streamlit! For the time being, because Mito generates pandas code for every edit you make, you can always use Mito in Jupyter to generate code, and then copy it over to VSCode. Admittedly, its not as nice of a workflow, but it does work!

I see you guys provide convenient installers that can be obtained with pip. Me, I run my JupyterLab that I got with Conda on a Windows setup. Can you comment on whether it's a thorn-free path to get Mito in such a setup? Or should I use this as another sign to completely migrate to a nix system for all my dev needs... :)

You should be good to go with your conda setup on windows! I run Mito on a windows machine through a conda virtual environment often! We have some instructions for how to do that here [1]

[1] https://docs.trymito.io/getting-started/installing-mito/inst...

Post reply on HN