Excel 2.0 – Is there a better visual data model than a grid of cells?

I did have a sort-kinda Excel replacement idea that I have half implemented, but it was based on distributed data sitting in OneDrive/DropBox/Google Drive etc. with formulas in a derived macro language that looked like Excel but worked like SQL.

It's built around Roslyn to do the evaluations. I did it that way because I wanted the flexibility of being able to run high-level style, predefined macros based around Excel, but with the ability to drop down to scripted C# if necessary.

It was a classic case of overkill. I need a way to do custom data extraction without re-writing the extract every time somebody made a new request. Some of the data had to be supplied by the client. I ended up with a monstrous, hash-based quasi database that runs inside an AWS Lambda.

I only half finished it because it does what I want at the moment (custom data exports for customers) but after I had it working I thought it was so stupid powerful I had to pause.

The main problem I see with something that is entirely distributed, but programmable at the C# level means it's also incredibly unsafe for anybody who is hosting it.

It would need a lot more work to be interactive (batch oriented for safety at the moment) but as an idea it seems like the next logical step from Google Sheets / Excel with a short learning curve for Excel jockeys.

Roslyn is an astonishing piece of tech really. The ability to precompile statements and call them by passing arguments is bonkers, I haven't played with anything like it since being a Smalltalk programmer in the 1990s.

The problem (and the reason for the success) with Excel is that it optimized for ease of use. The five minute out of the box experience is wonderful. However once you start "coding" (and I'll argue that if you using a vlookup or "if" function, you are programming with Excel) it is horrible. Ignores all of the best practices we have learned over the past 60 years for programming (modular, documentation, testing, source control, profiling).

I'm obviously biased given that I teach Python and Data Science. But I think you should optimize towards writing readable code. (From my experience teaching Pandas to thousands of students and seeing it in the wild, most people are using it wrong as well).

(It just so happens that I spent most of the day deep-diving into Excel and how to most effectively cross over to a proper programming paradigm...)

(I'm pretty biased since I wrote a book on this, Effective Pandas.)

My take is that if you embrace the limitation of chaining in Pandas it will force you to write better (easier to read, debug, deploy, share, collaborate) code.

Also works for exploration, reports, stats, aggregations. (Many examples here https://store.metasnake.com/ )

You can argue that spreadsheet formulas in cells are also a type of programming, including all the hierarchy you mentioned. Only in this case the hierarchy is not just the functions nested in each other, but also the cells calculating data based on other calculations in other cells.

And while I see the OP explanation on how cells are a great way to arrange data, I would argue that the existing way of programming in Excel is pretty horrible.

Anyone who have ever tried doing something slightly complex with Excel functions soon realized that it is pretty impossible to do certain things without a lot of "magic" involved. Which is why MS added the whole VBScript thing, and even Google Sheets have their JS App Script (or whatever it is called) to provide additional options to program based on data beyond the basic formulas.

It would be great to see more visual programming languages tested in an Excel like data entry environment. Some of the PLC [1] languages come to mind, or even languages like MIT Scratch.

[1] https://en.wikipedia.org/wiki/Programmable_logic_controller#...

I don't feel that a grid of cells is the best visual representation for data. However, it is a natural representation for tabular data.

Tables are horrible for "visualizing". We didn't evolve looking at tables with hundreds of columns and millions of rows.

Once you learn some pivoting tools and charting, you can visualize things that you would never be able to find in a table of data. (Or if you did it would take a lot longer.)

(Sample size 1, but I teach Python, Pandas, and visualization (Jupyter w/ matplotlib/seaborn/bokeh). I've had clients tell me that one chart they came up with during a class on visualization more than paid for the training. They would have never seen that in the table of data. I've also found bugs in code by visualizing failure patterns.)