Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Tyranny of Spreadsheets

timharford.com

Post reply on HN
Print view View original

231–240 of 282 posts

Re: The Tyranny of Spreadsheets

#231

Post by beckingz »

otherme123 wrote: post #195
zoomastigophore wrote: post #189

Earlier quoted context omitted.

I think one of the main problem with databases is how do you populate them. I wish there were simple-as-Excel frontends for databases where a normal user could input his data like in preformated Excel table without having to deal with the database mechanics.

Sadly databases are hard because data is hard. I've seen people trying to stuff one-to-many relationships in a single Excel tab duplicating the one side as many times as needed, e.g. Author_A Book_1 Author_A Book_2 Author_A Book_3 Author_B Book_4 Author_B Book_5 Another strategy I've seen is to stuff the many-side into a single cell, splitted with some less used symbol like |, ; or : . I'm curious to find what would…

This format is referred to as third normal form (boyce-codd 3NF). Also referred to as "tidy data" in the hadley wickham world.

Re: The Tyranny of Spreadsheets

#233

Post by benhurmarcel »

beefield wrote: post #184
max46 wrote: post #143

I think I big mistake people make here is that they think they are far better at everything on a computer than a non-programmer. They don't realize how much you can do in Excel and how much they suck at it. If you can't use it without a mouse and/or if you don't know pivot tables, you are just as much of a beginner as a C++ programmer who doesn't know what a pointer is. It's only after trying to convert an excel shee…

Opposing anecdote: I have converted excel spreadsheets built over years to SQL, and typically it did not take weeks to do that. ( Any organization that has a smallest reason to care about their data should remove save button from excel and start educating their personnel. Using excel in any important role should be seen as making the eventual mistakes on purpose and someone should be kept responsible.)

> I have converted excel spreadsheets built over years to SQL, and typically it did not take weeks to do that.

The "years" of work are typically not for implementing the spreadsheet, but to define what it does. The requirements evolve over many versions.

Of course it's fast to re-implement it after all the work to define the requirement is done, but it's only possible because Excel allowed all those prototypes.

Re: The Tyranny of Spreadsheets

#234

Post by benhurmarcel »

kfk wrote: post #151
rexreed wrote: post #87

Earlier quoted context omitted.

You're also correct, but I think miss the point as well. The spreadsheet isn't going anywhere. Curse it all we might, blame it for deaths or attribute any aspect of malignment to it, and you'd be right. But the spreadsheet isn't going anywhere. Spreadsheet mistakes have been directly attributed not only to thousands of deaths but billions of dollars of mistakenly wasted money. Heck, in the UK alone, they have attribu…

It is very clear that they are useful but it is also very clear that they can be problematic. In Finance people check each number - multiple times per day - because there is no way to write tests in Excel. Sounds good? Until you can trust the people crunching the numbers, of course. Would you build a bridge like that? Or a building? I think it is worth to talk about this problem and, at the very minimum, inform peopl…

> Would you build a bridge like that? Or a building?

That's exactly what's being done. I work in mechanical engineering and Excel is the primary calculation tool all over the industry.

Re: The Tyranny of Spreadsheets

#235

Post by benhurmarcel »

dan-robertson wrote: post #65
ttul wrote: post #61

Earlier quoted context omitted.

Personally, I strongly prefer working with GSheets to Excel. It handles large sheets better and you never worry about the app crashing and taking your precious work along with it.

Instead one only needs to worry that one can’t get one’s work done. I don’t think I’ve ever had a time when I wanted to use a spreadsheet for anything moderately complicated where gsheets was up to snuff. That’s not to say that excel doesn’t have problems or does things easily, but it doesn’t lead to quite the same level of frustration and hair-pulling, and indeed it’s usually possible to achieve things in excel when…

Google Sheets has more useful formulas than Excel. For example query(), but also all the array formulas that they include.

Re: The Tyranny of Spreadsheets

#236

Post by yodelshady »

Andy_G11 wrote: post #174

'000s of rows of data in a spreadsheet - sigh. But say 'database' to some people and they hear 'The Devil'. Excel is a glorious tool which welcomes all, the savvy and the unskilled but imaginative newbies alike. There is something about all those little cells that presents an itch everyone wants to scratch, and you just know that for some that scratching is going to produce something akin to a spreadsheet version of…

Say "database" and IT says "no, unless we spend a year reviewing requirements".

This is only a failure story for every software development route that involves procurement, which is every route except Excel. Those routes tracked, within the only timescale that mattered, precisely zero patients.

You want to give smart users DB and DVCS tooling? I'm right behind you. Start asking why they don't have those things already.

Re: The Tyranny of Spreadsheets

#237

Post by gopher_space »

dsr_ wrote: post #205
zoomastigophore wrote: post #189

Earlier quoted context omitted.

I think one of the main problem with databases is how do you populate them. I wish there were simple-as-Excel frontends for databases where a normal user could input his data like in preformated Excel table without having to deal with the database mechanics.

What do we know about most database projects? They need CRUD plus queries. What's the easiest way to generate a CRUD form? HyperCard. We need a ubiquitous HyperCard equivalent that talks to the Real Database(TM) of your choice on the backend -- sqlite, postgresql, mysql/mariadb, Oracle, MS-SQL, whatever -- with easy functions for "get a sequence number", "get a unique identifier cookie" and "do this block atomically"…

Filemaker is a neat program.

Re: The Tyranny of Spreadsheets

#238

Post by tutuncommon »

rnavi wrote: post #227
zoomastigophore wrote: post #189

Earlier quoted context omitted.

I think one of the main problem with databases is how do you populate them. I wish there were simple-as-Excel frontends for databases where a normal user could input his data like in preformated Excel table without having to deal with the database mechanics.

> I wish there were simple-as-Excel frontends for databases This is exactly what we are attempting to solve at nocodb : https://github.com/nocodb/nocodb Which is, nocodb gives you a google-drive like collaborative spreadsheets on your existing databases (MySQL, Postgres etc) And the original problem in article could be countered with nocodb as we keep audit of all changes done to the database.

This looks like a good tool.

Re: The Tyranny of Spreadsheets

#239

Post by quietbritishjim »

mkl wrote: post #218
akho wrote: post #191

Earlier quoted context omitted.

No, it wouldn’t. Case in point: Excel has ability to spread a calculation over multiple lines, but the bugs are not down. In fact, people who are aware of Alt-enter produce buggier code: they end up writing longer formulas, with fewer intermediate results displayed, and have less visibility of the functioning of their spreadsheets. Write simpler formulas.

> Write simpler formulas. Excel's formula language seems deliberately designed to prevent that.

They mean simpler formula per cell, by splitting the formula across multiple cells (as I mentioned in my sibling comment).

Re: The Tyranny of Spreadsheets

#240

Post by FabHK »

CJefferson wrote: post #56
goatlover wrote: post #38

Earlier quoted context omitted.

Notebooks are the next-gen spreadsheets, except they use programming languages instead of formulas.

Jupyter is a joke compared to spreadsheets, it doesn't auto-recalculate when you edit cells.

Not only that, it doesn't even keep track of the calculation graph, so it doesn't know what needs calculating, or in which order to calculate.
Post reply on HN