Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Tyranny of Spreadsheets

timharford.com

Post reply on HN
Print view View original

21–30 of 282 posts

Re: The Tyranny of Spreadsheets

#21

Post by version_five »

simlan wrote: post #15
version_five wrote: post #12

Earlier quoted context omitted.

I actually stopped reading there and went to the comments. I dont mean any disrespect, just trying to give a counterpoint: I like to have some more upfront info about an article's thesis before I invest in reading about an Italian history lesson hoping it will be relevant. I see this style a lot for example on substack, as if people are always trying to weave in a relevant historical anecdote or clever fact, which is…

Well tastes are different. I liked it because of the interspersed backdrop. Granted i am reading this on a chill evening not on the go. Would be different if i had expected a to the point article.

Yeah I definitely don't find it odd to like that style, I was just reacting because I found it interesting that the thing someone highlighted as being a strong point was exactly the thing that had put me off. Different tastes as you say.

Re: The Tyranny of Spreadsheets

#22

Post by dwohnitmok »

Jaepa wrote: post #5

This was adapted from Tim Harharford's podcast Cautionary Tales eps Wrong Tools Cost Lives https://timharford.com/2021/05/cautionary-tales-wrong-tools-... If you have not listened to the series I highly recommend it. His episode "LaLa Land: Galileo’s Warning" is by far one of my favorite pieces of media. In brief it is why redundant tightly coupled fail safes will often lead to cascading failures. https://timharford.…

To what extent is the Galileo effect described in that episode really just a kind of survivorship bias? If you've eliminated all the failures that can be caused by non-redundant systems with no failsafes, then the only remaining failures are those that can be caused by redundant systems with failures (or at least slipped through). The way to see whether those redundancies and failsafes are working is to see whether the overall failure rate is decreasing, not solely whether failures are being caused by those redundancies and failsafes.

Re: The Tyranny of Spreadsheets

#23

Post by markus_zhang »

In my last company we keep a spreadsheet that has tabs occupying almost the maximun number of columns allowed and in different headers. Updating that Excel file needs the full team to work for two full days each month. That was after I created a VBA script to scrape data from internal dashboards and put them directly in the spreadsheet.

From my experience these monstrosities are usually created by two reasons: Push from business side and a data team too slow to react. Thus the analysts have to create the first version that seems to be OK and smart but eventually evolved into a monstrosity.

Re: The Tyranny of Spreadsheets

#24

Post by themodelplumber »

joe_the_user wrote: post #6

Data caught in spreadsheets is kind of an instance of the general siloing of data. The situation is something of a general problem of a data-driven Society. One group of people want information available with one interface and spends resources only in putting it in a format suitable - to the detriment of others who/want need it. With spreadsheets in particular, naturally you have a data-integrity but that's still an…

It's a general psychological problem, too. A huge part of storage organization psychology, whether in software or in a physical room, is subjective/proprietary perception and judgment. People who prefer and naturally warp any job role or position to use these tools, as all humans do with subjectively preferred processes and perspectives, will never stop creating them.

The good part of this not-invented-here mindset is that it's also how technology moves forward. "I need this new system to be mine/fit my way of organizing" is also often another way of creating a "this never existed before" product or outcome.

These storage and organization tools and silos are born as subjective (context-fitting) design processes. So they tend to get locked up behind depth-oriented (subjective) processes, and eventually frozen by stabilizer groups.

Stabilizer groups follow up to use the tools and promote the idea of keeping the data siloed the way it is. They don't like change at work because change breaks their preferred psychological processes, causes them to have to re-build their perceptual frameworks, and because they're humans, this makes them do stupid things at work, like lash out or become passive-aggressive or detonate their new diet plan or become late for a baseball game. They will blame all of this on "open data" or whatever it is that caused their stable workflow to destabilize.

There are lots of solutions to this, and Excel by itself can be seen as an attempt at a solution to the problem...also the problem can be moderated by the system of energy surrounding the data valuation and access to the data.

And "just don't silo stuff, guys" seems to be the proposal with the worst track record so far.

Re: The Tyranny of Spreadsheets

#28

Post by anthk »

chasd00 wrote: post #13
gcthomas wrote: post #4

Spreadsheets are a hammer so if that's all you know then all problems look like the nail. Really, many power users of Excel ought to be looking at more capable, testable and readable solutions from professional data analysts and scientists. Python would be a good start, but there are many options better than Excel for critical systems.

what would python be working on then? a .csv, textfile, database, something else? Or would the data be stored as part of the script itself?

TSV, you have several tools to parse it. Even AWK.

Re: The Tyranny of Spreadsheets

#30

Post by spoonjim »

Excel has its flaws, as do spreadsheets in general, but the upside is that spreadsheets allow people to become programmers who never would grok programming in purely abstract terms.

Spreadsheets allow the user to see each intermediate state in their application, as well as each iteration over the input data set, laid out in a 2-dimensional grid. There just isn't any substitute I've seen that is as accessible to the computing "layperson."

Post reply on HN