The Tyranny of Spreadsheets • Hacker News Board

Re: The Tyranny of Spreadsheets

#141

Post by markus_zhang » Fri, Jul 23, 2021, 3:49 AM UTC

kwyjobojoe wrote: post #101

markus_zhang wrote: post #23

In my last company we keep a spreadsheet that has tabs occupying almost the maximun number of columns allowed and in different headers. Updating that Excel file needs the full team to work for two full days each month. That was after I created a VBA script to scrape data from internal dashboards and put them directly in the spreadsheet. From my experience these monstrosities are usually created by two reasons: Push f…

We use spreadsheets to manage data that would 1000% be better suited for storage in a database (and extracted the data would be sooooo much simpler than my monthly half day job of updating management reports). Why? Because we don't have access to any databases. I dread to think of the requests and committees I would need to find and traverse to get a database set up for my team, let alone a system that I can write an…

Now that you mentioned, we didn't have access to SQL Server dwh until 6 months before I left for good.

Can you try sqlite? It's usually enough for simple things I think.

Re: The Tyranny of Spreadsheets

#142

Post by max46 » Fri, Jul 23, 2021, 4:18 AM UTC

navneetloiwal wrote: post #118

rexreed wrote: post #37

* The Utility of Spreadsheets No single application has been as widely adopted by as wide a number of people for such a wide range of uses as the spreadsheet. Love them or hate them, but the spreadsheet metaphor is at once highly useful, highly adaptable, and highly usable requiring minimal support to extract needed value. The spreadsheet, with all its questionable glory, is here to stay. From the first Visicalc to E…

This! As programmers, it is easy to list N reasons why spreadsheets suck and deserve to die. But the reality is that it is the most ubiquitous business software and the non-tech/business users would rather be in a spreadsheet than anywhere else. Companies run on spreadsheets, even if they have the best "data stack" and tools. Because few, if any, software can beat the * flexibility (throw data anywhere and link it to…

I think I big mistake people make here is that they think they are far better at everything on a computer than a non-programmer. They don't realize how much you can do in Excel and how much they suck at it. If you can't use it without a mouse and/or if you don't know pivot tables, you are just as much of a beginner as a C++ programmer who doesn't know what a pointer is.

It's only after trying to convert an excel sheet to [insert your favorite language] that they realize it would take them 6 month with a team of 5 to replicate what a single accountant did in a week.

Re: The Tyranny of Spreadsheets

#143

Post by max46 » Fri, Jul 23, 2021, 4:24 AM UTC

I think I big mistake people make here is that they think they are far better at everything on a computer than a non-programmer. They don't realize how much you can do in Excel and how much they suck at it. If you can't use it without a mouse and/or if you don't know pivot tables, you are just as much of a beginner as a C++ programmer who doesn't know what a pointer is.

It's only after trying to convert an excel sheet to [insert your favorite language] that they realize it would take them 6 month with a team of 5 to replicate what a single accountant did in a week.

Re: The Tyranny of Spreadsheets

#144

Post by catherd » Fri, Jul 23, 2021, 4:30 AM UTC

LeifCarrotson wrote: post #49

swader999 wrote: post #25

Many businesses risk it all by building actual applications with Excel. Security, version control, automated testing are pretty much set aside. Case in point: https://insideparadeplatz.ch/wp-content/uploads/2013/01/ubs-...

It's not like they had a meeting where a consultant said "We reviewed all your requirements, and you could build this with normal software engineering practices, or we could build this in spreadsheets with associated deleterious effects on privacy, correctness, durability, extensibility, readability, and interoperability, likely resulting in an estimated $N to $NN million shortfall in 5 years. What do you want? Yeah,…

Spreadsheets are like the ubiquity of motorcycles in 3rd world countries. Jumping on one and blasting off without any safety equipment is cheap, fast, everyone uses them, and they get 80% of the job done. And they will kill you, but that's life.

The thing about spreadsheets is you don't have to commission bespoke software. And just the value of that swamps everything else in the vast majority of cases.

In small businesses I doubt the success rate is anywhere near 50% if a non-technical person recognizes a business need for a software solution, hires someone to build it for them, and then it actually reaches a useful conclusion. But if you are numerate at all you can drive a spreadsheet. Not having to swim with the sharks for months or years while you hemorrhage cash is hard to appreciate for the sharks (us, the people who build technical things). Most small businesses and many large ones don't have anyone who can manage even contracting something like this out.

If things have gone on long enough that the problem is stable, clearly defined, and the costs of 3rd order effects like lack of revision control are losing enough money to justify the risk of a custom software project, sure, it needs to be done. But for many companies that risk will always be bigger than the ones that come with spreadsheets.

ERP systems are supposedly finished software that you deploy and tweak for particular business needs. The success rate on deploying those is waaaay below 100%, and the expenses are huge. If your company or department is non-technical the time for custom business software is usually when your hair is on fire and it's the only option.

Re: The Tyranny of Spreadsheets

#145

Post by cwillu » Fri, Jul 23, 2021, 4:43 AM UTC

hartator wrote: post #14

Yes, hijack my scrolling. I really don't like have a consistent feedback when I am scrolling pages.

And the content doesn't load if you disable javascript!

Re: The Tyranny of Spreadsheets

#146

Post by kumarvvr » Fri, Jul 23, 2021, 5:00 AM UTC

gcthomas wrote: post #4

Spreadsheets are a hammer so if that's all you know then all problems look like the nail. Really, many power users of Excel ought to be looking at more capable, testable and readable solutions from professional data analysts and scientists. Python would be a good start, but there are many options better than Excel for critical systems.

I disagree.

Spreadsheets are like a Swiss Army Knife.

I have used them for

- Templates for Printing - Data Analysis - Financial Tracking - Report Generation - Calculation sheet for engineering stuff

and a multitude more uses that I cannot even remember now.

They are not a hammer, though that is one form in which they can work.

Ultimately they are a tool, and a damn fine tool at that.

Their potential is limited only by the creativity of the user using them.

Re: The Tyranny of Spreadsheets

#147

Post by [deleted] » Fri, Jul 23, 2021, 5:05 AM UTC

LeifCarrotson wrote: post #49

swader999 wrote: post #25

Many businesses risk it all by building actual applications with Excel. Security, version control, automated testing are pretty much set aside. Case in point: https://insideparadeplatz.ch/wp-content/uploads/2013/01/ubs-...

It's not like they had a meeting where a consultant said "We reviewed all your requirements, and you could build this with normal software engineering practices, or we could build this in spreadsheets with associated deleterious effects on privacy, correctness, durability, extensibility, readability, and interoperability, likely resulting in an estimated $N to $NN million shortfall in 5 years. What do you want? Yeah,…

[deleted]

Re: The Tyranny of Spreadsheets

#148

Post by catherd » Fri, Jul 23, 2021, 5:11 AM UTC

The thing that's always baffled me about Excel is why you must always work in "minified" mode when composing formulas.

It seems like just adding the ability to spread a calculation out over multiple lines and add some indentation would make the bugs everyone complains about go down by... a lot.

Re: The Tyranny of Spreadsheets

#149

Post by qwrshr » Fri, Jul 23, 2021, 5:18 AM UTC

nicoffeine wrote: post #125

asdff wrote: post #39

Earlier quoted context omitted.

I don't know why we think the user is some child and try to abstract away any and all complexity from their job, which ends up leading to some proprietary solution with huge inefficiencies somewhere compared to a flat file and a script in python or R. Software like excel is often seen as a way to do stuff you could do in R but without having to write code. Imo that thinking is wrong because it makes this assumption t…

> I'd guess it would be up to business programs and accounting programs and all these other college majors to actually teach classes in python and R, rather than what they do now which is teach classes in excel. Imo the entrenchment in excel is rooted in ignorance to other (often simpler) options available with python or R, than in any actual critical proprietary features offered by excel. Every time a business user…

there are languages that have better support with dependencies though, e.g. Julia (every time I have to debug dependency issues in some other person's Python code/requirements.txt, I find myself wishing it had been written in Julia). I'm not disagreeing with the overall tenor of your post, but I do think there's a medium that's possible to strike here: maybe something like a simplified DSL in Julia, or something like https://www.visidata.org/

Re: The Tyranny of Spreadsheets

#150

Post by mulmen » Fri, Jul 23, 2021, 5:32 AM UTC

rossdavidh wrote: post #42

A nice post, brought low by the unfortunate fact that it misses the primary reason we were able to eradicate smallpox (and it wasn't data tracking): smallpox had no animal reservoirs Most viruses (and other diseases) have animal reservoirs, and that's why few diseases which ever get widespread, are ever eradicated, and it's nothing to do with spreadsheets. Now if he wanted to make a point about data, it would be good…

The article makes two points. First that data is important but must be curated and maintained. Second that spreadsheets require specific methodologies and expertise to prevent data quality issues from propagating.

I don't think that part of the story had anything to do with spreadsheets. It just illustrated the value of (timely) data, especially when fighting a virus and specifically in distributing scarce vaccine doses.

A virus with animal reservoirs could similarly be controlled based on monitoring where outbreaks occur and taking some action. That could be distributing vaccines, limiting access to the animals or taking some action to reduce the animal reservoir.