Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Spreadsheets Are Hot–and Cranking Out Complex Code

wired.com

Post reply on HN
Print view View original

101–110 of 142 posts

Re: Spreadsheets Are Hot–and Cranking Out Complex Code

#101

Post by civilized »

cpeterso wrote: post #99
civilized wrote: post #85

Earlier quoted context omitted.

Those are great! But they are coding disciplines one must choose to follow, and continue following every step of the way. You aren't inherently following them because that's how the tool works. Sometimes the tool-enforced discipline has advantages.

How would one write tests for a complex, mission-critical Excel spreadsheet? Or use version control? Spreadsheets often mix the data and code/formulas and the formulas are hidden behind the sheet view and sprinkled across many cells. At least Python scripts separate the code from the data so you can write tests using known good or fuzzing data. And you can use version control to track and review code changes.

I agree that, in the hands of a hypothetical ideal person, Python should be better than Excel for almost everything. But as my story above shows, even people you'd expect to be experts, seemingly following best practice, don't end up being very close to this ideal person.

Re: Spreadsheets Are Hot–and Cranking Out Complex Code

#102

Post by arrosenberg »

civilized wrote: post #65
theptip wrote: post #62

Earlier quoted context omitted.

I think there is scope to replace Excel, but it’s hard. You’re seeing a gradual spread of “data science” tools in finance, as the more technical analysts start to use Python over Excel. But as you say, the management layer still uses Excel to poke and prod a model. I think that there will be a generational shift here - you are not going to train an SVP to use Python, but the next generation of SVPs might have more ex…

I have a colleague who is a PhD in Applied Math. Pretty bright guy, huge Python/Jupyter lover with many years of experience. Loves to use git, loves to write dozens of unit tests. A true Man of the Future, according to Excel haters. He wrote some Python to solve some mildly complex business problem. I told him to translate it into Excel for stakeholders. He did, and the answer came out completely different. It turned…

I have an equal number of anecdotes where a few lines of easily readable python replace complex Excel calcs that invariably have a mistake somewhere.

The tools aren’t the issue.

Re: Spreadsheets Are Hot–and Cranking Out Complex Code

#103

Post by nerdponx »

syshum wrote: post #81
nerdponx wrote: post #32

Earlier quoted context omitted.

It's not about autonomy in an abstract sense. It's the fact that regular people can customize, hack, modify, automate, and otherwise program the spreadsheet. Data analysts and managers are more than trained monkeys; they actually need to do those kinds of things (at least sometimes) in order to do their jobs. I agree that the ideal world is one where you can connect the spreadsheets to other data sources, so you get…

This is a double edge sword, because often time there is no one validing the results of the spread sheets... "The number looks ok" is not a good validation, and there has been some very public data errors as a result of bad spreadsheets. I often wondered if in the average business is even 10% of the spreadsheets where actually audited what would happen.... I suspect the results would be rather shocking

Indeed, and I think this is the real problem with non-programmer/data people working with spreadsheets. The problem isn't the spreadsheet interface itself. The problem is that people are doing what amounts to engineering without any training in engineering and no sense of best practices.

Re: Spreadsheets Are Hot–and Cranking Out Complex Code

#104

Post by crispyambulance »

fivea wrote: post #89
rr808 wrote: post #84

Earlier quoted context omitted.

> I've learned products designed to replace spreadsheets have a huge hurdle because the people who use spreadsheets treat operating the sheet as their job. Replacing them removes their autonomy and control over an information process, and subsumes the value they bring to their employers - so they will resist products that threaten that. Excel is a complete management subculture. Spreadsheets are better because as you…

> Spreadsheets are better because as you say the owner is their job to maintain it. If you replace with an IT process the new "owner" is likely a below-average developer that probably is uninterested in the business. I disagree. Spreadsheets are incomparably worse because what you charitably described as "the owner is their job to maintain it" in real life it's reflected as having a single employee who abused a first…

> Advocating for spreadsheets is advocating for these vulnerabilities.

Spreadsheets will exist regardless of what developers think of them. Ironically, that's a good thing.

Many projects that put food on dev's tables started out as out-of-control Excel monstrosities that were created and operated for long spans of time by well-meaning and productive folks. They start as simple manual spreadsheets with some formulas and then evolve into much more involved beasts. Work gets done and it's all nicely contained in somebody's cube and they look good and can be rightfully proud of their accomplishment.

Things just get done. For a while. Sometimes a LONG while. Until the bitter realities that software developers have learned to deal with over the decades start to seep into these projects and drown the unwitting folks who created them, slowly but surely, like an ever-increasing number of small holes in the bottom of boat. That's when things break or become unmanageable and that's when developers start getting engaged-- assuming these excel masterpieces have actually become mission-critical.

There's a guy at work that operates one of these excel monstrosities. It's been going for ~7 years now. It's a monster excel spreadsheet that, among an ever-growing list of things, does dubious probabilistic forecasting of future PO's based on shit ripped from salesforce (not even using the api). He has a dedicated laptop behind him pulling in data from multiple sources, like clockwork, and has recently started making attractive Power-BI dashboards using his excel worksheets as the data sources. And you know what? He looks GOOD to the people that matter. Does the forecasting actually work? Not really, but being so immersed in all that data has made him knowledgeable about many details of the operation. He's able to keep track of costs and stay on top of things. It doesn't matter (to him) that the whole thing will vaporize when he leaves, or that he could tire of it and just foist it upon some hapless supply-chain person who's just learned to use formulas in excel.

Re: Spreadsheets Are Hot–and Cranking Out Complex Code

#105

Post by doliveira »

syshum wrote: post #81
nerdponx wrote: post #32

Earlier quoted context omitted.

It's not about autonomy in an abstract sense. It's the fact that regular people can customize, hack, modify, automate, and otherwise program the spreadsheet. Data analysts and managers are more than trained monkeys; they actually need to do those kinds of things (at least sometimes) in order to do their jobs. I agree that the ideal world is one where you can connect the spreadsheets to other data sources, so you get…

This is a double edge sword, because often time there is no one validing the results of the spread sheets... "The number looks ok" is not a good validation, and there has been some very public data errors as a result of bad spreadsheets. I often wondered if in the average business is even 10% of the spreadsheets where actually audited what would happen.... I suspect the results would be rather shocking

I'm not sure if I should be proud of this or not, but once I had some pretty critical field values in a Excel spreadsheet that I had to make sure were compatible with my code. I ended up adding some derived formulas in a new locked sheet within the file, committed the XLSX to Git and then created a JUnit test to make sure everything was in sync. In a nutshell, the XLSX became the source of truth

Re: Spreadsheets Are Hot–and Cranking Out Complex Code

#106

Post by [deleted] »

syshum wrote: post #81
nerdponx wrote: post #32

Earlier quoted context omitted.

It's not about autonomy in an abstract sense. It's the fact that regular people can customize, hack, modify, automate, and otherwise program the spreadsheet. Data analysts and managers are more than trained monkeys; they actually need to do those kinds of things (at least sometimes) in order to do their jobs. I agree that the ideal world is one where you can connect the spreadsheets to other data sources, so you get…

This is a double edge sword, because often time there is no one validing the results of the spread sheets... "The number looks ok" is not a good validation, and there has been some very public data errors as a result of bad spreadsheets. I often wondered if in the average business is even 10% of the spreadsheets where actually audited what would happen.... I suspect the results would be rather shocking

[deleted]

Re: Spreadsheets Are Hot–and Cranking Out Complex Code

#107

Post by wolverine876 »

narush wrote: post #6

Warning: I'm a founder working in the spreadsheet space, so take the rest of my this comment with a (large) grain of salt. I've written before [1] (HN and elsewhere) about how I think spreadsheets are the most popular programming paradigm ever, we just don't talk about it much. As this article mentions, there are many ways we can push this forward. I personally think the most powerful low-code spreadsheet tools we ca…

> We're open core [3]

What does that mean? The citation doesn't mention "open core", but does say Mito is open source.

Re: Spreadsheets Are Hot–and Cranking Out Complex Code

#108

Post by josephcsible »

wolverine876 wrote: post #107
narush wrote: post #6

Warning: I'm a founder working in the spreadsheet space, so take the rest of my this comment with a (large) grain of salt. I've written before [1] (HN and elsewhere) about how I think spreadsheets are the most popular programming paradigm ever, we just don't talk about it much. As this article mentions, there are many ways we can push this forward. I personally think the most powerful low-code spreadsheet tools we ca…

> We're open core [3] What does that mean? The citation doesn't mention "open core", but does say Mito is open source.

"Open core" means that most of the software is open source, but that some pieces of it are proprietary. See here: https://github.com/mito-ds/monorepo/blob/c4e225e7fcb1caea930...

Re: Spreadsheets Are Hot–and Cranking Out Complex Code

#109

Post by josephcsible »

haolez wrote: post #14
narush wrote: post #6

Warning: I'm a founder working in the spreadsheet space, so take the rest of my this comment with a (large) grain of salt. I've written before [1] (HN and elsewhere) about how I think spreadsheets are the most popular programming paradigm ever, we just don't talk about it much. As this article mentions, there are many ways we can push this forward. I personally think the most powerful low-code spreadsheet tools we ca…

You need the commercial license to disable telemetry? That's... new.

> That's... new.

Sadly, it isn't. Microsoft set the precedent with Windows 10's telemetry, which they only give you a setting to turn off if you bought Enterprise Edition.

Re: Spreadsheets Are Hot–and Cranking Out Complex Code

#110

Post by wolverine876 »

josephcsible wrote: post #108
wolverine876 wrote: post #107

Earlier quoted context omitted.

> We're open core [3] What does that mean? The citation doesn't mention "open core", but does say Mito is open source.

"Open core" means that most of the software is open source, but that some pieces of it are proprietary. See here: https://github.com/mito-ds/monorepo/blob/c4e225e7fcb1caea930...

Thanks. I'm a bit dubious about using the jargon - which people can easily misunderstand - isntead of saying, "mostly open source components, some priopriatary ones".
Post reply on HN