Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Spreadsheets Are Hot–and Cranking Out Complex Code

wired.com

Post reply on HN
Print view View original

111–120 of 142 posts

Re: Spreadsheets Are Hot–and Cranking Out Complex Code

#111

Post by bigDinosaur »

syshum wrote: post #81
nerdponx wrote: post #32

Earlier quoted context omitted.

It's not about autonomy in an abstract sense. It's the fact that regular people can customize, hack, modify, automate, and otherwise program the spreadsheet. Data analysts and managers are more than trained monkeys; they actually need to do those kinds of things (at least sometimes) in order to do their jobs. I agree that the ideal world is one where you can connect the spreadsheets to other data sources, so you get…

This is a double edge sword, because often time there is no one validing the results of the spread sheets... "The number looks ok" is not a good validation, and there has been some very public data errors as a result of bad spreadsheets. I often wondered if in the average business is even 10% of the spreadsheets where actually audited what would happen.... I suspect the results would be rather shocking

Yup, and this is frequently a total disaster. Like, hundreds of thousands of dollars get lost either in spreadsheet errors or by paying the salaries of people who manage spreadsheets when their jobs could probably be replaced by a much better solution.

But, whatever - more work for everyone! Certainly suits me. It's like wanting the average person to keep programming C because you're in infosec.

Re: Spreadsheets Are Hot–and Cranking Out Complex Code

#112

Post by cgio »

fivea wrote: post #89
rr808 wrote: post #84

Earlier quoted context omitted.

> I've learned products designed to replace spreadsheets have a huge hurdle because the people who use spreadsheets treat operating the sheet as their job. Replacing them removes their autonomy and control over an information process, and subsumes the value they bring to their employers - so they will resist products that threaten that. Excel is a complete management subculture. Spreadsheets are better because as you…

> Spreadsheets are better because as you say the owner is their job to maintain it. If you replace with an IT process the new "owner" is likely a below-average developer that probably is uninterested in the business. I disagree. Spreadsheets are incomparably worse because what you charitably described as "the owner is their job to maintain it" in real life it's reflected as having a single employee who abused a first…

I have heard these horror stories so many times but never witnessed them in reality. From my point of view, excel is a great way to ensure knowledge is not hidden, as you have a file format that embeds calculations, data and outputs. It can be ugly, but nothing that a seasoned excel warrior cannot parse and there’s plenty of them around. Now if someone as an employer does not even have copies of the files, they have bigger problems than Excel itself.

The other horror stories of errors in spreadsheets, them yes I have witnessed them regularly.

Re: Spreadsheets Are Hot–and Cranking Out Complex Code

#113

Post by yen223 »

haolez wrote: post #14
narush wrote: post #6

Warning: I'm a founder working in the spreadsheet space, so take the rest of my this comment with a (large) grain of salt. I've written before [1] (HN and elsewhere) about how I think spreadsheets are the most popular programming paradigm ever, we just don't talk about it much. As this article mentions, there are many ways we can push this forward. I personally think the most powerful low-code spreadsheet tools we ca…

You need the commercial license to disable telemetry? That's... new.

This sounds reasonable to me? In exchange for telemetry data, you get to use this product without paying a single cent.

Re: Spreadsheets Are Hot–and Cranking Out Complex Code

#114

Post by somat »

ogogmad wrote: post #98
janci wrote: post #86

Excel is great and versatile, yet lacks some critical features. - It's easy to load data from database or API without VBA, yet impossible to write updates back without VBA. With VBA it's still messy string concatenation of SQL queries. - VBA is security nightmare - Version control is bad. Distributing spreadsheets in emails and collecting changes back is nightmare. Sharing a file on network drive lacks fine-grained p…

> Impossible to make reusable blocks. The click and drag down feature results in a kind-of code reuse. It bypasses the need to consciously name things. Is this generalisable? For small scale code, these concerns may be overkill.

Click and drag down is the opposite of code reuse, it is effectively a one time code generator.

In fact it is one of the main reason I have largely abandoned spreadsheats, the behaviour I actually want is found in sql views. Calculate all items in column with one expression vs calculate items in column with unique expressions that were copy, pasted then transformed for each and every row.

The other main reason I abandoned spreadsheets is row level integrity. too many times data that goes together in a row has drifted apart(sort on column subset is main culprit), another inherent problem solved by using a relational database.

the solution to having excel at every desk is to have postgres at every desk. the code will be just as bad but the data integrity will be better.

Re: Spreadsheets Are Hot–and Cranking Out Complex Code

#115

Post by catgary »

civilized wrote: post #101
cpeterso wrote: post #99

Earlier quoted context omitted.

How would one write tests for a complex, mission-critical Excel spreadsheet? Or use version control? Spreadsheets often mix the data and code/formulas and the formulas are hidden behind the sheet view and sprinkled across many cells. At least Python scripts separate the code from the data so you can write tests using known good or fuzzing data. And you can use version control to track and review code changes.

I agree that, in the hands of a hypothetical ideal person, Python should be better than Excel for almost everything. But as my story above shows, even people you'd expect to be experts, seemingly following best practice, don't end up being very close to this ideal person.

Honestly, your story just indicates that someone senior needs to have a discussion with the applied mathematician about their coding practices (and they may even need a bit of training in that regard).

Re: Spreadsheets Are Hot–and Cranking Out Complex Code

#116

Post by civilized »

catgary wrote: post #115
civilized wrote: post #101

Earlier quoted context omitted.

I agree that, in the hands of a hypothetical ideal person, Python should be better than Excel for almost everything. But as my story above shows, even people you'd expect to be experts, seemingly following best practice, don't end up being very close to this ideal person.

Honestly, your story just indicates that someone senior needs to have a discussion with the applied mathematician about their coding practices (and they may even need a bit of training in that regard).

His code style is fine. His problem is that he is short on the discipline and focus to validate his results in meaningful ways. An exhortation to follow some vague "coding practices" won't fix that - he'll respond e.g. by writing lots of tests, none of which catch the problem his code actually has.

The two ways I know to get correct work out of him are (1) review it and kick it back to him when problems are found, (2) have him implement what he's trying to do in Excel.

Re: Spreadsheets Are Hot–and Cranking Out Complex Code

#117

Post by cutler »

civilized wrote: post #65
theptip wrote: post #62

Earlier quoted context omitted.

I think there is scope to replace Excel, but it’s hard. You’re seeing a gradual spread of “data science” tools in finance, as the more technical analysts start to use Python over Excel. But as you say, the management layer still uses Excel to poke and prod a model. I think that there will be a generational shift here - you are not going to train an SVP to use Python, but the next generation of SVPs might have more ex…

I have a colleague who is a PhD in Applied Math. Pretty bright guy, huge Python/Jupyter lover with many years of experience. Loves to use git, loves to write dozens of unit tests. A true Man of the Future, according to Excel haters. He wrote some Python to solve some mildly complex business problem. I told him to translate it into Excel for stakeholders. He did, and the answer came out completely different. It turned…

You said he "loves to write dozens of unit tests" so why so many errors?

Re: Spreadsheets Are Hot–and Cranking Out Complex Code

#118

Post by civilized »

cutler wrote: post #117
civilized wrote: post #65

Earlier quoted context omitted.

I have a colleague who is a PhD in Applied Math. Pretty bright guy, huge Python/Jupyter lover with many years of experience. Loves to use git, loves to write dozens of unit tests. A true Man of the Future, according to Excel haters. He wrote some Python to solve some mildly complex business problem. I told him to translate it into Excel for stakeholders. He did, and the answer came out completely different. It turned…

You said he "loves to write dozens of unit tests" so why so many errors?

Sometimes it's hard to think of the right tests, especially when you're solving a mathematical model that hasn't been solved before.

Even for things that are intuitive and have been implemented thousands of times before, like web logins and shopping carts, where the tests one should do are not hard to think of... even so, software engineers rarely develop tests that catch all possible bugs on the first try.

Re: Spreadsheets Are Hot–and Cranking Out Complex Code

#119

Post by cutler »

mch82 wrote: post #47
wildrhythms wrote: post #2

Every day I think about how Microsoft Access allowed otherwise not-so-tech-savvy users to, with just a little training and practice, build a complete relational database for their entire business, supported by a relatively sane GUI and a way to build forms and reports with very little (if any) 'code'. I have no idea what small companies are using nowadays, but I think there has to be some untapped middle ground somew…

Claris Works on Macs in the 1990s is the best database product I’ve ever used. It was amazingly user friendly. I used it to make form letters & reports. Claris Works prioritized the application layer of the database and hid a lot of the complexity at the data layer. DB Browser for SQLite is the coolest database product I’ve used recently. Similar to Mito, DB Browser generates SQL into a log as operations are performe…

Filemaker Pro occupied that space a while ago as the MS Access for Mac.

Re: Spreadsheets Are Hot–and Cranking Out Complex Code

#120

Post by cutler »

joejohnston wrote: post #59

Check out Budibase. It's the leading open source low code platform and perfect for building UIs on top of spreadsheets https://github.com/Budibase/budibase

Budibase looked perfect until I learned there is no option for hosting locally. Doesn't make sense.
Post reply on HN