Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I was wrong about spreadsheets (2017)

reifyworks.com

Post reply on HN
Print view View original

281–290 of 378 posts

Re: I was wrong about spreadsheets (2017)

#281

Post by mcphage »

navigatesol wrote: post #260
ozim wrote: post #207

Earlier quoted context omitted.

Yeah it works and you don't need support contracts and then: - Fidelity's "Minus Sign Mistake": loss of $1.3 billion - TransAlta "Clerical Error": loss of $24 million - Fannie Mae "Honest mistake": loss of $1.3 billion Then you get employee turn over where new employees don't get "arcane" knowledge passed down by people who left and took their spreadsheet foo with them. Excel does not have "access control", "auditing…

> Then you get employee turn over where new employees don't get "arcane" knowledge passed down by people who left and took their spreadsheet foo with them. Yeah, this never happens with a proprietary codebase!

Who needs historical knowledge when you can just rewrite the program every few years in a different environment with all new bugs? (and a smattering of the old ones, too)

Re: I was wrong about spreadsheets (2017)

#282

Post by PaulHoule »

eej71 wrote: post #218
ozim wrote: post #207

Earlier quoted context omitted.

Yeah it works and you don't need support contracts and then: - Fidelity's "Minus Sign Mistake": loss of $1.3 billion - TransAlta "Clerical Error": loss of $24 million - Fannie Mae "Honest mistake": loss of $1.3 billion Then you get employee turn over where new employees don't get "arcane" knowledge passed down by people who left and took their spreadsheet foo with them. Excel does not have "access control", "auditing…

I think it will be easier to solve those problems in the Excel development space than it will be to export the Excel solutions to another solution domain. Also the kind of mistakes you've described can (and do) happen in other programming domains that are deemed more respectable.

Unfortunately no.

Excel is hamstrung by having to maintain backwards compatibility for an endless number of hacks.

For instance, it is notorious that

0.1 + 0.2 != 0.3

in binary exponent floating point math. Excel does funny stuff with number formatting that hides this, but it is like having a bubble under a plastic sheet that moves someplace else when you push on it -- numeric strangeness appears in different places.

The right answer is to go to decimal exponent floating point math, but that is only HW accelerated on IBM Mainframes, maybe on RISC-V at some point. You'll probably crash Excel if you have enough numbers in it for performance to matter, but Microsoft would be afraid of any performance regression and it would break people's sheets so it won't happen.

On a technical basis we could use an Excel replacement that has some characteristics of Excel, and other characteristics of programming languages; one old software package to look to for inspiration is

https://en.wikipedia.org/wiki/TK_Solver

What makes it almost impossible to do on a marketing basis is that Excel is bundled into Microsoft Office so if you have an Office subscription you have Word, Powerpoint, Excel, Access, etc.

Re: I was wrong about spreadsheets (2017)

#283

Post by jchb »

ekianjo wrote: post #267
phonon wrote: post #253

Earlier quoted context omitted.

> it does not support native UTF8 csv files It does though? You can export as a UTF-8 csv file...check the export options.

I am talking about importing CSV files in UTF8. If I use Japanese characters for example it fails utterly to display properly unless I import it as "Text" first.

The secret trick for importing data into Excel (without actually writing a file in the Excel file format): don't try CSV - as you have already noticed that will fail in all kind of fragile ways depending on your character encoding and operating system locale. Instead, convert your data into an html table and then paste or import that into Excel.

But not any html table, you need to have the appropriate proprietary css properties in there, such as the mso-data-placement:same-cell. Those are actually documented ...in the Office HTML and XML Reference published in 1999.

Re: I was wrong about spreadsheets (2017)

#284

Post by o_nate »

tonygrue wrote: post #43
viraptor wrote: post #21

My issue with spreadsheets is that they could be improved a lot with minimal changes, not nobody seems to do it in popular office packages. Making them more database-like and making table data first-class (at least you can make named tables in excel on windows) could be used to push people a bit more towards organised data, without changing how anything works. Half of the mess that makes excel hell comes from the fac…

I'd be over the moon to see one change to Excel. Native support for a language other than VBA. Perhaps the CLR. Perhaps Java/Typescript. Just something (optional) for programmers who want to use Excel and not want to deal with the garbage that VBA is. Some workarounds exist, but afaik they require collaborators to also have the tool install, which is dead in the water. I know MS has considered it, I'm still pretty su…

I dislike VBA too, but it would be easier to stomach if Excel at least had a decent IDE. Compare the VBA editor in Excel with Visual Studio and the difference is stark.

Re: I was wrong about spreadsheets (2017)

#285

Post by akavi »

kinj28 wrote: post #137
viraptor wrote: post #21

My issue with spreadsheets is that they could be improved a lot with minimal changes, not nobody seems to do it in popular office packages. Making them more database-like and making table data first-class (at least you can make named tables in excel on windows) could be used to push people a bit more towards organised data, without changing how anything works. Half of the mess that makes excel hell comes from the fac…

In my opinion - "No Code" tools will fix this gap to build real world apps yet being as simple as excel. While airtable is just a DB as a service, a real no code tool would have ideally all the basic building blocks: a. DB as a service (like Airtable) b. BPM capability -> to add workflows to the data c. Ui Building to abstract data d. API integration (may not advanced learning for power users)

Airtable has blocks for c) and exposes an API for d)

Not sure what you specifically want for b)

Re: I was wrong about spreadsheets (2017)

#286

Post by swiley »

mbesto wrote: post #251
ozim wrote: post #207

Earlier quoted context omitted.

Yeah it works and you don't need support contracts and then: - Fidelity's "Minus Sign Mistake": loss of $1.3 billion - TransAlta "Clerical Error": loss of $24 million - Fannie Mae "Honest mistake": loss of $1.3 billion Then you get employee turn over where new employees don't get "arcane" knowledge passed down by people who left and took their spreadsheet foo with them. Excel does not have "access control", "auditing…

> - Fidelity's "Minus Sign Mistake": loss of $1.3 billion - TransAlta "Clerical Error": loss of $24 million - Fannie Mae "Honest mistake": loss of $1.3 billion Weigh that against makes $X Billion due to correct usage of Excel multiplied thousands and thousands of times over. Survivorship bias at it's finest. Excel isn't perfect and has notable downsides, but work gets done all of the time on them for the aforemention…

I’m not convinced there’s a such thing as “correct usage of excel.”

What a lot of companies do is just scare entry level employees into being very careful. My friend’s girlfriend works at a place like this and people just accept all the problems with manually editing large spreadsheets because “that’s just life.”

Anything in excel is a hack and most of its users don’t know any better.

Re: I was wrong about spreadsheets (2017)

#287

Post by inputcoffee »

ozim wrote: post #207
halfeatenpie wrote: post #26

I work in critical infrastructure planning. My organization builds software in R, Python, and other programming languages customized for these major organizations. So many critical infrastructures, billions of dollars in planning, and just systems are built out of Excel. It's amazing. You'd assume something that services millions of people a day would have some more sophisticated and customized solution, but you're w…

Yeah it works and you don't need support contracts and then: - Fidelity's "Minus Sign Mistake": loss of $1.3 billion - TransAlta "Clerical Error": loss of $24 million - Fannie Mae "Honest mistake": loss of $1.3 billion Then you get employee turn over where new employees don't get "arcane" knowledge passed down by people who left and took their spreadsheet foo with them. Excel does not have "access control", "auditing…

The Fidelity "minus sign mistake" didn't create a loss. The mistake was in relaying the information to the end user. It didn't actually cause a loss of that magnitude.

That is like saying if I mistyped in a word doc, that word created the loss.

Re: I was wrong about spreadsheets (2017)

#288

Post by chaseha »

jchb wrote: post #283
ekianjo wrote: post #267

Earlier quoted context omitted.

I am talking about importing CSV files in UTF8. If I use Japanese characters for example it fails utterly to display properly unless I import it as "Text" first.

The secret trick for importing data into Excel (without actually writing a file in the Excel file format): don't try CSV - as you have already noticed that will fail in all kind of fragile ways depending on your character encoding and operating system locale. Instead, convert your data into an html table and then paste or import that into Excel. But not any html table, you need to have the appropriate proprietary css…

I frequently go from SQL query results to HTML table to excel in a similar manner, but I haven't had issues w/ any proprietary CSS properties. Using Paste Special -> Match Destination Formatting it always comes out without any issues

Re: I was wrong about spreadsheets (2017)

#289

Post by BeetleB »

ekianjo wrote: post #268
astine wrote: post #252

Earlier quoted context omitted.

Well if you're using Python, you're following an imperative/oop paradigm which goes back to the sixties, so I don't know why you'd criticize Excel for using a dataflow model. I can understand not wanting to use Excel because it's proprietary (but Libreoffice isn't) but I can can understand not wanting to deal with a lot of it's legacy cruft, but for the stuff that Excel does well, it does it a lot more quickly, more…

> that you can see every step of your calculation which creates a degree of self documentation which you don't get with Python, for example. If you want to do that in Python then Jupyter Lab/notebook is a good solution to document your code and see the result of each operation. And way faster than Excel for large data tables.

Yep. My first job had a lot of Excel. I hated it. Once I learned pandas, I rarely needed Excel again for my work.

Re: I was wrong about spreadsheets (2017)

#290

Post by chaseha »

hef19898 wrote: post #211
TeMPOraL wrote: post #199

Earlier quoted context omitted.

> It is using Eycel in addition to what ever system is being used the first place (SAP for example is a pretty popular thing to circumvent with Excel sheets). (...) And circumventing existing systems with local offline spreadsheets just screws up everything. I usually side with the circumventers on this one. This misbehavior happens for a reason, which usually is that it's impossible or infeasible to do the work with…

True that. I used to both back then, first a user and then a SAP system guy. For me it is always a death spiral: people don't trust the system or it is too cumbersome, they use Excel, the system gets even less reliable, the use Excel even more,... and so on. I always saw it as my role a system guy to give users a system they can use and then force them to actually use it. Because, when providing the systemnI talked t…

Agree but slight modification - goal should always be for the users to be able to do their main functional tasks in-application and maintain it as the system of record.

But there will always be a use for Excel as a tool for informal data-monkeying and analysis - exporting a table and doing some pivots to summarize or use a different view. Non-technical users will always expect this capability and flock back to it. Not everyone's a programmer or has the interest to be - you'll have to pry excel out of most accountant's cold, dead hands for example

Post reply on HN