Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stop Using Excel, Finance Chiefs Tell Staffs

wsj.com

Post reply on HN
Print view View original

301–310 of 435 posts

Re: Stop Using Excel, Finance Chiefs Tell Staffs

#301

Post by jimbokun »

dalbasal wrote: post #62
GiorgioG wrote: post #37

Earlier quoted context omitted.

When I consulted at a health insurance company (a Blue Cross Blue Shield licensee with 1000 employees, 200 IT staff) about 10 years ago, I was absolutely horrified that the system of record for health care plan data was Excel spreadsheets. They had staff who would update the claims processing system based on thousands of spreadsheets. When you called customer service, the CSR app would auto-open the member's plan inf…

There are upsides to excel in cases like that… The health insurance people of various sorts who use this setup are the ones with domain knowledge. With an excel program, they can (1) understand how it works (2) add to it, modify it (3) create a copy and break it (4) debug it (5) have ideas about how it could work differently… I’m not saying that excel is the tool for the job (sounds like it is definitely not), and th…

"If the excel team was blessed with a plague of inspiration tomorrow morning, Is there a way they could make it so that excel is the right way to manage an insurance business?"

What a professional software developer's workflow has and Excel lacks:

* principled way of sharing code across a team with distributed source control (or similarly powerful tool),

* automated regression tests,

* input validation,

* continuous deployment tools to automatically deploy working code (default mode for Excel I guess), and a mechanism to roll back to a previous version (not so much),

* build and dependency management tools for bringing in libraries from a vast open source ecosystem,

* systems for reporting bugs and feature requests and tracking their progress,

* networked database that is the shared "source of truth" for all users,

* security and auditing,

* hardware capacity planning (run locally or in AWS? what are the cost trade offs?)

So sure, get the health insurance people doing all of those things, and have Microsoft figure out how to do all of them well in Excel, and congratulations, they are now software developers. None of these require using a "real programming language", at least not directly.

Re: Stop Using Excel, Finance Chiefs Tell Staffs

#302

Post by anigbrowl »

gbrown wrote: post #184
Robotbeat wrote: post #174

Earlier quoted context omitted.

Completely agree. It's common to tell kids (and everyone) that they need to "learn to code." That is what Excel already is! And it's incredibly powerful, especially if you don't have any other coding skills. But, of course, it's messy. When tens of millions of people can each write their own programs and easily modify them and change them to suit their own needs without needing to recompile or anything, then you get…

> ...and, in fact, that's also what you'd get if you taught everyone how to program Python in high school instead of Excel. I disagree. Python leaves a readable instruction set, while Excel buries the logic in an opaque two dimensional grid of cells (plus or minus atrocious VBA). I think this gives a better ladder for people to improve their skills to the point of not producing unmaintainable garbage.

The grid of cells is your memory space. Coders like looking at a stack. Spreadsheet users like looking at the registers.

Which you prefer depends to some extent on which end of the problem you're coming from; if you're starting out with a lot of domain knowledge, a spreadsheet is attractive because you can look at the data, see whether the way it's being stored/processed is basically correct by spot checking it, and then automate. I'll still often reach for a spreadsheet if I don't know off the top of my head what the rules or relationships are and I need the equivalent of a whiteboard with logic.

If you have a top-down perspective and you want to automate a process whose fundamentals you know well, code is more comfortable. And you can go back and expand, modify, or adjust it later. But two things I don't like about code for handling large volumes of data are that I have to construct some sort of viewer to format the output (requiring more coding, or slower bug discovery if I press ahead without), and endless typing I like selecting things from menus and tool palettes because while I don't mind holding an instruction set in my head I don't like doing the same for every library I might want to use, especially remembering all the parameters. I hate hate hate writing glue code and want the computer to do that stuff for me.

Fortunately a lot of these problems seem to be receding thanks to tools like Jupyter and some of the promising Flow-based programming tools that let you easily visualize structure, code, and visualization without getting bogged down in details.

I should add that I prefer hacking to software engineering; that is, I'm only interested in programming as a means to an end of having a customizable tool for myself rather than building a product for other people, so I want spend as little of my time as possible on constructing the tool.

Re: Stop Using Excel, Finance Chiefs Tell Staffs

#303

Post by anigbrowl »

jimbokun wrote: post #301
dalbasal wrote: post #62

Earlier quoted context omitted.

There are upsides to excel in cases like that… The health insurance people of various sorts who use this setup are the ones with domain knowledge. With an excel program, they can (1) understand how it works (2) add to it, modify it (3) create a copy and break it (4) debug it (5) have ideas about how it could work differently… I’m not saying that excel is the tool for the job (sounds like it is definitely not), and th…

"If the excel team was blessed with a plague of inspiration tomorrow morning, Is there a way they could make it so that excel is the right way to manage an insurance business?" What a professional software developer's workflow has and Excel lacks: * principled way of sharing code across a team with distributed source control (or similarly powerful tool), * automated regression tests, * input validation, * continuous…

That's great (though I'd quibble about a few of them, eg I was doing input validation in Excel 25 years ago, and boy do I feel old now), but while you're learning to do all that great stuff you're not learning anything about the domain you want to work in.

OK, I don't know who wants to work in health insurance but the basic point is Excel is a good tool for people who have specialist knowledge in a particular domain and want to automate their work. Programmers know the best way of programming but frequently overestimate their understanding of other people's problems.

Re: Stop Using Excel, Finance Chiefs Tell Staffs

#304

Post by jokoon »

Shouldn't you be able to do the same thing R does with excel?

I mean at some point if you cannot teach everyone a programming language, you can at least give them some tool that can do a similar job.

Teaching programming is so expensive, the money has always been into making programming just easier and easier.

Re: Stop Using Excel, Finance Chiefs Tell Staffs

#305

Post by anigbrowl »

ajlburke wrote: post #104
mabbo wrote: post #18

I would argue that the most commonly used programming language is Excel. But few of the people using it realize they're programming. It's a brilliant reactive data programming model that makes intuitive sense to non-technical users. They feel empowered to use it to solve problems right now with a computer. They experiment with it, try things, Google how to do more things- just like any programmer does. And they feel…

I used to be a Lotus Notes specialist, and one interesting thing I noticed is that from the earliest versions it was designed specifically to be a "step up" from a spreadsheet (Lotus of course also made Lotus 1-2-3, which was the original killer-app spreadsheet before Excel took over). Notes still let you see data as rows and columns and had very spreadsheet-like functions, many of which could be transferred directly…

What would you use now if you needed the same sort of functionality? I find myself with some unexpected needs to build databases whose specifications and scope I'm still trying to figure out, and for now I'm missing about in Google sheets and figuring Fusion tables will probably be sufficient, but I'd be interested in any alternatives as long as they're secure and shareable.

Re: Stop Using Excel, Finance Chiefs Tell Staffs

#306

Post by Pamar »

ajlburke wrote: post #104
mabbo wrote: post #18

I would argue that the most commonly used programming language is Excel. But few of the people using it realize they're programming. It's a brilliant reactive data programming model that makes intuitive sense to non-technical users. They feel empowered to use it to solve problems right now with a computer. They experiment with it, try things, Google how to do more things- just like any programmer does. And they feel…

I used to be a Lotus Notes specialist, and one interesting thing I noticed is that from the earliest versions it was designed specifically to be a "step up" from a spreadsheet (Lotus of course also made Lotus 1-2-3, which was the original killer-app spreadsheet before Excel took over). Notes still let you see data as rows and columns and had very spreadsheet-like functions, many of which could be transferred directly…

I basically agree - but in my case, instead of Lotus, it was Framework: https://en.wikipedia.org/wiki/Framework_(office_suite) - I think I used versione III and IV, mostly.

Re: Stop Using Excel, Finance Chiefs Tell Staffs

#307

Post by askvictor »

guiomie wrote: post #211
jasode wrote: post #22

The phrases "avoid data silos" and "we want single source of truth" have been used repeatedly about Excel misuse/abuse/overuse for 20+ years. The WSJ story talks about the complaint but not about the underlying tension between data redundancy vs data manipulation. Excel endures because it's the non-programmer's REPL for data. Like a Lisp REPL, using Excel is nimble and has an instant feedback loop. Click "A-Z" to sor…

"One middle ground between Excel data silos and the utopia of a centralized system is a shared document store like Sharepoint or Dropbox." ... 100% agree, and add I find that Google Sheet is the best for this role: 1 source of truth, modification history, read/write permissions and shareable. I am now starting to use spreadsheets for use cases I would have strongly objected in the past. I'm surprised Google sheet isn…

And an API that lets data be used in other 'real' applications

Re: Stop Using Excel, Finance Chiefs Tell Staffs

#308

Post by Pamar »

sheetjs wrote: post #147
ajlburke wrote: post #104

Earlier quoted context omitted.

I used to be a Lotus Notes specialist, and one interesting thing I noticed is that from the earliest versions it was designed specifically to be a "step up" from a spreadsheet (Lotus of course also made Lotus 1-2-3, which was the original killer-app spreadsheet before Excel took over). Notes still let you see data as rows and columns and had very spreadsheet-like functions, many of which could be transferred directly…

> my career in business software has still mostly been about converting spreadsheets into more "proper" applications. Still true for many people in 2017. @patio11 probably said it best: > Every spreadsheet shared in a business is an angel announcing another SaaS app still needs to be built. https://twitter.com/patio11/status/655674551615942657 FWIW we ended up in business software after one of our open source librari…

> Every spreadsheet shared in a business is an angel announcing another SaaS app still needs to be built.

Really? I am working in company which is not handling super-sensitive data (i.e. not a Bank, for example) but I can assure you that if went around proposing to adopt a SAAS for what Excel is used internally I would not get very far.

No matter how good you are as a SAAS provider, our data stay in-house. End of story.

Re: Stop Using Excel, Finance Chiefs Tell Staffs

#309

Post by anigbrowl »

CabSauce wrote: post #282
will_pseudonym wrote: post #271

Earlier quoted context omitted.

1,048,576 rows to be precise. ;) Though you'll run into performance issues in more complex spreadsheets once you get into the 100,000 row area, or even 10,000 or so. Then you'll see power user departments switch to the more opaque but more performant Access.

They switch to Access until they hit the data size limit of 2GB. Then they come back to IT who has to deal with the mess that you get when you allow domain experts to build databases.

Domain experts are not the best people to build databases (says a domain expert who's failing miserably at that task right now), but database experts are sort of terrible at understanding domain problems.

It seems to me that many programmers don't appreciate that industry practices or regulatory requirements impose quirky requirements on domain specialists that everyone knows are far from optimal but which need to be complied with anyway if the business is to turn a profit or retain its licensure.

Someone mentioned a health insurance example above: there's 50 states, with 50 different sets of insurance regulations, and then some additional territories, federal regulations, and then wird business practices of healthcare providers who are subject to their own 57 varieties of regulation, and there are powerful economic and political interests that get in the way of burning it all down and replacing it with a single system designed by a committee of one patient, one doctor, one nurse, one pharmacist, one lawyer, and one economist.

And even that highly simplistic requirements committee went through 3 versions while I was writing it up.

The domain experts already have lots of complex knowledge and experience they need to automate. They don't want to learn about databases. Unfortunately, the reverse is equally true.

Post reply on HN