Live data from Hacker News

Finance Pros Say You’ll Have to Pry Excel Out of Their Cold, Dead Hands

wsj.com

221–230 of 317 posts

Re: Finance Pros Say You’ll Have to Pry Excel Out of Their Cold, Dead Hands

#221

One huge reason why Finance pros in general, and Fund Managers, Brokers, Traders in particular, love Microsoft Excel is because all your formulas and calculations update almost instantly (unless you explicitly disable this feature). So they can apply their business and process knowledge to create tables and formulas, pull data from various sources, and create something that is self-contained, stored locally (inside t…

>I've seen very sophisticated OMSs (Order Management Systems) built completely using excel workbooks, with 1 sheet containing orders and formula's to pull realtime pricing from bloomberg, reuters, another sheet for pre-trade analysis, another for the actual trades, and a summary sheet with pivot tables and charts to show the trade progress and summary.

Wait, people are making API calls in VBA to insert into Excel spreadsheets?

Re: Finance Pros Say You’ll Have to Pry Excel Out of Their Cold, Dead Hands

#222

The issue in finance is the speed of change in demands. No formalized system platform with proper change control can keep up. The ability to play with data, the ability to break your work is valuable in end user computing and the type of thing that is hard to recreate in a formal financial system that must pass internal and external audits. edit: I entire a whole.

I wonder if this also describes scientific programming pretty well.

Not sure... would be interesting if you could describe a situation in that field?

Re: Finance Pros Say You’ll Have to Pry Excel Out of Their Cold, Dead Hands

#223
post #66

Finance dev guy here. Excel's dominance in the field is because it is an _application container_ that _non_ dev people can use. The workflow is this: - old trader guy says to his junior guy: "hey can you look into xxx." - junior trader guy says: "sure I'll make a spreadsheet for it" - old trader guy: "great your model is all I need, let's trade" - several weeks later, IT guy says: "hey you're running a $100m book out…

As unpopular as it would be to say - Stop trying. Excel is a phenomenal accomplishment, and while everyone may talk about a better spread sheet, the market really doesn't want it or need it. Its basically craigslist. The only people who think it can be or needs to be done better are primarily not the users. Excel is the human condition, messy, varied, capable of doing many things and most of the features aren't used,…

I agree and I have stopped trying as well. I'm in bioinformatics and I do the big data, Linux cluster, data pipeline stuff. Then in the end I dump it all in Excel and let the very smart and knowledgeable (lab) biologists play with it. It's important that they play and have flexibility to massage data as they see fit, they understand the data deeply. I started out constantly trying to figure out what they want and build it into the pipeline... It doesn't work because often they don't know what they want exactly when starting.

Now I just offer handy things like pre-sorting, filtering (some boolean columns to allow them to quickly filter stuff), better column names or reports with extra info. I try to figure out the things they find tedious and automate them once and for all. But I hold back as much as I can.

Re: Finance Pros Say You’ll Have to Pry Excel Out of Their Cold, Dead Hands

#225

Everyone talks about trying to create a new type of programming environment, something intuitive and visual and that lets you think in new and creative ways. Excel is exactly that. It's easy and nonintimidating to nonprogrammers. It's visual, intuitive, and with a basic understanding of math and an idea of cells you can do a large number of tasks that would otherwise require a program. Yes, it's a problem when it get…

"Everyone talks about trying to create a new type of programming environment, something intuitive and visual and that lets you think in new and creative ways. Excel is exactly that" - yes, it is, except it is a way too old already though all the notable competitors seem dedicating all the efforts to reproducing its legacy as closely as possible. I'd love to see an attempt to design a brand-new spreadsheet from scratc…

How do you mean it's "way too old already"? If it works, it works. You're guaranteed to introduce problems every time you introduce change so there needs to be a strong reason to do. Why change what works?

Re: Finance Pros Say You’ll Have to Pry Excel Out of Their Cold, Dead Hands

#227
post #143

Earlier quoted context omitted.

I am a typical excel power user, and have started to increasingly do work outside of Excel (C# mostly). I find however that Excel is still a formidable GUI, particularly for tabular inputs. One middle ground is libraries like SpreadsheetGear that give you a UI that has an almost perfect excel look and feel while giving you more control. You don't need to integrate your tool in Excel, you can integrate a quasi-Excel i…

I'm a professinal C# dev. When I need to do any one off analysis of data or a running system, I just spin up linqpad, write some C# to grab my data, put it through a functional transformation pipeline with LINQ, then, optionally, graph the results using some WPF graphing components. I hit F5, and my results appear in the window below as either a table of data or a graph. I don't even have excel installed on my window…

Yeah but data analysis is only one of many usages for Excel.

Let me give you a practical example I ran last week. I want to construct a portfolio of loans starting with an existing portfolio, such that it follow a certain total balance and maturity profile through time, by replenishing it every month with new loans of various maturity (I am solving for the individual notionals of these new loans).

So inputs are

1. the existing portfolio, hundreds of entries with various parameters: start, end, notional, etc

2. the target maturity profile at every point in time

3. the target balance of the portfolio at every point in time

A simple LINQ query won't do. For the example above, you want to model it in C#, but the inputs are lots of tables. In that case I find the optimal way is for the script to accept spreadsheets as input files (wrote a "xlsx (de)serialiser") and output files. Effectively Excel becomes the UI of the script. Nothing beats the Excel UX in giving you the ability to build a table, change or copy/paste data in bulk, etc.

Re: Finance Pros Say You’ll Have to Pry Excel Out of Their Cold, Dead Hands

#228
post #66

Finance dev guy here. Excel's dominance in the field is because it is an _application container_ that _non_ dev people can use. The workflow is this: - old trader guy says to his junior guy: "hey can you look into xxx." - junior trader guy says: "sure I'll make a spreadsheet for it" - old trader guy: "great your model is all I need, let's trade" - several weeks later, IT guy says: "hey you're running a $100m book out…

As unpopular as it would be to say - Stop trying. Excel is a phenomenal accomplishment, and while everyone may talk about a better spread sheet, the market really doesn't want it or need it. Its basically craigslist. The only people who think it can be or needs to be done better are primarily not the users. Excel is the human condition, messy, varied, capable of doing many things and most of the features aren't used,…

The problem with Excel is that whilst it's an easy development environment, and a vastly flexible one, it's absolute death to debug.

At Uni, decades ago (and using Lotus-1-2-3 rather than Excel), we were given a set of spreadsheet templates for a business management exercise (the project ran much the duration of the course). I discovered looking at these that there was an error in the spreadsheet double-counting all inputs (rows and columns were added and the row-totals included in the grand total). Best I can tell, finding that bug was not in fact part of the course goal....

At an early gig doing analytics we hired an old fart who actually introduced some rigour to our project practices, including code walkthroughs. Some of this involved more obviously programmatic tools, but the process was also applied, over the strenuous objections of the analyst involved, to a project largely done in Excel. Numerous errors were found, and corrected. The spreadsheet was also significantly rationalised by having "dumb" data-entry area, and then relying on a sheet with VBA code rather than in-cell logic for the computations.

The analyst continued to object to the code review, after the errors were found and corrected, insisting that it was inappropriate because he "wasn't a programmer". (I looked him up recently, he's continuing to work on analytic projects, two decades on.)

This was also about the time that I first heard of Ray Panko, at the University of Hawaii, who'd found reliably that 88% of spreadsheets had errors. This still holds largely true.

http://panko.shidler.hawaii.edu/

I'm not saying "get rid of all Excel". But I am saying that the tool makes for very, very, very buggy processes, and this is itself a problem.

I'd love to see work toward alternatives that retain the flexibility and versatility of Excel (or other spreadsheets) but reduce the possibility of bugs and errors.

Re: Finance Pros Say You’ll Have to Pry Excel Out of Their Cold, Dead Hands

#229

Earlier quoted context omitted.

... wow. Instead of hashing the file contents, it just checked file size? Who came up with this algorithm?! There's plenty of applications that use files of fixed large sizes initialized at install, for performance reasons or just simply so if you're going to run out of space you do so during interactive installation, rather than silently breaking later on.

From what I can determine, the authoritative version was always the one with the later timestamp. This might have worked at least in theory, however there were always issues. The foremost issue was that the timeserver was virtualized, and drifted to an unbelievable degree. This issue was compounded by the fact that we used citrix virtual machines for some applications, and our local desktops for other applications. T…

This would make a great story for The Daily WTF: https://what.thedailywtf.com/
Post reply on HN