Live data from Hacker News

I was wrong about spreadsheets (2017)

reifyworks.com

181–190 of 378 posts

Re: I was wrong about spreadsheets (2017)

#181

I work in critical infrastructure planning. My organization builds software in R, Python, and other programming languages customized for these major organizations. So many critical infrastructures, billions of dollars in planning, and just systems are built out of Excel. It's amazing. You'd assume something that services millions of people a day would have some more sophisticated and customized solution, but you're w…

I just hate one single thing about Excel - all function names are localized - and in case of my native language - they are horrible and inconsistent.

The worst of it is that there are some very specific (and undocumented) edge cases where the translation won't be done properly. It create very 'fun' to debug issues where a spreadsheet would work in a French Excel and not in an English one.

(For instance if you use the "Row-Column" cell reference, in English it is `R1C1` while in French `L1C1` - and it won't translate)

Re: I was wrong about spreadsheets (2017)

#182

I work in critical infrastructure planning. My organization builds software in R, Python, and other programming languages customized for these major organizations. So many critical infrastructures, billions of dollars in planning, and just systems are built out of Excel. It's amazing. You'd assume something that services millions of people a day would have some more sophisticated and customized solution, but you're w…

I just hate one single thing about Excel - all function names are localized - and in case of my native language - they are horrible and inconsistent.

[deleted]

Re: I was wrong about spreadsheets (2017)

#183
Spreadsheets are fine. Excel is not.

Excel conflates the ideas of data and presentation. This leads to an entire class of headaches that just aren't necessary. It's the desktop application equivalent of the string 'null'.

If the spreadsheet layer (calculations and formatting) was separate from the data layer (types and values) then we could all be happy.

Wrap that up with a UI that wasn't designed at an office in Redmond (or for a web browser) and you'd really have a winner.

Re: I was wrong about spreadsheets (2017)

#184

Earlier quoted context omitted.

While I haven't encountered localized names, localized formats make Excel an absolute pain for me. My language uses the "European" number format of a comma for decimals and periods for thousands separators. Excel tries to adjust to that by using semicolons for argument separators (i.e. ADD(1.5, 3.5) -> ADD(1,5; 3,5)). The problem is that their locale detection is wildly inconsistent and there isn't a good way to over…

Excel goes one step further, and assumes that users in some locales (e.g., Dutch) want semicolons as separators in every CSV file you open in Excel . I don't care (I just use LibreOffice Calc, which accepts any delimited values file just fine, and just asks which separator to assume), but it means that when you develop an option for users to download some statistical data as comma-separated values file (which is easy…

You can specify what separator to use as the first line in the CSV file:

sep=,

My experience is that Excel 2007 and later will correctly parse the file and use the specified separator. However, other software, such as Google Sheets, will simply render the declaration as-is.

Then there's the issue of what character encoding to use to encode the file, whether to include a byte-order-mark with UTF-8 to make Excel recognize the file as UTF-8 and the effect that has on whether the separator line is recognized (spoiler: it isn't).

Here's part of the documentation I wrote for Calcapp's CSV exporter, which digs into these issues in more detail (Javadoc):

  /**
   * The prologue of files containing comma-separated values (CSV). This
   * prologue contains an instruction detailing the separator character that is
   * used in the file. This instruction is known to be understood by Microsoft
   * Excel 2007 and later versions, but is rendered as-is by other spreadsheets,
   * including Google Sheets.
   * 

* Microsoft Excel expects either a comma or a semicolon to separate values in * CSV files, depending on the Windows locale. The only way to produce a CSV * file that can be read by Excel regardless of what locale Windows is set to * use is to use a prologue similar to this one, which explicitly tells Excel * which separator is used. */ private static final String PROLOGUE = "sep=" + SEPARATOR_CHARACTER + "\n"; /** * The character set used to encode files containing comma-separated values * (CSV): UTF-16LE (UTF-16 for little-endian systems). Using UTF-16LE allows * characters that cannot be represented by the ASCII character encoding to be * correctly read by Microsoft Excel and other spreadsheets. *

* There is no way to formally specify the character set used by a CSV file. * With one exception, Excel assumes that CSV files use the ASCII character * encoding, unless the first three bytes consist of a byte-order mark, in * which case the UTF-8 encoding is used. (A byte-order mark is redundant for * UTF-8, as it does not depend on endianness, but is traditionally used by * Microsoft Windows applications to detect whether a text file uses the UTF-8 * encoding.) *

* Excel only recognizes a file as being encoded with UTF-8 if a byte-order * mark is included, but doing so prevents Excel from recognizing the * information of the {@linkplain #PROLOGUE prologue}, which in turn prevents * CSV files from being produced which work regardless of the locale Windows * is set to use. This is likely due to a bug, present in Excel 2007 and * likely later versions as well (based on anecdotal evidence). *

* Fortunately, Excel does recognize another character set which can encode * all of Unicode: UTF-16. Excel likely uses heuristics to determine that a * file is encoded using UTF-16. (Text written using Western languages and * encoded using UTF-16 tend to include many null bytes for various reasons, * making the detection of UTF-16 trivial, but only for Western languages.) * Unfortunately, UTF-16 is dependent on endianness, meaning that it would be * desirable to include a byte-order mark at the beginning of the file. * However, that does not work due to the aforementioned bug. *

* In other words, using UTF-16LE should work well for CSV files containing * mostly Western text and parsed on little-endian systems. It is probable, * though, that files produced using this converter will not work if the text * mostly contains Chinese, Japanese or Korean characters or if the file is * parsed on a big-endian system. */ public static final Charset CHARACTER_SET;

Re: I was wrong about spreadsheets (2017)

#185
post #121

Earlier quoted context omitted.

while paying a monthly fee...

The monthly fee for Photoshop is around 1/100th of the old retail price. That works out at about 8 years of use before you have to 'buy' the software again. Seems fair to me. And for those who don't like paying for Photoshop - which, given it's an astonishingly powerful piece of software, probably means "people who don't actually need Photoshop" - there's always cheap or free alternatives that provide about half the…

> And for those who don't like paying for Photoshop - which, given it's an astonishingly powerful piece of software, probably means "people who don't actually need Photoshop" - there's always cheap or free alternatives that provide about half the functionality.

Now? Absolutely! 20 years ago this was much harder. Adobe used to dominate the market so that you used Photoshop even if you often didn't really need all its complexity. But these days, there is so much competition that you can probably find something that works well enough. I would assume that hurts their bottom line, but as a consumer, I am happy.

Re: I was wrong about spreadsheets (2017)

#186

Earlier quoted context omitted.

While I haven't encountered localized names, localized formats make Excel an absolute pain for me. My language uses the "European" number format of a comma for decimals and periods for thousands separators. Excel tries to adjust to that by using semicolons for argument separators (i.e. ADD(1.5, 3.5) -> ADD(1,5; 3,5)). The problem is that their locale detection is wildly inconsistent and there isn't a good way to over…

Excel goes one step further, and assumes that users in some locales (e.g., Dutch) want semicolons as separators in every CSV file you open in Excel . I don't care (I just use LibreOffice Calc, which accepts any delimited values file just fine, and just asks which separator to assume), but it means that when you develop an option for users to download some statistical data as comma-separated values file (which is easy…

Oh I didn't know that is a localization thing. I use the german version and got used to open csv's in notepad first (search and replace ; with ,). I always thought that's some kind of the usual MS vs. the rest of the world thing.

But this a whole other level of stupidity.

Re: I was wrong about spreadsheets (2017)

#187
post #72

Earlier quoted context omitted.

Meanwhile open any document in Photoshop with a few layers and effects and the computer grinds to a halt, no matter the specs, no matter the year, through the ages. New hardware comes out? Booya, new Photoshop XYZ -> let's put your fancy hardware on its knees, begging for air.

> Meanwhile open any document in Photoshop with a few layers and effects and the computer grinds to a halt, no matter the specs, no matter the year, through the ages. Hasn't been my experience ever. Photoshop is one of the speediest image manipulation programs out there...

I'd agree, at least when compared to GIMP. It's also like 100 times more intuitive to use Photoshop than GIMP for a beginner.

Re: I was wrong about spreadsheets (2017)

#188

It's interesting how often people's response to the manifest problems caused by people using Excel in unfortunate ways is to either suggest tools with much steeper learning curves or to suggest additional Excel features. 99% of these problems can be solved by proper training and standards appropriate to the problem and context at hand. For instance: Always separate and label inputs, calculations, and outputs. Documen…

Don't try to fix the user.

If we lived in a world where your suggestions were followed Excel would indeed be an OK tool. Here in the real world however Excel has just the right combination of power and usability to shoot off every left foot in a five cube radius, and frequently does.

Re: I was wrong about spreadsheets (2017)

#189
post #99
post #48

Just don't try to use dates before January 1, 1900 (or 1904 on Mac). You can work with dates in Excel 1,000 years in the future, but if you try to do that for dates from the 1800s it will completely screw them up. I don't understand why this problem still exists today. If you work in a museum, or anywhere else where you deal with old dates, you have to constantly be on the lookout for this "gotcha."

This is not directly related to issues you see, but might shed some light: https://www.joelonsoftware.com/2006/06/16/my-first-billg-rev... (plus this is one of the funnies Joel Spolsky's text and an amazing dive into computers history).

Thank you for sharing that, it's pretty interesting. I wouldn't have guessed that it was to maintain compatibility with Lotus 1-2-3.

Re: I was wrong about spreadsheets (2017)

#190
post #33

I work in critical infrastructure planning. My organization builds software in R, Python, and other programming languages customized for these major organizations. So many critical infrastructures, billions of dollars in planning, and just systems are built out of Excel. It's amazing. You'd assume something that services millions of people a day would have some more sophisticated and customized solution, but you're w…

On the same hardware, with the same table, Excel in Windows 7 does a cross-tab faster than MySQL in Ubuntu does. Indeed, MySQL chokes if there are more than a few hundred columns, but Excel just keeps going. And it uses all CPU cores. Edit: In case anyone is wondering why I did that, I wanted a simple visualization of ping-location results for thousands of IPv4 from several hundred ping-probe locations. So that meant…

I wonder how something like python/pandas would do on that.
Post reply on HN