Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Norway Problem

hitchdev.com

Post reply on HN
Print view View original

331–339 of 339 posts

Re: The Norway Problem

#331

Post by atoav »

simion314 wrote: post #22
atoav wrote: post #9

Earlier quoted context omitted.

I'd go further and say this is why you write tests . Creating tests that cover a lot (or all) possible inputs is sometimes not that hard and really pays off if you manage to catch a very common error like the Norway thing. Even better if you catch something that would have been a nightmare to fix in production. I say this because two days ago I wrote a test that used all country codes as input. It took 15 minutes to…

>I say this because two days ago I wrote a test that used all country codes as input. It took 15 minutes to write that test. During the whole testing session I found at least 5 mistakes of which 3 would have been quite dramatic. And how many minutes to test all city/state/region/street/person names ? It can also happen that you test s will become outdated, like when url standard changed and more characters codes were…

For something like URLs I'd use the hypothesis Python module and rely on their implementation of URLs (and if that changes the test will fail for newly formated URLs), for everything "custom" I would extract problematic test cases and include them as examples.

Testing doesn't take too long on my machine (maybe 10 seconds), but even if it would, it would be totally acceptable as I run it pre commit only.

Re: The Norway Problem

#332

Post by alexvoda »

andrepd wrote: post #117
keeperofdakeys wrote: post #25

This is part of more general problem, they had to rename a gene to stop excel auto-completing it into a date. https://www.theverge.com/2020/8/6/21355674/human-genes-renam... Edit: Apparently Excel has its own Norway Problem ... https://answers.microsoft.com/en-us/msoffice/forum/msoffice_...

I'd say the more general problem is a bad type system! In any language with a half decent type system where you can define `type country = Argentina | ... | Zambia` this would be correctly handled at compile-time, instead of having strange dynamic weak typing rules (?) which throw runtime errors in production (???).

I would like to see how your solution handles the case of new countries or countries changing name. Recompile and push an update? If the environment is governmental this can take a very very very long time.

The proper solution, in my opinion, is a lookup table stored in the database. It can be updated, it can be cached, it can be extended.

And for transfer of data, use formats to which you can attach a schema. This way type data is not lost on export. XML did this but everyone hates XML. And everyone hates XSD (the schema format) even more. However, if you use the proper tools with it, it is just wonderful.

Re: The Norway Problem

#333

Post by spion »

goatinaboat wrote: post #216
teddyh wrote: post #211

Earlier quoted context omitted.

Ctrl-\, Ctrl-], Ctrl-^ and Ctrl-_ for file, group, record and unit separator, respectively. However, your tty driver, terminal or program are all likely to eat them or munge them. Also, virtually nothing actually uses these characters for these purposes.

virtually nothing actually uses these characters for these purposes. Right. Which is why we have all these hilarious escaping and interpolation problems. Any why programmers will never be taken seriously by real engineers. It's like we have cement mixed and ready to go but we decide to go and forage for mud instead and think that makes us cleverer than the cement guys.

> your tty driver, terminal or program are all likely to eat them or munge them

Maybe that has something to do with this?

Re: The Norway Problem

#334

Post by [deleted] »

djedr wrote: post #242
atombender wrote: post #191

The world desperately needs a replacement for YAML. TOML is fine for configuration, but not an adequate solution for representing arbitrary data. JSON is a fine data exchange format, but is not particularly human-friendly, and is especially poor for editable content: Lacks comments, multi-line strings, is far too strict about unimportant syntax, etc. Jsonnet (a derivative of Google's internal configuration language)…

Still early, but here's my baby I hope can improve things: website with grammar spec: https://tree-annotation.org/ prototype of a JSON/YAML alternative for JS: https://github.com/tree-annotation/tao-data-js same thing, even less finished for C#: https://github.com/tree-annotation/tao-data-csharp working on it constantly, more to come soon

[deleted]

Re: The Norway Problem

#335

Post by Arrath »

imtringued wrote: post #319
mattkrause wrote: post #159

Earlier quoted context omitted.

The analysis itself isn’t (usually) happening in Excel. A lot of tools operate on CSV files. People use Excel to peek at the results or prepare input for other tools, and that’s how the date coercion slips in. Sometimes, people do use it to collate the results of small manual experiments, where a database might be overkill. Even so, the data is usually analyzed elsewhere (R, graphPad, etc).

>A lot of tools operate on CSV files. The mistake was to believe that Excel can operate on CSV files. It doesn't support them in any meaningful way. It supports them in a "I can sort of pretend that I support CSV files" way.

What is a good alternative to working with CSV files than Excel? Excel sure isn't ideal but it's always there as part of the MS Office suite, so I've never looked for anything esle.

Re: The Norway Problem

#336

Post by blacktriangle »

imtringued wrote: post #318
njarboe wrote: post #170

Earlier quoted context omitted.

Yes. Excel cells are set to a "General" format that, by default, tries to guess the type of data the cell should be from its content. A date looking entry gets converted to a date type. A number looking string to a number (so 5.80 --> 5.8, very annoying since I believe in significant digits) When you import cvs data, for example, the default import format is "General" so date looking strings will be changed to a date…

Excel doesn't support CSV files. Anyone who believes that has never used Excel. [0] You're supposed to use spreadsheets as is. Programs that have excel export features should always directly export xlsx files. [0] The only thing you can safely do with CSV files is to interpret every value as text cell. CSV files always require out of band negotiation on everything, including delimiters, quotation, escape characters,…

However....

Users BELIEVE Excel supports CSV file. That's the reality on the ground. Fighting against that is a losing battle.

Re: The Norway Problem

#338

Post by zomglings »

BlueTemplar wrote: post #324
zomglings wrote: post #322

Earlier quoted context omitted.

Because we were not using any numerical properties of the hash. We were not adding it to other hashes, seeing if it was greater than or less than other hashes, etc. Literally the only thing we were doing was passing it between shell commands, helm charts, Kubernetes deployments and then back (if we needed to debug). It sounds like you have a more attractive alternative in this case than to treat hashes as strings. Wo…

In the same thread : https://news.ycombinator.com/item?id=26679590 Concise encoding seems to have an hex-int type ?

Problem is that we were using Kubernetes + helm, which means that we were bound to whatever configuration language they use.

Re: The Norway Problem

#339

Post by BlueTemplar »

zomglings wrote: post #338
BlueTemplar wrote: post #324

Earlier quoted context omitted.

In the same thread : https://news.ycombinator.com/item?id=26679590 Concise encoding seems to have an hex-int type ?

Problem is that we were using Kubernetes + helm, which means that we were bound to whatever configuration language they use.

So the fault is on them.
Post reply on HN