Earlier quoted context omitted.
This isn't wildly different from what I've done but it's the sheer volume of crap that's scattered through all the different fields. The countries are the least of my problems. There are others where I'm faced with tens of millions of different combinations. The countries are a relatively trivial problem in comparison.
My solution will catch a lot of trivial errors like simple misspellings. You could have a relational tables with dozens of columns, each with its own dictionary; but that won't catch wrong combinations between columns. For example, there is a Paris, Texas; but I doubt there is a London, Texas. Dictionaries of state names and city names would not catch someone's error of matching the wrong city with a state when enter…
Re: Ask HN: Data engineers, What suck when working on exploratory data-related task?
#21Oh don't even get me started on cities, states, towns and addresses :D
https://www.google.com/maps/place/95450+Us,+France/@49.10531...