Earlier quoted context omitted.
> So the built in csv module needs to support a reasonable behavior. Which it does not. It could be argued that the csv module's behaviour is reasonable, and NumPy's isn't. (I'm not 100% sure about all the details of this issue) Hopefully, NumPy will change it's behaviour to match Python 3, but if not you could still use the NumPy CSV routines like `loadtxt` or `genfromtxt` [0]. So then this becomes a documentation c…
I say this as someone who uses the latest version of Python available in every new project or script. Text encoding issues are absolute garbage in Python 3.x I fucking hate the way that csv module works with text encodings. As soon as I can figure out a reliable way to take latin-1 and save it as UTF-8 without breaking everything, I will try to shoehorn in a PR. Right now, it's fucking awful. My ETL pipeline hates it…
with open('some latin-1 file', 'rb) as f:
text = f.read().decode('latin-1')
with open('some utf8 file', 'wb') as f:
f.write(text.encode('utf-8'))
Python 3's string encoding support is super good. I've said it before and I'll say it again: if you use bytes as a string you are Doing It Wrong.If you use bytes as a string you are Doing It Wrong.
If you use bytes as a string you are Doing It Wrong.