51–60 of 102 posts
Earlier quoted context omitted.
Ah, I thought you were going to link to the fascinating skunkworks Log Parser tool https://www.microsoft.com/en-us/download/details.aspx?id=246... Wonder if it still runs on Windows 10?
If you need a UI, don't forget the tool: Log Parser Lizard ^^ http://www.lizard-labs.com/log_parser_lizard.aspx
Another trick along these lines is to cut out the middleman (middleprogram? middleware?) and use SQLite to do it: $ sqlite3 sqlite3> .mode csv sqlite3> .import foo.csv foo sqlite3> SELECT * FROM foo WHERE bar = 'baz'; -- you get the gist... (a bunch of rows) Q seems to be much easier to use (and certainly easier to remember), but I've always found it handy to have the full power of SQL at my fingertips when needing t…
Real-world CSV files generally contain some or all of the following horrors: - some strings enclosed in speechmarks, but some not - empty fields - speechmarks within strings - commas within strings - carriage returns within strings How does Q do up against a CSV file with those traits?
All of your “horrors” seem...correct? Its comma delimited, so anything that is between two commas should be parsed without issue; if it’s a string with a comma in it, and unquoted, you simply have a broken csv file. If its quoted, than anything until the next (unescaped) quote is fine, including commas Unless you’re trying to parse csv files with regexes, none of those should be difficult, or even unexpected, to hand…
Many people are not in the position to just return the file to the client/boss and tell them they have a "broken csv file". (They'll tell you they saved it in Excel and it reads back fine, so the problem must be on your end. E.g.: https://stackoverflow.com/questions/43273976/escaping-quotes...)
“Any file is a database if you awk hard enough.” —Bryan Horstmann-Allen https://twitter.com/neilkod/status/914217352564137984 Furthermore, no thread on CSV files can be complete without mentioning this infamous bit of fact-trolling: the ASCII standard has had unit- and record-delimiters baked into it from the beginning. https://ronaldduncan.wordpress.com/2009/10/31/text-file-form...
Looks like PostgreSQL has a foreign data wrapper for CSV. https://wiki.postgresql.org/wiki/Foreign_data_wrappers In fact, seems it has several.
https://en.m.wikipedia.org/wiki/Q_%28programming_language_fr...
After you're done preparing your data in PowerQuery you can run PowerPivot on it for aggregations.
[0]: https://cdn-5a6cb102f911c811e474f1cd.closte.com/wp-content/u...
Veeeery nice for data munging :)
http://fsharp.github.io/FSharp.Data/library/CsvProvider.html
