61–70 of 110 posts
Instead of the gawk manual, I recommend the OpenBSD manpage: https://man.openbsd.org/awk.1 It's concise and well written (as most OpenBSD manpages), so you can learn AWK even in less than 2 hours. Of course it handles only AWK, not the gawk extensions.
Standard awk warning: it's tempting to try to use awk on csv files. You'll even get good results on simple csv files that leave you encouraged to go further. Don't. Csv is not standardized and the quoting rules are weird (and not standardized). If you can live with a certain amount of loss of fidelity in your output, you can get away with using awk. If you want a coarse prototype, use awk. If you need robust, product…
Segue to something that I've been fighting with for years: Do you have a suggestion for working with CSVs in the CLI? Something that will show a table, with a single pixel border between cells, that allows searching for a value, copying a cell?
csvlook -I | less -S
or
csvformat -T | less -S
from csvkit - https://csvkit.readthedocs.io/en/1.0.2/scripts/csvlook.html
Tons of useful CSV tools in there - csvcut, csvjoin, csvgrep.
csvformat -T
Standard awk warning: it's tempting to try to use awk on csv files. You'll even get good results on simple csv files that leave you encouraged to go further. Don't. Csv is not standardized and the quoting rules are weird (and not standardized). If you can live with a certain amount of loss of fidelity in your output, you can get away with using awk. If you want a coarse prototype, use awk. If you need robust, product…
Standard awk warning: it's tempting to try to use awk on csv files. You'll even get good results on simple csv files that leave you encouraged to go further. Don't. Csv is not standardized and the quoting rules are weird (and not standardized). If you can live with a certain amount of loss of fidelity in your output, you can get away with using awk. If you want a coarse prototype, use awk. If you need robust, product…
There is https://tools.ietf.org/html/rfc4180 Most CSV files do not follow this standard of course. But you could normalize all CSV files to RFC4180 (or any other consistent format) as the first step of your processing pipeline.
Standard awk warning: it's tempting to try to use awk on csv files. You'll even get good results on simple csv files that leave you encouraged to go further. Don't. Csv is not standardized and the quoting rules are weird (and not standardized). If you can live with a certain amount of loss of fidelity in your output, you can get away with using awk. If you want a coarse prototype, use awk. If you need robust, product…
I would love to have a command-line tool that reads CSV and has a ton of features to cover different quirks and errors, which can output cleaner formats that I can pipe into other command-line tools. csvkit [0] might be that tool; I discovered it after my last painful encounter with CSV files and haven't used it in anger yet. Among other things, it translates CSV to JSON, so you can compose it with jq. [0] https://cs…
[0]: https://bitbucket.org/rbr/csvtools
[1]: https://en.wikipedia.org/wiki/Delimiter#ASCII_delimited_text
[2]: https://www.gnu.org/software/gawk/manual/html_node/awk-split...
[3]: https://www.gnu.org/software/gawk/manual/html_node/Field-Sep...
> He said, “If you get the awk programming language manual…you’ll read it in about two hours and then you’re done. That’s it. You know all of awk.” It's hyperlinked to the Gawk manual, but it seems likely he actually meant A, W & K's The Awk Programming Language (1988), which you could conceivably read in 2 hours, as it's a joy to read. I used it and The C Programming Language as exemplars of great documentation when…
Standard awk warning: it's tempting to try to use awk on csv files. You'll even get good results on simple csv files that leave you encouraged to go further. Don't. Csv is not standardized and the quoting rules are weird (and not standardized). If you can live with a certain amount of loss of fidelity in your output, you can get away with using awk. If you want a coarse prototype, use awk. If you need robust, product…
Tangent, but another interesting problem with date parsing besides lack of standardization is that it's ridiculously context-sensitive. For example, if I schedule an event for Oct 1-Jan 1 right now, the most likely (least surprising) parse would be Oct 1, 2021 through January 1, 2022. Which is both surprising because parsing Oct 1 depends on the current date, and because the parsing of Jan 1 depends on the parse of O…
If your CSV file contains any field entered by humans AWK isn't going to be powerful enough to parse it at scale. Someone somewhere is going to have the name 'Mbat"a, Sho,dlo' in some bizarre ass romanization (and this assume you're not accepting Unicode, which is a whole other can of worms that AWK is not prepared to deal with) that breaks your parser.
Standard awk warning: it's tempting to try to use awk on csv files. You'll even get good results on simple csv files that leave you encouraged to go further. Don't. Csv is not standardized and the quoting rules are weird (and not standardized). If you can live with a certain amount of loss of fidelity in your output, you can get away with using awk. If you want a coarse prototype, use awk. If you need robust, product…
Everything you just described seems like reasons to not use CSV files in general, rather than not use AWK on CSV.
Standard awk warning: it's tempting to try to use awk on csv files. You'll even get good results on simple csv files that leave you encouraged to go further. Don't. Csv is not standardized and the quoting rules are weird (and not standardized). If you can live with a certain amount of loss of fidelity in your output, you can get away with using awk. If you want a coarse prototype, use awk. If you need robust, product…
I would love to have a command-line tool that reads CSV and has a ton of features to cover different quirks and errors, which can output cleaner formats that I can pipe into other command-line tools. csvkit [0] might be that tool; I discovered it after my last painful encounter with CSV files and haven't used it in anger yet. Among other things, it translates CSV to JSON, so you can compose it with jq. [0] https://cs…