Live data from Hacker News

DuckDB as the New jq

pgrs.net

71–74 of 74 posts

Re: DuckDB as the New jq

#71
post #18

The most effective combination I've found so far is jq + basic shell tools. I still think jq's syntax and data model is unbelievably elegant and powerful once you get the hang of it - but its "standard library" is unfortunately sorely lacking in many places and has some awkward design choices in others, which means that a lot of practical everyday tasks - such as aggregations or even just set membership - are a lot m…

I do the same thing all the time, lol

but have in the last year or so, tried to start writing things that will be read/used by other people in python or java

why? because most people don't have a clue how bash scripts work, but can read/debug python and java with ease, and both can be run as shell scripts too

the mac has sqlite installed by default as well, so there's a powerful combo available on every mac

but I totally do what you're doing all the time, and it is my default go to if I have to get something done fast :D

Re: DuckDB as the New jq

#72
post #44
post #41

Earlier quoted context omitted.

You don't even need to use file() for a lot of things recently. These just work with clickhouse local. Even wildcards work. select * from `foobar.csv` or select * from `monthly-report-*.csv`

Just had to try: $ function _select_aux () { clickhouse local -q "SELECT $* FORMAT Pretty" } $ alias SELECT='noglob _select_aux' $ SELECT COUNT(*) as count FROM file('repos.json', JSON) ┏━━━━━━━┓ ┃ count ┃ ┡━━━━━━━┩ │ 30 │ └───────┘

This is very nice!! I went down the format clause[1] rabbit hole a couple minutes ago and it looks like they have `FORMAT Markdown` as an option. FORMAT PrettySpaceNoEscapes also looks a bit better than Pretty or the default PrettyCompact imho

1. https://clickhouse.com/docs/en/interfaces/formats

Re: DuckDB as the New jq

#73
post #44
post #41

Earlier quoted context omitted.

You don't even need to use file() for a lot of things recently. These just work with clickhouse local. Even wildcards work. select * from `foobar.csv` or select * from `monthly-report-*.csv`

Just had to try: $ function _select_aux () { clickhouse local -q "SELECT $* FORMAT Pretty" } $ alias SELECT='noglob _select_aux' $ SELECT COUNT(*) as count FROM file('repos.json', JSON) ┏━━━━━━━┓ ┃ count ┃ ┡━━━━━━━┩ │ 30 │ └───────┘

Where's this noglob command coming from? I haven't heard of it before.

Re: DuckDB as the New jq

#74
post #44

Earlier quoted context omitted.

Just had to try: $ function _select_aux () { clickhouse local -q "SELECT $* FORMAT Pretty" } $ alias SELECT='noglob _select_aux' $ SELECT COUNT(*) as count FROM file('repos.json', JSON) ┏━━━━━━━┓ ┃ count ┃ ┡━━━━━━━┩ │ 30 │ └───────┘

Where's this noglob command coming from? I haven't heard of it before.

Ah sorry that is probably zsh specific, it makes sure to not glob expand * etc
Post reply on HN