Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Sq.io: jq for databases and more

sq.io

Post reply on HN
Print view View original

71–80 of 134 posts

Re: Sq.io: jq for databases and more

#71

Post by wvh »

nashashmi wrote: post #65

> sq is pronounced like seek. Its query language, SLQ, is pronounced like sleek As a person who is apart from the tech scene, and lurks in the tech space out of interest, I appreciate this guidance. For the longest time I didn’t know nginx was pronounced Engine-X; I called it N-jinx.

Don't sweat it. It's a running joke amongst guitar players no two people pronounce D'Addario the same way, not to mention the tremolo bar which technically should be called a vibrato bar. I surmise any scene has its trip-up words.

Re: Sq.io: jq for databases and more

#72

Post by seanhunter »

mschuster91 wrote: post #51
dewey wrote: post #48

Sometimes I wonder if it wouldn't be more efficient for people to just learn SQL instead of trying to build tools or layers on top of it that introduce more complexities and are harder to search for.

SQL interop is where the pain is at. Using standard tooling of most database systems, best you can get is CSV with all the pains this shithole of a data transfer format brings.

That doesn't match my experience at all. Every database I'm aware of has multiple options for data export and import for batch and command processing in the standard tooling. CSV is almost never "the best you can get". What is going to be best depends on your use case, and indeed at a bare minimum you can almost always change the field and line seperators and get something that works just fine on the commandline and avoids most of the stuff people find hard about CSV.

Additionally, if people like jq, they don't need a special tool like this. They can just get the database to output json and use json/jq - another tool isn't needed. In postgres you can do something like this

   select array_to_json(array_agg(row_to_json (r))) from (
      ... put your sql query here...
   ) r;
...and postgres will output a json array for you where each item in the array is a json map of a single row of the output.

I'm sure other databases have similar functionality.

Re: Sq.io: jq for databases and more

#73

Post by seanhunter »

mschuster91 wrote: post #70
itohihiyt wrote: post #68

Earlier quoted context omitted.

What's wrong with CSV? I love me a CSV file.

Out of the random shit I had to deal with CSV file wrangling in the last two years: - no defined encoding, so it may be anything from US-ASCII over ISO-8859-x to UTF-8 and it's always a guesswork what it actually is - no definition of the separator sign, usually it can be a literal comma, a semicolon, or tabs - escaping of the separator sign in column values is always fun - escaping of newlines is even MORE fun - lin…

Most of these issues while real don't actually arise in this case, because we're not trying to ETL some random file, we are the ones talking to the database so we get to choose exactly how the data gets formatted on extract.

For example, here's your list fully handled in postgres: 1. SET CLIENT_ENCODING TO 'value'; (eg 'UTF8') 2. COPY ... with FORMAT CSV DELIMITER 'delimiter_character' QUOTE 'quote_character'

Now the output format is fully specified and everything just works fine (including for input into excel)

   The values in each record are separated by the DELIMITER character. If the value contains the delimiter character, the QUOTE character, the NULL string, a carriage return, or line feed character, then the whole value is prefixed and suffixed by the QUOTE character, and any occurrence within the value of a QUOTE character or the ESCAPE character is preceded by the escape character.
https://www.postgresql.org/docs/current/sql-copy.html

Re: Sq.io: jq for databases and more

#74

Post by adammarples »

itohihiyt wrote: post #68
mschuster91 wrote: post #51

Earlier quoted context omitted.

SQL interop is where the pain is at. Using standard tooling of most database systems, best you can get is CSV with all the pains this shithole of a data transfer format brings.

What's wrong with CSV? I love me a CSV file.

Having the control characters mixed in with the data, and having no defined encoding.

Re: Sq.io: jq for databases and more

#75

Post by ddispaltro »

6LLvveMx2koXfwn wrote: post #63
dlisboa wrote: post #58

Earlier quoted context omitted.

> It's a weird trend that seems almost entirely motivated by people wanting open source projects in their resume That’s really harsh and misguided. If people didn’t do “paper thin abstractions” projects on their own time for the simple pleasure of doing it we wouldn’t have 90% of the successful projects we have today. Let people have fun and don’t judge their motives when they’re making something Open Source. I can g…

Unless you're the GP your guarantee about the persons motivation is as meaningless as the post you're replying to.

I think his point is that we should treat something given freely, charitably

Re: Sq.io: jq for databases and more

#76

Post by fforflo »

dewey wrote: post #48

Sometimes I wonder if it wouldn't be more efficient for people to just learn SQL instead of trying to build tools or layers on top of it that introduce more complexities and are harder to search for.

Paraphrasing Spencer:

Those who don't understand SQL are doomed to reinvent it, poorly

Re: Sq.io: jq for databases and more

#78

Post by nikolamus »

Xenoamorphous wrote: post #60
dewey wrote: post #48

Sometimes I wonder if it wouldn't be more efficient for people to just learn SQL instead of trying to build tools or layers on top of it that introduce more complexities and are harder to search for.

It's funny because recently a "full stack" dev who's in reality 95% frontend was telling me he's not a fan of Tailwind and that he'd rather learn "proper" CSS. And the irony is of course, he never wants to use a relational DB to avoid SQL, so No-SQL DB it is.

Wat

Re: Sq.io: jq for databases and more

#79

Post by School-Cotton »

dewey wrote: post #56
zmmmmm wrote: post #54

Earlier quoted context omitted.

I think it'd be a moot point if SQL wasn't painful and awkward to work with in the first place. But database purists control it and won't let go, so we will have to live with everyone else inventing layers to make their lives easier.

For me this feels like the complaints about error handling in Go. People who work with it all the time, don't even think about it past the first week. If you are starting out it might bother people because they are not used to it. Personally I really like working with SQL and find it quite elegant, I always encourage people to use it as it really is a job-superpower if you can just dig up issues directly in the DB qu…

I spent 5 years working on a SQL database (materialize.com) and I think SQL is awful.

https://www.scattered-thoughts.net/writing/against-sql (by another former Materialize employee) is a good takedown.

Re: Sq.io: jq for databases and more

#80

Post by krosaen »

dewey wrote: post #48

Sometimes I wonder if it wouldn't be more efficient for people to just learn SQL instead of trying to build tools or layers on top of it that introduce more complexities and are harder to search for.

I know sql pretty well and still find value in this kind of tool - creating schemas and inserting data is a clunky part of sql - the query language is where it really shines. So I can imagine using this to quickly insert some data or to get familiar with the schema and then dive in with normal sql queries.
Post reply on HN