Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google's new pipe syntax in SQL

simonwillison.net

Post reply on HN
Print view View original

131–140 of 192 posts

Re: Google's new pipe syntax in SQL

#131

Post by _Wintermute »

dan-robertson wrote: post #119
minkles wrote: post #115

That is basically R with tidyverse. flights |> filter( carrier == "UA", dest %in% c("IAH", "HOU"), sched_dep_time > 0900, sched_arr_time group_by(flight) |> summarize( delay = mean(arr_delay, na.rm = TRUE), cancelled = sum(is.na(arr_delay)), n = n() ) |> filter(n > 10) If you haven't used R, it has some serious data manipulation legs built into it.

An interesting thing to me about all these dplyr-style syntaxes is that Wickham thinks the group_by operator was a design mistake. In modern dplyr you can often specify a .by on an operation instead. I found switching to this style a pretty easy adjustment, and I think it’s a bit better. Example: d |> filter(id==max(id),.by=orderId) I think PRQL were thinking a bit about ways to avoid a group_by operation and I think…

Given 10 more years dplyr syntax might resemble data.table's

Re: Google's new pipe syntax in SQL

#132

Post by ahmed_ds »

This is why I like tools like datastation and hex.tech. You write the initial query using SQL than process the results as a dataframe using Python/pandas. Surely, mixing Pandas and SQL like that is not good for data pipelines but for exploration and analytics, I have found this approach to be enjoyable.

Re: Google's new pipe syntax in SQL

#133

Post by foldr »

maxbond wrote: post #43
anitil wrote: post #38

Earlier quoted context omitted.

It's so ambitious in an almost boring way, exactly the right steward for a project like this

Dr. Hipp is one of my heroes. He seems to labor quietly in semi obscurity for decades, and at the end of it he's produced some amazing software. I was tickled by the curfuffle over his use of a set of guidelines for living in a Christian monastery as SQLite's code of ethics for the purpose of checking a box on an RFQ (part of the fallout of the libsql fork), because he does seem like a sort of programmer monk. (For w…

In fairness, I think the complaint over the tongue-in-cheek 'code of conduct' was that it was transparently unsuitable if considered as an actual code of conduct (i.e. a list of rules that SQLite contributors must obey in order to participate in the project). For example, it seems unlikely that Dr. Hipp would wish to exclude contributors who have committed adultery, or who do not pray with sufficient frequency.

(The erstwhile code of conduct is now labeled a 'code of ethics', and AFAIK SQLite has no official CoC currently.)

Re: Google's new pipe syntax in SQL

#134

Post by spatulon »

crazygringo wrote: post #56
aragonite wrote: post #47

> This remains a long-standing pet peeve of mine. PDFs like this are horrible to read on mobile phones, hard to copy-and-paste from ... I've never understood why copying text from digitally native PDFs (created directly from digital source files, rather than by OCR-ing scanned images) is so often such a poor experience. Even PDFs produced from LaTex often contain undesirable ligatures in the copied text like fi and fl.…

> Is it due to something inherent in PDF technology? Exactly. PDF doesn't have instructions to say "render this paragraph of text in this box", it has instructions to say "render each of these glyphs at each of these x,y coordinates". It was never designed to have text extracted from it. So trying to turn it back into text involves a lot of heuristics and guesswork, like where enough separation between characters sho…

I've never looked into the PDF format, but, does it not allow for annotations that say, "the glyphs in the rectangle ((x0, y0), (x1, y1)) represent the text 'foobar'")? That's been my mental model for how they are text-searchable.

Re: Google's new pipe syntax in SQL

#135

Post by quectophoton »

bvrmn wrote: post #128
andrewshadura wrote: post #90

Earlier quoted context omitted.

You can't "just" extend the IPv4 address space while keeping the compatibility.

Extending src/dst in current IPv4 protocol headers is much easier than adopting a completely new suite.

> Extending src/dst in current IPv4 protocol headers is much easier than adopting a completely new suite.

And that's precisely why that was also one of the competing proposals back then, so that tells me that just being easier probably wasn't enough.

You can search for RFC 1475 ("IPv7") and its surrounding history.

Re: Google's new pipe syntax in SQL

#136

Post by maxbond »

foldr wrote: post #133
maxbond wrote: post #43

Earlier quoted context omitted.

Dr. Hipp is one of my heroes. He seems to labor quietly in semi obscurity for decades, and at the end of it he's produced some amazing software. I was tickled by the curfuffle over his use of a set of guidelines for living in a Christian monastery as SQLite's code of ethics for the purpose of checking a box on an RFQ (part of the fallout of the libsql fork), because he does seem like a sort of programmer monk. (For w…

In fairness, I think the complaint over the tongue-in-cheek 'code of conduct' was that it was transparently unsuitable if considered as an actual code of conduct (i.e. a list of rules that SQLite contributors must obey in order to participate in the project). For example, it seems unlikely that Dr. Hipp would wish to exclude contributors who have committed adultery, or who do not pray with sufficient frequency. (The…

To me it seemed like they had incompatible visions (SQLite wants to work in 2050 in the contexts it's been traditionally used in, libsql wants to modernize and lean into the more recent use cases) and so a fork was the appropriate and inevitable course of action.

Given that SQLite isn't really open to contribution (one of libsql's frustrations) it doesn't really worry me that they didn't & don't have a clear code of conduct. To me, digging through the repository [ETA: the website, rather] for what amounts to a cringey Easter egg and then linking to it as if it were a serious issue is uncalled for. To be honest, I think the complaints shouldn't stayed out of their announcement entirely - they have a legitimately cool vision for what their fork could be, and the complaints were only a distraction.

Re: Google's new pipe syntax in SQL

#137

Post by countrymile »

minkles wrote: post #115

That is basically R with tidyverse. flights |> filter( carrier == "UA", dest %in% c("IAH", "HOU"), sched_dep_time > 0900, sched_arr_time group_by(flight) |> summarize( delay = mean(arr_delay, na.rm = TRUE), cancelled = sum(is.na(arr_delay)), n = n() ) |> filter(n > 10) If you haven't used R, it has some serious data manipulation legs built into it.

My thoughts exactly, it even uses the same pipe syntax, though I do prefer `%>%`. I've been avoiding SQL for a while now as it feels so clunky next to the tidyverse

Re: Google's new pipe syntax in SQL

#138

Post by bvrmn »

quectophoton wrote: post #135
bvrmn wrote: post #128

Earlier quoted context omitted.

Extending src/dst in current IPv4 protocol headers is much easier than adopting a completely new suite.

> Extending src/dst in current IPv4 protocol headers is much easier than adopting a completely new suite. And that's precisely why that was also one of the competing proposals back then, so that tells me that just being easier probably wasn't enough. You can search for RFC 1475 ("IPv7") and its surrounding history.

Yes I know. And IPv6 win because it's an objectively a superior standard. No politics and all committee garbage of course.

Re: Google's new pipe syntax in SQL

#139

Post by foldr »

maxbond wrote: post #136
foldr wrote: post #133

Earlier quoted context omitted.

In fairness, I think the complaint over the tongue-in-cheek 'code of conduct' was that it was transparently unsuitable if considered as an actual code of conduct (i.e. a list of rules that SQLite contributors must obey in order to participate in the project). For example, it seems unlikely that Dr. Hipp would wish to exclude contributors who have committed adultery, or who do not pray with sufficient frequency. (The…

To me it seemed like they had incompatible visions (SQLite wants to work in 2050 in the contexts it's been traditionally used in, libsql wants to modernize and lean into the more recent use cases) and so a fork was the appropriate and inevitable course of action. Given that SQLite isn't really open to contribution (one of libsql's frustrations) it doesn't really worry me that they didn't & don't have a clear code of…

Yes, it's an important point that SQLite is not a project with an open contribution model. However, they do presumably accept external contributions in the form of bug reports, suggested patches, etc. etc.

You didn't have to dig through the repository to find the CoC. It was right there on the website at /codeofconduct.html: https://web.archive.org/web/20180315125217/https://www.sqlit...

Re: Google's new pipe syntax in SQL

#140

Post by maxbond »

foldr wrote: post #139
maxbond wrote: post #136

Earlier quoted context omitted.

To me it seemed like they had incompatible visions (SQLite wants to work in 2050 in the contexts it's been traditionally used in, libsql wants to modernize and lean into the more recent use cases) and so a fork was the appropriate and inevitable course of action. Given that SQLite isn't really open to contribution (one of libsql's frustrations) it doesn't really worry me that they didn't & don't have a clear code of…

Yes, it's an important point that SQLite is not a project with an open contribution model. However, they do presumably accept external contributions in the form of bug reports, suggested patches, etc. etc. You didn't have to dig through the repository to find the CoC. It was right there on the website at /codeofconduct.html: https://web.archive.org/web/20180315125217/https://www.sqlit...

Another cool project from Dr. Hipp is the fossil SCM, which SQLite is developed in, and one of it's features is that it ships with a web view similar to GitHub. The website is actually the web view of the repo. (Apologies for expressing that in a confusing way, I knew it was on the website, I was referring to the website as the repository.)
Post reply on HN