PartiQL: One query language for all your data • Hacker News Board

Re: PartiQL: One query language for all your data

#81

Post by _msw_ » Sat, Aug 03, 2019, 1:10 AM UTC

ahl wrote: post #77

@dlurton since you seem to be speaking for the PartiQL team on this (congrats on the launch!): The reference implementation is open source; what's the plan for the language spec? Is that something that AWS is going to own and control? The website references the PartiQL Steering Committee -- is that just AWS folks or is the intention to make it more broadly composed of members of the community you build? I'm intereste…

Disclosure: I work for AWS and provided some opinions and non-prescriptive advice to the team behind PartiQL about Open Source.

The same question was raised on Twitter, and I put my thoughts there: https://twitter.com/_msw_/status/1157405984823758848

TL;DR, my advice is that successful open source projects and open specifications usually have diverse communities. You will have a hard time attracting people to your community of they do not share goals with the rest of the community. We should have some bounding boxes around how the spec evolves through clear tenets. Otherwise welcome diverse opinions, experience, and problems to solve collaboratively.

Re: PartiQL: One query language for all your data

#82

Post by yannisGP » Sat, Aug 03, 2019, 2:43 AM UTC

ahl wrote: post #77

@dlurton since you seem to be speaking for the PartiQL team on this (congrats on the launch!): The reference implementation is open source; what's the plan for the language spec? Is that something that AWS is going to own and control? The website references the PartiQL Steering Committee -- is that just AWS folks or is the intention to make it more broadly composed of members of the community you build? I'm intereste…

hi @ahl, I'm a member of PartiQL's steering committee and glad to see your interest to participate in PartiQL's evolution. The language spec source will be open-sourced, as well, early next week (week of Aug 5). Overall, we look forward to a community effort and participants that are interested in making significant investments to achieve the project's goals. Diverse opinions and viewpoints, both on the language and on the process, are very welcome.

At this point, the maintainers/committee is only Amazon members. As PartiQL grows towards a diverse community, we expect to add maintainers/committee (for code and spec) that have non-Amazon affiliations and explore more formalized methods of governance,as they will emerge from our community discussions.

Please email us at partiql-committee@amazon.com to further coordinate.

Re: PartiQL: One query language for all your data

#83

Post by recursive » Sat, Aug 03, 2019, 3:00 AM UTC

pvg wrote: post #80

recursive wrote: post #79

Earlier quoted context omitted.

I don't know much about compilers, but basically the ones don't have anywhere else to put the context of a closure.

I don't think that's what happens in the JVM, you can google up the invokevitual + other supporting gunk/mechanism for this.

https://docs.oracle.com/javase/specs/jvms/se7/html/jvms-6.ht... makes plenty of references to dispatching based on class.

Re: PartiQL: One query language for all your data

#84

Post by yannisGP » Sat, Aug 03, 2019, 3:02 AM UTC

xpe wrote: post #63

A common query language, while appealing, is unlikely to fully abstract over different types of databases with different features and performance trade offs. It will be a leaky abstraction. Now, in practice, perhaps with sufficient adoption and integration, PartiQL might be good enough for 80% of use cases.

(I'm part of the PartiQL effort.) You are right about the challenge you point out and we are realistic about it. Thus this line in the charter: {{{ While the adopting query engines generally may not support all features of PartiQL, a database engine that “supports PartiQL” is expected to be consistent with the PartiQL specification in the syntax subset it supports. }}}

Re: PartiQL: One query language for all your data

#85

Post by yannisGP » Sat, Aug 03, 2019, 3:07 AM UTC

kodablah wrote: post #15

Is there a specification in anything besides PDF easily available to link to?

(I'm a member of the PartiQL team.) The language spec source will be open-sourced, as well, early next week (week of Aug 5). As I said above, to @ahl: Overall, we look forward to a community effort and participants that are interested in making significant investments to achieve the project's goals. We invite diverse opinions and viewpoints. As PartiQL grows towards a diverse community, we expect to add maintainers (for code and spec) that have non-Amazon affiliations and explore more formalized methods of governance.

Re: PartiQL: One query language for all your data

#86

Post by pvg » Sat, Aug 03, 2019, 3:11 AM UTC

recursive wrote: post #83

pvg wrote: post #80

Earlier quoted context omitted.

I don't think that's what happens in the JVM, you can google up the invokevitual + other supporting gunk/mechanism for this.

https://docs.oracle.com/javase/specs/jvms/se7/html/jvms-6.ht... makes plenty of references to dispatching based on class.

D'oh, sorry I sent you on a wild goose chase - you want invokedynamic. Invokedynamic and lambdas.

Re: PartiQL: One query language for all your data

#87

Post by yannisGP » Sat, Aug 03, 2019, 3:17 AM UTC

ohnoesjmr wrote: post #6

I wonder how this deals with nested parquet data, and whether it's able to optimise on the things parquet provides.

(PartiQL team member) AWS Redshift Spectrum supports PartiQL on parquet since last year. Except that the language had not had a name yet and was referred to as "SQL extensions for nested data.

Re: PartiQL: One query language for all your data

#88

Post by vgt » Sun, Aug 04, 2019, 4:09 AM UTC

monsieurbanana wrote: post #68

jiggawatts wrote: post #62

Earlier quoted context omitted.

You missed the point. Modern infrastructure's scalability is irrelevant if even one user's experience is poor. In the era of 64-core processors, scaling horizontally is meaningless for 99.9% of architecture designs. Latency matters to everyone, always. Trivial queries taking nearly 1/10th of a second on modern kit is absolutely atrocious, and shows a total lack of awareness of performance as a feature.

> Latency matters to everyone, always. I used to do a lot of BigQuery for analytics. Latency in BigQuery is crap and clearly not it's selling point, we're not talking ms here, we're talking seconds at a minimum. Yet it's a really nice database for it's use cases.

We've improved our floor latency by a factor of 5 since two years ago, we've introduced clustering, and newly introduced BI engine gets you into dozen digit millisecond range, so give it a try again :)

(Product manager on BigQuery)