Faster hash joiner with vectorized execution • Hacker News Board

Re: Faster hash joiner with vectorized execution

#11

Post by angelachang27 » Fri, Feb 01, 2019, 2:20 AM UTC

DLA wrote: post #9

angelachang27 wrote: post #4

Hey everyone, I'm the intern who did this work, happy to answer any questions if you have them!

Your work is awesome as is your excellent detailed writeup. Appreciate the links and refs to the background papers also. Thank you for helping make CRDB even better. You should be very proud of this work.

:D

Re: Faster hash joiner with vectorized execution

#12

Post by gary__ » Fri, Feb 01, 2019, 2:31 AM UTC

nimish wrote: post #3

Awesome work. Really cool! I have a hobby project to write an analytics DB that uses ISPC for vectorized execution. Currently not much (sums are real easy) but I really wonder if it could reduce the effort to vectorize these sorts of things.

Is it open source / online?

Re: Faster hash joiner with vectorized execution

#13

Post by JanecekPetr » Fri, Feb 01, 2019, 5:25 AM UTC

Genetic specialization (List) is coming. See https://openjdk.java.net/projects/valhalla/. Not this year (value types will be first). But they're working in it really hard.

Re: Faster hash joiner with vectorized execution

#14

Post by curiousDog » Fri, Feb 01, 2019, 9:34 AM UTC

angelachang27 wrote: post #4

Hey everyone, I'm the intern who did this work, happy to answer any questions if you have them!

Excellent work and write-up. Very happy to see the next generation of engineers are top notch like you!

Re: Faster hash joiner with vectorized execution

#15

Post by sAbakumoff » Fri, Feb 01, 2019, 12:44 PM UTC

I looked at the title and thought that the article is about rolling joints with hash by using some advanced robotic technology and software. Damn!

Re: Faster hash joiner with vectorized execution

#16

Post by ddorian43 » Fri, Feb 01, 2019, 12:48 PM UTC

sAbakumoff wrote: post #15

I looked at the title and thought that the article is about rolling joints with hash by using some advanced robotic technology and software. Damn!

Pretty sure that they can use same tech as cigarettes.

Re: Faster hash joiner with vectorized execution

#17

Post by blr246 » Fri, Feb 01, 2019, 12:52 PM UTC

Great write-up. Is the long-term vision to go completely to the vectorised query execution model, or are there cases where a row-oriented plan might be better, such as cases when there are complex computations involving multiple columns of a single row?

Re: Faster hash joiner with vectorized execution

#18

Post by jnordwick » Fri, Feb 01, 2019, 2:17 PM UTC

Column oriented dbs have been doing parallel (column per thread) joins for a while now, no? And I know they have been leaning heavily on vectorization for over a decade.

Re: Faster hash joiner with vectorized execution

#19

Post by jordanlewis » Fri, Feb 01, 2019, 4:20 PM UTC

blr246 wrote: post #17

Great write-up. Is the long-term vision to go completely to the vectorised query execution model, or are there cases where a row-oriented plan might be better, such as cases when there are complex computations involving multiple columns of a single row?

We don't have any plans to remove the row-by-row execution engine. Likely, we'll have some analysis during planning that can inform whether to use the row-oriented or column-oriented engine. I think the use cases for the row-oriented engine are exactly what you mention - things like single-row computations or more OLTP queries like point scans, inserts, update etc - where the overhead of setting up the data structures required to use the column-oriented engine would dominate.

Re: Faster hash joiner with vectorized execution

#20

Post by nimish » Sat, Feb 02, 2019, 12:04 AM UTC

gary__ wrote: post #12

nimish wrote: post #3

Awesome work. Really cool! I have a hobby project to write an analytics DB that uses ISPC for vectorized execution. Currently not much (sums are real easy) but I really wonder if it could reduce the effort to vectorize these sorts of things.

Is it open source / online?

Nope, and by toy I really mean toy. It can handle a single type of query and that's it. Maybe one day I'll polish it up.