Froid: Optimization of Imperative Programs in a Relational Database [pdf]

#1

Post by rodionos » Sun, Dec 23, 2018, 7:22 PM UTC

Froid: Optimization of Imperative Programs in a Relational Database [pdf]
vldb.org

Re: Froid: Optimization of Imperative Programs in a Relational Database [pdf]

#2

Post by karthiksr » Mon, Dec 24, 2018, 7:29 AM UTC

I am a co-author of the Froid paper, and am around if people have any questions/comments/feedback.

Froid is now available as a feature of SQL Server 2019 preview. The feature is called "Scalar UDF Inlining" https://blogs.msdn.microsoft.com/sqlserverstorageengine/2018...

Available to try out for free here: https://www.microsoft.com/en-us/sql-server/sql-server-2019

Re: Froid: Optimization of Imperative Programs in a Relational Database [pdf]

#3

Post by RMarcus » Mon, Dec 24, 2018, 9:11 AM UTC

karthiksr wrote: post #2

I am a co-author of the Froid paper, and am around if people have any questions/comments/feedback. Froid is now available as a feature of SQL Server 2019 preview. The feature is called "Scalar UDF Inlining" https://blogs.msdn.microsoft.com/sqlserverstorageengine/2018... Available to try out for free here: https://www.microsoft.com/en-us/sql-server/sql-server-2019

I've only read part of it, but it seems great so far! I always appreciate the clarity and practicality y'all at the JGL take.

I'm amazed that the implementation was under 1500 LOC! Was that the research prototype or the shipped preview?

Congratulations on the VLDB paper! Hopefully I'll come say "hi" in LA :)

Re: Froid: Optimization of Imperative Programs in a Relational Database [pdf]

#4

Post by gigatexal » Mon, Dec 24, 2018, 11:10 AM UTC

karthiksr wrote: post #2

I am a co-author of the Froid paper, and am around if people have any questions/comments/feedback. Froid is now available as a feature of SQL Server 2019 preview. The feature is called "Scalar UDF Inlining" https://blogs.msdn.microsoft.com/sqlserverstorageengine/2018... Available to try out for free here: https://www.microsoft.com/en-us/sql-server/sql-server-2019

Why does the first example return price as a char? Looking forward to reading the paper fully. I just scanned it.

Re: Froid: Optimization of Imperative Programs in a Relational Database [pdf]

#5

Post by karthiksr » Mon, Dec 24, 2018, 11:37 AM UTC

gigatexal wrote: post #4

karthiksr wrote: post #2

I am a co-author of the Froid paper, and am around if people have any questions/comments/feedback. Froid is now available as a feature of SQL Server 2019 preview. The feature is called "Scalar UDF Inlining" https://blogs.msdn.microsoft.com/sqlserverstorageengine/2018... Available to try out for free here: https://www.microsoft.com/en-us/sql-server/sql-server-2019

Why does the first example return price as a char? Looking forward to reading the paper fully. I just scanned it.

It returns a formatted string including the price and the currency code. Eg: "5000 USD".

Re: Froid: Optimization of Imperative Programs in a Relational Database [pdf]

#6

Post by karthiksr » Mon, Dec 24, 2018, 11:39 AM UTC

RMarcus wrote: post #3

karthiksr wrote: post #2

I am a co-author of the Froid paper, and am around if people have any questions/comments/feedback. Froid is now available as a feature of SQL Server 2019 preview. The feature is called "Scalar UDF Inlining" https://blogs.msdn.microsoft.com/sqlserverstorageengine/2018... Available to try out for free here: https://www.microsoft.com/en-us/sql-server/sql-server-2019

I've only read part of it, but it seems great so far! I always appreciate the clarity and practicality y'all at the JGL take. I'm amazed that the implementation was under 1500 LOC! Was that the research prototype or the shipped preview? Congratulations on the VLDB paper! Hopefully I'll come say "hi" in LA :)

Thank you.

The shipped preview has only a bit more than 1500LOC.

The VLDB paper was presented at Rio in Aug this year already, but I'll try to come over to LA anyways :)

Re: Froid: Optimization of Imperative Programs in a Relational Database [pdf]

#7

Post by maslam » Mon, Dec 24, 2018, 1:10 PM UTC

karthiksr wrote: post #2

I am a co-author of the Froid paper, and am around if people have any questions/comments/feedback. Froid is now available as a feature of SQL Server 2019 preview. The feature is called "Scalar UDF Inlining" https://blogs.msdn.microsoft.com/sqlserverstorageengine/2018... Available to try out for free here: https://www.microsoft.com/en-us/sql-server/sql-server-2019

Thank you for the paper - it is well-written and succinct. Karhik, do you think this approach can be applied to Apache Spark as well (given its well-known slowness with UDFs)?

Re: Froid: Optimization of Imperative Programs in a Relational Database [pdf]

#8

Post by karthiksr » Mon, Dec 24, 2018, 1:58 PM UTC

maslam wrote: post #7

karthiksr wrote: post #2

I am a co-author of the Froid paper, and am around if people have any questions/comments/feedback. Froid is now available as a feature of SQL Server 2019 preview. The feature is called "Scalar UDF Inlining" https://blogs.msdn.microsoft.com/sqlserverstorageengine/2018... Available to try out for free here: https://www.microsoft.com/en-us/sql-server/sql-server-2019

Thank you for the paper - it is well-written and succinct. Karhik, do you think this approach can be applied to Apache Spark as well (given its well-known slowness with UDFs)?

Thank you. Conceptually the ideas behind Froid follow from relational algebra so it can be applied to other relational engines as well. However, the details still need to be figured before making any concrete statement.

If you could share any pointers about UDFs and their performance problems in Spark, I would love to investigate more.

Re: Froid: Optimization of Imperative Programs in a Relational Database [pdf]

#9

Post by maslam » Mon, Dec 24, 2018, 2:34 PM UTC

karthiksr wrote: post #6

RMarcus wrote: post #3

Earlier quoted context omitted.

I've only read part of it, but it seems great so far! I always appreciate the clarity and practicality y'all at the JGL take. I'm amazed that the implementation was under 1500 LOC! Was that the research prototype or the shipped preview? Congratulations on the VLDB paper! Hopefully I'll come say "hi" in LA :)

Thank you. The shipped preview has only a bit more than 1500LOC. The VLDB paper was presented at Rio in Aug this year already, but I'll try to come over to LA anyways :)

Karthik, I'm no Spark expert but almost all advice I read is to avoid UDFs if at all possible. Examples below:

- https://medium.com/teads-engineering/spark-performance-tunin... - https://www.inovex.de/blog/efficient-udafs-with-pyspark/

Re: Froid: Optimization of Imperative Programs in a Relational Database [pdf]

#10

Post by RMarcus » Mon, Dec 24, 2018, 4:50 PM UTC

karthiksr wrote: post #6

RMarcus wrote: post #3

Earlier quoted context omitted.

I've only read part of it, but it seems great so far! I always appreciate the clarity and practicality y'all at the JGL take. I'm amazed that the implementation was under 1500 LOC! Was that the research prototype or the shipped preview? Congratulations on the VLDB paper! Hopefully I'll come say "hi" in LA :)

Thank you. The shipped preview has only a bit more than 1500LOC. The VLDB paper was presented at Rio in Aug this year already, but I'll try to come over to LA anyways :)

Doh! Guess I should've checked. I didn't make it to Rio last year... Figured I was gonna miss a bunch of good stuff.