Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

High Performance Numeric Programming with Swift: Explorations and Reflections

fast.ai

Post reply on HN
Print view View original

11–20 of 48 posts

Re: High Performance Numeric Programming with Swift: Explorations and Reflections

#11

Post by omaranto »

adamnemecek wrote: post #10
byt143 wrote: post #7

Thanks for writing up your thoughts! I find Julia's core design to be excellent for general purpose programming, better than python in fact since it essentially solves the expression problem with it's type system and multiple dispatch. It's external program interop is also more pleasant than Python's : https://docs.julialang.org/en/v1/manual/running-external-pro... Sure, it doesn't have the same general library ecosy…

> With regards to numerical programming, it's obviously already far ahead of swift, and IMO much better placed to beat it in the long run. Julia's secret sauce is LLVM. Considering the guy behind Swift also made LLVM, I'm inclined to think that Swift will come out ahead.

I'd say LLVM is an important part of Julia's success but not the whole story: it is also a very well designed language.

Re: High Performance Numeric Programming with Swift: Explorations and Reflections

#12

Post by byt143 »

adamnemecek wrote: post #10
byt143 wrote: post #7

Thanks for writing up your thoughts! I find Julia's core design to be excellent for general purpose programming, better than python in fact since it essentially solves the expression problem with it's type system and multiple dispatch. It's external program interop is also more pleasant than Python's : https://docs.julialang.org/en/v1/manual/running-external-pro... Sure, it doesn't have the same general library ecosy…

> With regards to numerical programming, it's obviously already far ahead of swift, and IMO much better placed to beat it in the long run. Julia's secret sauce is LLVM. Considering the guy behind Swift also made LLVM, I'm inclined to think that Swift will come out ahead.

>Julia's secret sauce is LLVM

That most certainly is not true. See this for example: https://arxiv.org/pdf/1810.09868.pdf

Re: High Performance Numeric Programming with Swift: Explorations and Reflections

#13

Post by xiaodai »

byt143 wrote: post #9
xiaodai wrote: post #8

Earlier quoted context omitted.

Agree with everything you've said, it's hard to see why one would prefer Swift over Julia for numerical computing. I use Julia for it's regex too; it's just nicer. Hopefully, the data munging packages in Julia can catch up to dplyr and data.table, then we are talking!

https://github.com/queryverse/Query.jl Allows for dplyr syntax to work with any iterable and custom table types using traits...So I think it's already beating R data munging in the flexibility department. Still missing some verbs, but these will be added.

Given Julia has macros, so it will definitely catch up to R and data.table in terms of syntax (if it's not already there). I am more thinking about performance, e.g see https://h2oai.github.io/db-benchmark/. It shows that Julia is lagging behind on group-by (and from my experience many other operations) when compared to R's data.table.

Although I have done some work to make thing fast see: https://github.com/xiaodaigh/FastGroupBy.jl. I have yet to update it to Julia v1. Hopefully, I will get to that soon. However, the improvement I have made only works for grouping up to 2 group-by variables and I need to learn more about generated functions to make the code more generic. So from my (someone who's actually spent time trying to optimise these data operations) perspective, Julia will take a while to catch up. Hats off to the data.table crew!

Re: High Performance Numeric Programming with Swift: Explorations and Reflections

#14

Post by byt143 »

xiaodai wrote: post #13
byt143 wrote: post #9

Earlier quoted context omitted.

https://github.com/queryverse/Query.jl Allows for dplyr syntax to work with any iterable and custom table types using traits...So I think it's already beating R data munging in the flexibility department. Still missing some verbs, but these will be added.

Given Julia has macros, so it will definitely catch up to R and data.table in terms of syntax (if it's not already there). I am more thinking about performance, e.g see https://h2oai.github.io/db-benchmark/ . It shows that Julia is lagging behind on group-by (and from my experience many other operations) when compared to R's data.table. Although I have done some work to make thing fast see: https://github.com/xiaodai…

Ah yes, that is true. However Julia is tackling a harder problem in that the speed lag is presumably due to optimizing for custom element and table types.

Re: High Performance Numeric Programming with Swift: Explorations and Reflections

#15

Post by slowswift »

jph00 wrote: post #2

Hello folks! I wrote this article - so if you have any questions, feel free to shoot them my way. :)

Curious if you ever benchmarked your approach vs, say, going through `Array`/`ContiguousArray` (I think these are slated to converge eventually, FWIW) and using the [`withUnsafeMutableBufferPointer(_:)`](https://developer.apple.com/documentation/swift/array/299477... calls?

You've gotten into a place with a lot of unidiomatic designs--direct pointer access on COW types, etc.--and it's not clear how much is really necessary:

    extension Array where Element:CanDoMath {
    
      // instead of this style:
      func sum_outside() -> Element {
        var result = 0
        let p = self.pointerToStorage // your "get the pointer" method, I think it was just `p`, too?
        for i in 0.. Element {
        return self.withUnsafeBufferPointer() {
          var result = 0
          for v in $0 {
            result += v
          }
          return result
        }
      }
    }
Going the `sum_inside` route for bulk operations makes it easier to remain idiomatic, keep COW around (assuming you want it), benefit from `var/let`, and so on. The only obvious concerns are (a) relative overhead--did you ever benchmark that?--and (b) alignment.

For (b) if you're planning to call things that need particular alignments then as far as I know you will need to write your own storage at this time.

Re: High Performance Numeric Programming with Swift: Explorations and Reflections

#16

Post by adamnemecek »

byt143 wrote: post #12
adamnemecek wrote: post #10

Earlier quoted context omitted.

> With regards to numerical programming, it's obviously already far ahead of swift, and IMO much better placed to beat it in the long run. Julia's secret sauce is LLVM. Considering the guy behind Swift also made LLVM, I'm inclined to think that Swift will come out ahead.

>Julia's secret sauce is LLVM That most certainly is not true. See this for example: https://arxiv.org/pdf/1810.09868.pdf

XLA is built on LLVM I think.

Re: High Performance Numeric Programming with Swift: Explorations and Reflections

#17

Post by byt143 »

adamnemecek wrote: post #16
byt143 wrote: post #12

Earlier quoted context omitted.

>Julia's secret sauce is LLVM That most certainly is not true. See this for example: https://arxiv.org/pdf/1810.09868.pdf

XLA is built on LLVM I think.

I don't think so(https://www.tensorflow.org/xla/developing_new_backend). And even if so, it's not relying on LLVM optimizations (for TPU) or the LLVM API. (which is why some of these had to be punted back up to the Julia optimizer for XLA, which was done in a third party package!).

The point is that Julia's design, type system and multiple dispatch facilitates writing dynamic yet highly optimized code for a variety of backends, even those requiring static semantics (unlike LLVM).

There is no way you can look at that paper (or the Flux ecosystem, or the prob programming languages or the SSA IR autodiff) and chalk up Julia's success to just LLVM.

Re: High Performance Numeric Programming with Swift: Explorations and Reflections

#18

Post by KenoFischer »

adamnemecek wrote: post #16
byt143 wrote: post #12

Earlier quoted context omitted.

>Julia's secret sauce is LLVM That most certainly is not true. See this for example: https://arxiv.org/pdf/1810.09868.pdf

XLA is built on LLVM I think.

The GPU backend is (and so's the CPU backend, but that's currently too slow to get much information from). The TPU backend isn't (which is why we targeted XLA in the first place).

Re: High Performance Numeric Programming with Swift: Explorations and Reflections

#20

Post by currymj »

xiaodai wrote: post #8
byt143 wrote: post #7

Thanks for writing up your thoughts! I find Julia's core design to be excellent for general purpose programming, better than python in fact since it essentially solves the expression problem with it's type system and multiple dispatch. It's external program interop is also more pleasant than Python's : https://docs.julialang.org/en/v1/manual/running-external-pro... Sure, it doesn't have the same general library ecosy…

Agree with everything you've said, it's hard to see why one would prefer Swift over Julia for numerical computing. I use Julia for it's regex too; it's just nicer. Hopefully, the data munging packages in Julia can catch up to dplyr and data.table, then we are talking!

I am a happy Julia user, but I can imagine if you had a use case where you wanted to compile a binary or shared library, Julia could be a pain.

C++ of course works fine for this but I imagine Swift would be less terrifying to use.

Post reply on HN