Julia is presented as a simple language, but is is anything but that in practice.
High-Performance GPU Computing in the Julia Programming Language (2017)
41–50 of 58 posts
Re: High-Performance GPU Computing in the Julia Programming Language (2017)
#42Earlier quoted context omitted.
Are you certain that the story is as simple as "CUDA performs better"? It's common folklore, but I have seen little evidence. The only situations I know of when CUDA performs better is when CUDA-specific features are used (if they are relevant for whatever problem is at hand). Also, CUDA libraries (like cuBLAS or cuFFT) tend to be more efficient than their OpenCL equivalent, which is likely because much more work has…
Did you compare the performance of nvrtc vs offline nvcc compiler?
No; the code we would need to generate would be rather different. Would you expect a significant difference? When we did research on nvrtc before implementing this, we couldn't find any concrete information that nvrtc should generate slower code.
Re: High-Performance GPU Computing in the Julia Programming Language (2017)
#43Earlier quoted context omitted.
In theory Julia is supposed to be fantastic. In practice, things either don't exist, or are poorly implemented: Plotting simple things take 30 seconds. And that's if you don't count the time it takes to `] add Plots`, especially on Windows! And the REPL is broken. And the editor is slow and annoying (Juno or vscode). And documentation ranges from poor (no examples, buggy between platforms, broken links due to version…
Yes plotting in Julia is slow upon first invocation due to the JIT. Annoys me too, but to say the REPL is broken is profoundly puzzling to me. It is the best REPL I have ever used. It beats anything I have used for Python, Ruby, JavaScript, Lua etc. Also your documentation issue is also strange. Yes certain things don’t exist but I would say the Julia docs is quite well made. In particular if you use the REPL documen…
> It is the best REPL I have ever used. It beats anything I have used for Python, Ruby, JavaScript, Lua etc.
This is true, but that's a _very_ low bar to pass. Julia is a Lisp, and deserve to be compared to other Lisps rather than to lesser languages. Every Common Lisp or Scheme I have used has a vastly superior REPL experience than Julia. Even Clojure is better.
Don't get me wrong: I love Julia, and I hope it will eventually replace Python as the main language for scientific computing, data science and machine learning. But the REPL experience, at this point, leaves a lot to be desired. I'm sure it will improve in the future.
Re: High-Performance GPU Computing in the Julia Programming Language (2017)
#44Earlier quoted context omitted.
Yes plotting in Julia is slow upon first invocation due to the JIT. Annoys me too, but to say the REPL is broken is profoundly puzzling to me. It is the best REPL I have ever used. It beats anything I have used for Python, Ruby, JavaScript, Lua etc. Also your documentation issue is also strange. Yes certain things don’t exist but I would say the Julia docs is quite well made. In particular if you use the REPL documen…
> It is the best REPL I have ever used. It beats anything I have used for Python, Ruby, JavaScript, Lua etc. This is true, but that's a _very_ low bar to pass. Julia is a Lisp, and deserve to be compared to other Lisps rather than to lesser languages. Every Common Lisp or Scheme I have used has a vastly superior REPL experience than Julia. Even Clojure is better. Don't get me wrong: I love Julia, and I hope it will e…
I’m curious what specifically you would want improved in the REPL.
Re: High-Performance GPU Computing in the Julia Programming Language (2017)
#45Why this infrastructure is so tightly coupled with CUDA? CUDA is very specific and closed APIs for NVidia hardware only. Programming languages should focus on more general primitives that might work on NVidia or TPUs or something else. PyTorch also has CUDA all over in its APIs and its frustrating to see such tight binding with closed one company API. Also take a look at OpenCL.
> Why this infrastructure is so tightly coupled with CUDA? It's not. It uses LLVM, which can easily target AMD GPUs. (Whether the Julia folks have invested in making this work, I dunno, but it's not Extremely Hard.) Understandably nvidia gives you the wrong impression.
We are indeed interested in targeting AMD GPUs. There is a prototype backend available at https://github.com/JuliaGPU/AMDGPUnative.jl and we are closely following the status of SPIR-V and Intel GPUs in LLVM.
The focus on CUDA comes from the fact that most HPC systems for scientific computing are using Nvidia GPUs. That is finally slowly changing.
Re: High-Performance GPU Computing in the Julia Programming Language (2017)
#46Why this infrastructure is so tightly coupled with CUDA? CUDA is very specific and closed APIs for NVidia hardware only. Programming languages should focus on more general primitives that might work on NVidia or TPUs or something else. PyTorch also has CUDA all over in its APIs and its frustrating to see such tight binding with closed one company API. Also take a look at OpenCL.
Because Khronos up to a little while lived on a bubble that we have to use C, write our own compiler and linking logic to use GPGPUs and collect debugging toolchains from each OEM.
Only when they started getting a beating of PTX bytecode and multi-language deployment on CUDA did they woke up and came up with SPIR (later SPIR-V) and SYCL, which still isn't widely deployed.
Re: High-Performance GPU Computing in the Julia Programming Language (2017)
#47> The performance possibilities of GPUs can be democratized by providing more high-level tools that are easy to use by a large community of applied mathematicians and machine learning programmers. How exactly CUDA is "democratizing" anything, if it's tied to Nvidia? Vulkan backend would make more sense for that purpose.
It allows us to use any programming language with PTX backend.
OpenCL on the other hand is C FTW and now kind of supports C++ if one has luck with the drivers.
From that point of view is democratizing GPGPU programming to anyone that doesn't want to deal with either C or C++.
Re: High-Performance GPU Computing in the Julia Programming Language (2017)
#48Julia is presented as a simple language, but is is anything but that in practice.
Julia is not presented as a simple language, it's presented as a "I want everything" language [1], a Python-Ruby-Perl-C-Fortran-Lisp-Matlab crossover with it's own unique spice. Which is completely opposite from something like Go. You can start programming knowing only one of Julia's inspiration, for example programming Julia like Python, but if you want all the language brings you'll have to dive in a lot of the other sides (which might clash a little with the cleverness of the compiler, as it will accept such varied styles it will not guide you to the one through way of idiomatic Julia code).
Still the Julia team did a great job in making all those diverse features feel part of one connected philosophy instead of an ad hoc pile of functionality, even if it does take a little while to fully internalize it.
Re: High-Performance GPU Computing in the Julia Programming Language (2017)
#49Earlier quoted context omitted.
In theory Julia is supposed to be fantastic. In practice, things either don't exist, or are poorly implemented: Plotting simple things take 30 seconds. And that's if you don't count the time it takes to `] add Plots`, especially on Windows! And the REPL is broken. And the editor is slow and annoying (Juno or vscode). And documentation ranges from poor (no examples, buggy between platforms, broken links due to version…
Yes plotting in Julia is slow upon first invocation due to the JIT. Annoys me too, but to say the REPL is broken is profoundly puzzling to me. It is the best REPL I have ever used. It beats anything I have used for Python, Ruby, JavaScript, Lua etc. Also your documentation issue is also strange. Yes certain things don’t exist but I would say the Julia docs is quite well made. In particular if you use the REPL documen…
The shell is not broken because it's slow.
It's broken because it has poor and puzzling exceptions, because output lags on Windows from gtk bugs (known for years, never fixed, huge GitHub discussion),and because the shell mode is often broken.
Re: High-Performance GPU Computing in the Julia Programming Language (2017)
#50Earlier quoted context omitted.
> It is the best REPL I have ever used. It beats anything I have used for Python, Ruby, JavaScript, Lua etc. This is true, but that's a _very_ low bar to pass. Julia is a Lisp, and deserve to be compared to other Lisps rather than to lesser languages. Every Common Lisp or Scheme I have used has a vastly superior REPL experience than Julia. Even Clojure is better. Don't get me wrong: I love Julia, and I hope it will e…
I’m curious what specifically you would want improved in the REPL.
Does the REPL on Windows have all the features and niceties and quality of life of the REPL on bash or other OSes?
If so (which isn't), then we can start suggesting new features, perhaps better text editing capabilities, or introspection, better access to documentation.