91–100 of 246 posts
What we've noticed is the vast majority of the time it's the data scientist's code that's slow not the actual ML model bit. So allowing them to write very performant code with a dumpy-like syntax and not have to deal with painfully slow pandas, lack of true parallelism, etc. would be a true game changer for ML in industry.
Earlier quoted context omitted.
You're not just one among them given how much control you have over the language itself. Those other companies aren't founded by the co-creators of and main contributors to the language.
Sure, but there's two separate concerns here. One is that money will turn us evil and we'll exercise undue influence. My counterpoint was that JC has been around for six years now and in that time we've actually strengthened Julia significantly as an independent project. My other point though was about concentration risk. I think far more common than people turning evil is that companies go all out raising money, bec…
Earlier quoted context omitted.
“Only” to write a very high amount of high-quality statistical and plot packages…
Right. R's killer feature is its ecosystem. I'm wondering if most statisticians or researchers deal with data big enough that massively better performance would be enough motivation to switch.
Benchmarks: https://www.ritchievink.com/blog/2021/02/28/i-wrote-one-of-t...
Frankly I think the key thing that'll really get a lot of Julia adoption is a full-featured ML framework on par with TF, Pytorch, etc. What we've noticed is the vast majority of the time it's the data scientist's code that's slow not the actual ML model bit. So allowing them to write very performant code with a dumpy-like syntax and not have to deal with painfully slow pandas, lack of true parallelism, etc. would be…
Frankly I think the key thing that'll really get a lot of Julia adoption is a full-featured ML framework on par with TF, Pytorch, etc. What we've noticed is the vast majority of the time it's the data scientist's code that's slow not the actual ML model bit. So allowing them to write very performant code with a dumpy-like syntax and not have to deal with painfully slow pandas, lack of true parallelism, etc. would be…
There are a number of components here which enable (what I would call) the expression of more advanced models using Julia's nice compositional properties.
Flux.jl is of course what most people would think of here (one of Julia's deep learning frameworks). But the reality behind Flux.jl is that it is just Julia code -- nothing too fancy.
There's ongoing work for AD in several directions -- including a Julia interface to Enzyme: https://github.com/wsmoses/Enzyme.jl
Also, a new AD system which Keno (who you'll see comment below or above) has been working on -- see Diffractor.jl on the JuliaCon schedule (for example).
Long story short -- there's quite a lot of work going on.
It may not seem like there is a "unified" package -- but that's because packages compose so well together in Julia, there's really no need for that.
it's not obvious to me what's their revenue model?
Earlier quoted context omitted.
Right. R's killer feature is its ecosystem. I'm wondering if most statisticians or researchers deal with data big enough that massively better performance would be enough motivation to switch.
"Massively better performance" is a bit misleading: Julia is only massively better at certain workflows. The fastest data.frame library in ALL interpreted languages is consistently data.table, which is R. For in-memory data analysis, Julia will have to offer more than performance to win over statisticians/researchers. Benchmarks: https://www.ritchievink.com/blog/2021/02/28/i-wrote-one-of-t...
DataFrames.jl is very rapidly catching up and starting to surpass it. After hitting a stable v1.0 they've begun focusing on performance and those benchmarks have changed significantly over the past three months. Here's the live view: https://h2oai.github.io/db-benchmark/
Congrats to the Julia team. I am a python developer who has dabbled with Julia but it never stuck for me. I think Julia was built by academics for other academics running innovative high performance computing tasks. It excels at the intersection of 1) big data, so speed is important, and 2) innovative code, so you can't just use someone else's C package. Indeed, Julia's biggest successful applications outside academi…
> The next PyTorch needs to be written in Julia. Will that happen? Maybe! I hope so! The Two Language Problem makes this more likely than one might think. Those high level python packages that plaster over python's many mediocrities have to be written and maintained by someone, and while extremistan bears the brunt of the pain and has done a remarkable job shielding mediocristan, it's extremistan that gets to decide…
Julia seems like such a superior language compared to R. What would be required for it to supplant R for statistical work (or some subset of it)?
“Only” to write a very high amount of high-quality statistical and plot packages…