Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Julia Computing raises $24M Series A

hpcwire.com

Post reply on HN
Print view View original

161–170 of 246 posts

Re: Julia Computing raises $24M Series A

#161

Post by amkkma »

systemvoltage wrote: post #157
leephillips wrote: post #147

Earlier quoted context omitted.

This is the first time I’ve heard somebody say that Julia’s package management is worse than Python’s! For most people who have spent years grappling with the succession of unofficial attempts to supply Python with something like package management, including virtual environments, etc., and the resulting dependency hell, Julia’s integrated and sophisticated package management (even with its own special REPL mode) is…

I’m a tortured soul, my opinions might be biased and I hope things improve with Julia. We absolutely cannot upgrade Julia version right now, dozens of repos full of complicated scientific code. Management doesn’t care as far as it barely runs. I don’t think it’s fair to blame Julia for it but it just shows how much more it needs to go. That should be looked at as a positive thing. I have one more complain to Julia co…

>I’ve seen a cult-like behavior in Julia community that is borderline toxic. Sorry, it’s the truth and needs to be said. Speed isn’t everything and people are magnetized by the benchmarks on the Julia website, especially non-software engineers.

I think all languages have this dynamic...I've seen it with python and R. To some extent it's fed by what we perceive as criticisms from people defending their favorite incumbent language with arguments that aren't at all informed- such as a focus on speed and how numba achieves parity there.

In the same vein, I and many Julia users are enthusiastic precisely because of thing other than speed, such as the type system, abstractions, differentiability and other things that make programming more joyful, fluid and productive.

Agree though, that we could always improve on acceptance of criticism.

Re: Julia Computing raises $24M Series A

#162

Post by nojito »

freemint wrote: post #133
nojito wrote: post #125

Earlier quoted context omitted.

40% slower in groupbys and 4x slower in joins isn’t convincing.

This seems quiet cherry picked as there are 3 different dataset sizes. However yes, it does not beat all other packages tested in performance.

Not really cherry picked. Data.table is designed for large data sets with many groups + complex joins.

Re: Julia Computing raises $24M Series A

#163

Post by cbkeller »

chrispeel wrote: post #155
awaythrowact wrote: post #149

Earlier quoted context omitted.

I second most of this. A lot of problems are fixable with time and money. Maybe the Series A will help! But some problems might be related to Julia's design choices. One thing I really missed in Julia is Object Oriented Programming as a first class paradigm. (Yes I know you can cobble together an approximation with structs.) OOP gets a lot of hate these days. Mostly deserved. But in some large complex projects it's a…

> One thing I really missed in Julia is Object Oriented Programming as a first class paradigm. Julia peeps would tell you that the multiple dispatch used by Julia is a generalization of OOP. And that they like multiple dispatch

Yes indeed. While Julia does generally eschew the "model.fit(X,Y); model.predict(new_data)" style, it's not because it's functional, it's because it's dispatch-oriented, which is arguably a superset of class-based OO, and even perhaps arguably closer to Alan Kay's claimed original vision for OO than modern class-based OO is.

Re: Julia Computing raises $24M Series A

#164

Post by leephillips »

systemvoltage wrote: post #157
leephillips wrote: post #147

Earlier quoted context omitted.

This is the first time I’ve heard somebody say that Julia’s package management is worse than Python’s! For most people who have spent years grappling with the succession of unofficial attempts to supply Python with something like package management, including virtual environments, etc., and the resulting dependency hell, Julia’s integrated and sophisticated package management (even with its own special REPL mode) is…

I’m a tortured soul, my opinions might be biased and I hope things improve with Julia. We absolutely cannot upgrade Julia version right now, dozens of repos full of complicated scientific code. Management doesn’t care as far as it barely runs. I don’t think it’s fair to blame Julia for it but it just shows how much more it needs to go. That should be looked at as a positive thing. I have one more complain to Julia co…

Well you’re right about the libraries—Python really does have everything (although sometimes those libraries are not great quality, but at least they mostly work). But Julia makes it easy to use Python libraries with Pycall. And there are big things that just don’t exist yet in the Julia world, such as a web framework. I recently tried to create a web project using Genie.jl, which advertises itself as a ready-for-prime-time web framework for Julia, and I gave up after a few days. It’s just not in the same universe as something like Django, plus it’s barely documented.

Re: Julia Computing raises $24M Series A

#165

Post by awaythrowact »

amkkma wrote: post #154
awaythrowact wrote: post #149

Earlier quoted context omitted.

I second most of this. A lot of problems are fixable with time and money. Maybe the Series A will help! But some problems might be related to Julia's design choices. One thing I really missed in Julia is Object Oriented Programming as a first class paradigm. (Yes I know you can cobble together an approximation with structs.) OOP gets a lot of hate these days. Mostly deserved. But in some large complex projects it's a…

How is "model.fit(X,Y)" better than "fit!(model,X,Y)"? Julia is object oriented in a broad sense, it just uses multiple dispatch which is strictly more expressive than single dispatch, so doesn't make sense to have dot notation for calling methods because types don't own methods. For giving up some facility in function discover, you get speed, composability, generic code...and a net gain in usability because you can…

I'll give you my two cents, recognizing that I very well might just be ignorant about Julia and multiple dispatch, and if so please continue to educate me.

Consider if we want to run many different types of models. Logistic regression, gradient boosting, NNs, etc. We want the ability to easily plug in any type of model into our existing code base. That's why model.fit(X,Y) is attractive. I just need to change "model = LogisticRegressionModel" to "model = GradientBoostingModel" and the rest of the code should still Just Work. This is a big part of SciKit's appeal.

But all these different models have very different training loops. So with "fit!(model,X,Y)" I need to make sure I am calling the compatible "fit" function that corresponds to my model type.

You might now say "Ah! Multiple dispatch handles this for you. The 'fit' function can detect the type of its 'model' argument and dispatch execution to the right training loop sub function." And I suppose that's theoretically correct. But in practice I think it's worse.

It should be the responsibility of the developer of "model" to select the "fit" algorithm appropriate for "model." (They don't have to implement it, but they do have to import the right one.) The developer of "fit" should not be responsible for handling every possible "model" type. You could have the developer of "model" override / extend the definition of "fit" but that opens up its own can of worms.

So is it possible to do the same thing with "fit!(model,X,Y)"? Yes of course it is. It's possible to do anything with any turing complete language. The point is, which system provides the best developer ergonomics via the right abstractions? I would argue, in many cases, including this one, it's useful to be able to bundle functions and state, even if that is in theory "less flexible" than pure functions, because sometimes programming is easier with less flexibility.

Re: Julia Computing raises $24M Series A

#166

Post by oscardssmith »

enriquto wrote: post #153
oscardssmith wrote: post #12

Earlier quoted context omitted.

Short answer is that it is (imo) by far the best language for writing generic and fast math. Multiple dispatch allows you to write math using normal notation and not have to jump through hoops to do so.

can you show a simple example of that? I tend to see multiple dispatch as a mental burden, (e.g.: when I see a function call, where will it be dispatched? the answer dependa on the types that I'm juggling, that may not even be visible at that point...)

The key to make multiple dispatch work well is that you shouldn't have to think about what method gets called. For this to work out, you need to make sure that you only add a method to a function if it does the "same thing" (so don't use >> for printing for example). To Dr the benefit of this in action, consider that in Julia 1im+2//3 (the syntax for sqrt(-1)+2 thirds) works and gives you a complex rational number (2//3+1//1 im). To get this behavior in most other languages, you would have to write special code for complex numbers with rational coefficients, but in Julia this just works since complex and rational numbers can be constructed using anything that has arithmetic defined. This goes all the way up the stack in Julia. You can put these numbers in a matrix, and matrix multiplication will just work, you can plot functions using these numbers, you can do gpu computation with them etc. All of this works (and is fast) because multiple dispatch can pick the right method based on all the argument types.

Re: Julia Computing raises $24M Series A

#167

Post by east2west »

amkkma wrote: post #154
awaythrowact wrote: post #149

Earlier quoted context omitted.

I second most of this. A lot of problems are fixable with time and money. Maybe the Series A will help! But some problems might be related to Julia's design choices. One thing I really missed in Julia is Object Oriented Programming as a first class paradigm. (Yes I know you can cobble together an approximation with structs.) OOP gets a lot of hate these days. Mostly deserved. But in some large complex projects it's a…

How is "model.fit(X,Y)" better than "fit!(model,X,Y)"? Julia is object oriented in a broad sense, it just uses multiple dispatch which is strictly more expressive than single dispatch, so doesn't make sense to have dot notation for calling methods because types don't own methods. For giving up some facility in function discover, you get speed, composability, generic code...and a net gain in usability because you can…

I don't miss OOP in Julia but I do feel there need to be more ways to abstract things than multiple dispatch and structs. One thing I do miss is interfaces, which can group functions for a common purpose. I understand it may be not feasible in a dynamic language, but hopefully there will be something above functions as an abstraction mechanism.

Re: Julia Computing raises $24M Series A

#168

Post by oscardssmith »

enriquto wrote: post #153
oscardssmith wrote: post #12

Earlier quoted context omitted.

Short answer is that it is (imo) by far the best language for writing generic and fast math. Multiple dispatch allows you to write math using normal notation and not have to jump through hoops to do so.

can you show a simple example of that? I tend to see multiple dispatch as a mental burden, (e.g.: when I see a function call, where will it be dispatched? the answer dependa on the types that I'm juggling, that may not even be visible at that point...)

One other really good example is BLAS. Since it is a C/Fortran library you have 26 different functions for matrix multiply depending on the type of matrix (and at least that many for matrix vector multiply). In Julia, you just have * which will do the right thing no matter what. In languages without multiple dispatch, any code that wants to do a matrix multiply will either have to be copied and pasted 25 times for each input type, or will have 50 lines of code to branch on the input type. Multiple dispatch makes all of that pain go away. You just use * and you get the most specific method available.

Re: Julia Computing raises $24M Series A

#169

Post by amkkma »

awaythrowact wrote: post #165
amkkma wrote: post #154

Earlier quoted context omitted.

How is "model.fit(X,Y)" better than "fit!(model,X,Y)"? Julia is object oriented in a broad sense, it just uses multiple dispatch which is strictly more expressive than single dispatch, so doesn't make sense to have dot notation for calling methods because types don't own methods. For giving up some facility in function discover, you get speed, composability, generic code...and a net gain in usability because you can…

I'll give you my two cents, recognizing that I very well might just be ignorant about Julia and multiple dispatch, and if so please continue to educate me. Consider if we want to run many different types of models. Logistic regression, gradient boosting, NNs, etc. We want the ability to easily plug in any type of model into our existing code base. That's why model.fit(X,Y) is attractive. I just need to change "model…

Thanks, I see where you are coming from.

>It should be the responsibility of the developer of "model" to select the "fit" algorithm appropriate for "model." (They don't have to implement it, but they do have to import the right one.) The developer of "fit" should not be responsible for handling every possible "model" type. You could have the developer of "model" override / extend the definition of "fit" but that opens up its own can of worms.

It's really the same thing as python, just better...I don't see the distinction you are drawing.

In python you have a base class with default behavior. You can subclass that and inherit or override.

Julia has abstract types with interfaces...instead of relying on implementation details like fields, you provide functions so that more types of models can work even if they don't have that one specific field. Otherwise everything is the same where it counts,- you can compose, inherit and override. Even better, you can work with multiple models and types of data, inheriting where you see fit.

I don't see any benefit to python's restrictions here, either in ease of use or in expressiveness.

For all intents and purposes it's a strict superset.

Even better, you can use macros and traits to group different type trees together.

https://www.stochasticlifestyle.com/type-dispatch-design-pos...

These seem to be in contradiction:

>It should be the responsibility of the developer of "model" to select the "fit" algorithm appropriate for "model.

>You could have the developer of "model" override / extend the definition of "fit" but that opens up its own can of worms.

It's the same in python, either you inherit Fit or you can override it. What's the difference with Julia?

Except in julia all types and functions have a multiple dispatch, parametric type and possible trait lattice of things you can override, customize and compose, so that even if the model author has to override fit, they can do it using small composable building blocks.

Re: Julia Computing raises $24M Series A

#170

Post by wdroz »

mbauman wrote: post #97
tylermw wrote: post #93

Earlier quoted context omitted.

"Massively better performance" is a bit misleading: Julia is only massively better at certain workflows. The fastest data.frame library in ALL interpreted languages is consistently data.table, which is R. For in-memory data analysis, Julia will have to offer more than performance to win over statisticians/researchers. Benchmarks: https://www.ritchievink.com/blog/2021/02/28/i-wrote-one-of-t...

> The fastest data.frame library in ALL interpreted languages is consistently data.table, which is R. DataFrames.jl is very rapidly catching up and starting to surpass it. After hitting a stable v1.0 they've begun focusing on performance and those benchmarks have changed significantly over the past three months. Here's the live view: https://h2oai.github.io/db-benchmark/

I would like to see this benchmark with much more modern hardware, especially for GPU-related tools as the 1080 Ti they used is 4 years old.
Post reply on HN