Weld: Accelerating numpy, scikit and pandas as much as 100x with Rust and LLVM
111–117 of 117 posts
Re: Weld: Accelerating numpy, scikit and pandas as much as 100x with Rust and LLVM
#112Earlier quoted context omitted.
No disrespect to you but passion in people can come from deep knowledge of a subject but IME much more commonly comes from lack of experience with alternatives. I've noticed the younger people are the more likely they are to be passionate, which I put down mainly to not knowing any better. Once one has more of the experience that places you higher up where you can see further, suddenly one's own plot of land doesn't…
Does that justify the snide remarks? We should show more tolerance, there might not be a tool in existence that does not have its disadvantages along with its advantages. Healthy discussion around facts is beneficial, dismissing people for being passionate is not.
> dismissing people for being passionate is not
Seems I didn't say that well either. Passion that comes from knowledge and experience is good. If it comes from inexperience then maybe not so good because it isn't "Healthy discussion around facts" but around existing biases.
Edit: genuinely no offence intended.
Re: Weld: Accelerating numpy, scikit and pandas as much as 100x with Rust and LLVM
#113Earlier quoted context omitted.
Does that justify the snide remarks? We should show more tolerance, there might not be a tool in existence that does not have its disadvantages along with its advantages. Healthy discussion around facts is beneficial, dismissing people for being passionate is not.
That did sound snide but it wasn't meant that way. I'm not sure how I should have done it better. > dismissing people for being passionate is not Seems I didn't say that well either. Passion that comes from knowledge and experience is good. If it comes from inexperience then maybe not so good because it isn't "Healthy discussion around facts" but around existing biases. Edit: genuinely no offence intended.
Re: Weld: Accelerating numpy, scikit and pandas as much as 100x with Rust and LLVM
#114Earlier quoted context omitted.
Are you coding as a hobby or a profession? If you are a professional, you will use the most effective tool for the job - to get results. What tool will produce the best results - schedule, budget, quality, maintainability, scalabi, portability, etc.? Other than outliers that will crush your productivity, or multiply it, your feelings are pretty irrelevant. Similarly, when you get into a racecar, your feelings about y…
I sort of disagree with your main assertion. I do big data for a living and what I have seen is that our architecture is dictated to us from above for reasons of "fashion" not really for any reasons of practicality. I'm actually looking for a different job for that reason. We are required to used Java on K8s, Kafka & Cassandra for every single solution big or small because it is fashionable, not because it gets the j…
Good reason to seek a new situation, since you have neither appropriate tools selected for you nor input to select better ones.
Racecars? Yeah, I've only won some SCCA super-regional championships. Yes the driver does have a very large inptut into the setup, BUT it is within the constraint of the combination of the setup change and the improved driver feel must make the combination of car/driver faster. And yes, sometimes a change that makes the car technically a bit slower but gives the driver more confidence will result in faster net lap times -- and those are OK. But whatever the setup is, at the end of the test sessions, whether the car feels great or feels like crap, it's the driver's job to get the most out of it.
And I've had many situations both in the racecar and in international alpine ski racing where something felt weird/odd/unfamiliar/scary, but was fast as heck, so it was my job to adapt, rather than go back into my comfort zone.
Better to keep pushing outside your comfort zone, use tools/setups that get better results, and change your 'feel' to appreciate the better setup.
Re: Weld: Accelerating numpy, scikit and pandas as much as 100x with Rust and LLVM
#115Can anyone pls tell me if there are any other tools out there to increase performance of pandas?
Re: Weld: Accelerating numpy, scikit and pandas as much as 100x with Rust and LLVM
#116Earlier quoted context omitted.
That would be like explaining C++ Concepts to an assembly programmer from the 60s that had never used a "function" as a way of abstracting code. If you really want to know, spend one afternoon learning any programming language with built in support for that (Rust, Ocaml, Haskell, ...). ADTs is one of the first things one learns. In Rust, the features you'd need to learn are enums, patterns, and pattern matching. But…
I know how these things work in Rust, and I'm still failing to see your point. It's not at all as complicated as what you are saying given that you can find many blog articles that explain it succinctly in a couple paragraphs. It's not at all helpful to say that there's something so complicated on these other languages that you can't possibly get the idea across without using them.
enum A { Foo{ x: i32, y: f32 }, Bar(B), Baz([u32; 4]), Moo(i32), Mooz(i32) }
struct B { z: f32, w: (f64, f32) }
let b = B { z: 42.0, ..}; // create a b with z == 42 and default w
let B{ w: (first, _), .. } = b; // get B.w.0 field
let a = A::Foo{ x: 42.0, ..};
if let A::Bar(Bar { z, ..}) = a {
// if a is an A::Bar, get b.z field of A::Bar(b)
}
if let A::Baz([0, 1, 2, 3]) = a {
// if a is an A::Baz containing an array
// with value [0, 1, 2, 3]
}
match a {
A::Moo(1..3 @ v) => {
// if a is an A::Moo where x in A::Moo(x) is in range [0, 3) and put the value in the local v variable
}
A::Moo(x) | A::Mooz(x) => {
// Either A::Moo or A::mooz, gives you the value of x
}
// ERROR: I forgot to match some patterns
}
foo(b);
fn foo(B{ z, ..}: B) -> f64 {
// get the z field of the first function argument
z
}
What in Rust are one liners, and can be used anywhere (let bindings, constructors, match, if-let, while-let, function arguments, ...) is a pain to write in C++ using `std::visit` and `std::variant`. The error messages of `std::visit` + `std::variant` are quite bad as well. And well, then there are also other fundamental problems with variant like `variant` having two variants, `variant` having 3 variants, but you can't reach the second `int`, etc.You can translate all the code above to C++ to use std::visit + std::variant instead. I personally find that C++ is unusable for programming like this, and almost never use std::variant in C++ as a consequence, while I use ADTs in Rust all the time.
Re: Weld: Accelerating numpy, scikit and pandas as much as 100x with Rust and LLVM
#117Earlier quoted context omitted.
Did they have to write the runtime in the same language? They could've used an ML with GC and it would've been better (for a compiler). It doesn't really have any functional programming paradigms. Pattern matching is present in imperative languages like past versions of ATS
> Did they have to write the runtime in the same language? No, they originally wrote the runtime in C++, but ended up re-implementing it in Rust because the C++ runtime had too many bugs. > They could've used an ML with GC and it would've been better (for a compiler). They originally wrote the compiler in Scala with the JVM GC, and they said it was much slower and much harder to embed. > It doesn't really have any fu…