Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Weld: Accelerating numpy, scikit and pandas as much as 100x with Rust and LLVM

notamonadtutorial.com

Post reply on HN
Print view View original

91–100 of 117 posts

Re: Weld: Accelerating numpy, scikit and pandas as much as 100x with Rust and LLVM

#91

Post by fluffything »

shaklee3 wrote: post #72
codr7 wrote: post #70

Earlier quoted context omitted.

The way Python has macros, sure :) You would have to keep everything in variants, or wrap/unwrap manually all over the place to get similar functionality. And C has tagged unions.

Can you elaborate more? What do other languages have over std::variant/visit?

That would be like explaining C++ Concepts to an assembly programmer from the 60s that had never used a "function" as a way of abstracting code.

If you really want to know, spend one afternoon learning any programming language with built in support for that (Rust, Ocaml, Haskell, ...). ADTs is one of the first things one learns.

In Rust, the features you'd need to learn are enums, patterns, and pattern matching.

But be warned that using C++ variant and std::visit will feel like you are being forced to only write C instead of C++ for the rest of your life, knowing that life could be much better. Once you learn this, there is no way to un-learn it.

Re: Weld: Accelerating numpy, scikit and pandas as much as 100x with Rust and LLVM

#92

Post by janered »

Interesting that initial implementation was in Scala but then they switched to Rust because of minimal runtime, language embeddability, functional paradigms, community and high quality packages. The hype bandwagon is so real in here. So basically one could say the same for several other well established languages, e.g. Haskell. Also what saddens me is that everyone forgets about D which has the same benefits and a syntax that does not make scratch your eyes out, especially when it comes to FP. Also D has not actually "skipped the leg day" ;)

Re: Weld: Accelerating numpy, scikit and pandas as much as 100x with Rust and LLVM

#94

Post by atoav »

laichzeit0 wrote: post #81
mlevental wrote: post #80

Earlier quoted context omitted.

how do you know someone uses Julia? don't worry they'll tell you.

The same could be said for every Rust fanboy that feels the need to mention Rust whenever an article about C or anything implemented in C is brought up.

Or every C fanboy brings up C whenever Rust is mentioned.

As somebody who uses both, I don’t understand the whole territorial conflicts in this space. Any C programmer can learn a ton by the paradigm Rust uses. Even if Rust would fade into oblivion tomorrow, the lessons I learned by using it will remain valueable enough for me not to regret having learned it.

C is always going to be needed, given the size of the codebase and the amount of embedded stuff written in it. It comes – however – with a waggon full of dangerous traps and gotchas and in practise very few people are good enough to always avoid them or mitigate the risks created by them. I don’t see any reason why C shouldn’t get better in these areas for the benefit of everyone involved.

Re: Weld: Accelerating numpy, scikit and pandas as much as 100x with Rust and LLVM

#95

Post by integricho »

mlevental wrote: post #80
superdimwit wrote: post #49

Earlier quoted context omitted.

If these things interest you, check out Julia

how do you know someone uses Julia? don't worry they'll tell you.

The vegans of the programming community.

Re: Weld: Accelerating numpy, scikit and pandas as much as 100x with Rust and LLVM

#96

Post by carlmr »

tclancy wrote: post #53
shpongled wrote: post #25

Earlier quoted context omitted.

Not to sound like a member of the Rust evangelism strike force, but after using Rust for a couple years, I don't have any desire to go back to C - sum types alone are worth the switch to me, not to mention iterators, concurrency story, etc.

I can’t decide if Rust Evangelism Strike Force sounds like an awful or awesome cartoon.

Rust United Strike Team would have a better acronym.

Re: Weld: Accelerating numpy, scikit and pandas as much as 100x with Rust and LLVM

#97

Post by tempguy9999 »

L0stLink wrote: post #85
laichzeit0 wrote: post #81

Earlier quoted context omitted.

The same could be said for every Rust fanboy that feels the need to mention Rust whenever an article about C or anything implemented in C is brought up.

lets generalize it further: People will take chances to bring up what they are passionate about. I can relate to this, it is not unreasonable for me.

No disrespect to you but passion in people can come from deep knowledge of a subject but IME much more commonly comes from lack of experience with alternatives.

I've noticed the younger people are the more likely they are to be passionate, which I put down mainly to not knowing any better. Once one has more of the experience that places you higher up where you can see further, suddenly one's own plot of land doesn't seem so special.

Re: Weld: Accelerating numpy, scikit and pandas as much as 100x with Rust and LLVM

#98

Post by marmaduke »

choeger wrote: post #79
vmchale wrote: post #51

Earlier quoted context omitted.

Not that you should replace Rust with Haskell, but Haskell would've been a better choice than Scala. It has its own runtime, but it's not difficult to call Haskell code from C or ATS or whatever.

Not difficult to call from C? How does that work, exactly? Wouldn't you need to properly setup the whole runtime (incl. GC) first?

Yes but I think the parent meant its 'just' an #include and ghc_init() away.

Re: Weld: Accelerating numpy, scikit and pandas as much as 100x with Rust and LLVM

#99

Post by infinite8s »

laughinghan wrote: post #26
yorwba wrote: post #18

Earlier quoted context omitted.

And Cython is an AOT compiler for a superset of Python. RustPython seems to be modestly aiming for a reimplementation of CPython.

Maybe "dialect" would be more accurate than "superset"? I don't think Cython is technically a superset of Python, since I think runtime metaprogramming features like __dict__ and monkey-patching are significantly altered or restricted?

Cython is a reification of the interpretation of a python program. Ie it converts the python code into the equivalent CPython API calls (which are all in C) thereby allowing the developer to interperse real C code. Anything you could do in python you could technically do in Cython, although it would be much more verbose.

Re: Weld: Accelerating numpy, scikit and pandas as much as 100x with Rust and LLVM

#100

Post by ris »

So... this requires cooperation from the underlying libraries (numpy, pandas...) - what is the likelihood of said libraries adopting this upstream vs Weld having to maintain their own shadow implementations for the foreseeable future?

Numpy et al of course already have N python acceleration frameworks hammering at their doorsteps to integrate more closely...

Post reply on HN