61–70 of 146 posts
[note: see more nuanced comment below] The Julia benchmark two links deep at https://github.com/h2oai/db-benchmark doesn't follow even the most basic performance tips listed at https://docs.julialang.org/en/v1/manual/performance-tips/ .
The non-const global variables stand out to me, but I'm not experienced enough tell whether that would make a large difference.
Pretty impressed with the data.table benchmarks. The syntax is a little weird and takes getting used to but once you have the basics it’s a great tool.
I use it a lot but it really breaks the tidyverse, which makes using R actually enjoyable. Why aren’t these other libraries (not in R; I’m talking the others in the benchmark) consistently as fast as data.table? Are the programmers of data.table just that much better?
You may want to look at tidyfst.
> Are the programmers of data.table just that much better?
Pixie dust, R's C API (and yes, they're just exceptionally good).
If this will read a csv that has columns with mixed integers and nulls without converting all of the numbers to float by default, it will replace pandas in my life. 99% of my problems with pandas arise from ints being coerced into floats when a bull shows up.
I've been intrigued about this library, and specifically the possibility about a Python workflow, but a fallback to rust if needed. I mean, I haven't really looked at what the interop is but should work, right? It's not going to happen for now though because the project is still immature and there's zero documentation in Python from what I can see. But it's something in keeping a close eye on, I often work with R and…
This is super cool. Anyone know if Pandas is also planning to adopt Arrow ?
If this will read a csv that has columns with mixed integers and nulls without converting all of the numbers to float by default, it will replace pandas in my life. 99% of my problems with pandas arise from ints being coerced into floats when a bull shows up.
"Polars is based on the Rust native implementation Apache Arrow. Arrow can be seen as middleware software for DBMS, query engines and DataFrame libraries. Arrow provides very cache-coherent data structures and proper missing data handling." This is super cool. Anyone know if Pandas is also planning to adopt Arrow ?
"Polars is based on the Rust native implementation Apache Arrow. Arrow can be seen as middleware software for DBMS, query engines and DataFrame libraries. Arrow provides very cache-coherent data structures and proper missing data handling." This is super cool. Anyone know if Pandas is also planning to adopt Arrow ?
As a side note, Wes McKinney, the creator of Pandas, is heavily involved in Arrow.
This is a very strange way to write “Polaris is the second fastest” but I guess that doesn’t grab headlines