Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster than Python and R

juliacomputing.com

Post reply on HN
Print view View original

161–170 of 236 posts

Re: CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster than Python and R

#161

Post by mlthoughts2018 »

imtringued wrote: post #148
mlthoughts2018 wrote: post #120

Earlier quoted context omitted.

> “ Interestingly, CSV.jl can do all of those.“ That sounds like a poorly designed library to me. Instead of doing one thing well, it forces users to consume something meant as a jack of all trades, which may not meet their use case. I wish such a thing wasn’t forced in a standard package - rather split into different ones. Then I can easily compose them according to my needs, instead of having the stdlib force feed…

I think parents' point is that a fast CSV parser can be written once and then used to implement many different features. Meanwhile in python every library has to reimplement its own CSV parser because switching from C to python and then back to C would ruin performance. To avoid this context switch the csv parser has to be written again and again.

This is not true at all. Most libraries do not reimplement csv parsing - I’m saying only a select few libraries do this because they have very different design goals. For example optimizing for maximum rows per second parsed is a totally different design goal from optimizing to read into Spark DataFrame types to connect to Spark. Prioritizing one of these might come at the cost of the other, which is totally fine and having more than one library to choose from is a good thing.

> “ switching from C to python and then back to C would ruin performance.”

This is explicitly wrong in regards to Python. Optimizing Python often involves writing an implementation directly in C (or better, Cython) and autogenerating bindings that marshal between CPython native C types and the types of your extension module. This is extremely fast - as fast as calling native C code, there is no performance penalty for this.

Re: CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster than Python and R

#162

Post by orbifold »

ViralBShah wrote: post #160
orbifold wrote: post #155

Earlier quoted context omitted.

So I want to plot the data I just loaded. Since I wrote the comment I had closed the Julia window (not to make a point, btw.) julia> import CSV (takes ~1-2 seconds) julia> data = CSV.File("/User/orbifold/Downloads/FL_insurance_sample.csv) (takes seconds again...) I want to explore the data. I vaguely remember there is Gadfly to do that. For some reason it depends on FFTW and a whole bunch of other packages, but the d…

Yes, Gadfly does have a large compile time. Reducing the time to first plot (essentially compilation time) has been a major focus for the 1.5 release. People often use Plots.jl or PyPlot which have significantly lesser compile times.

time to precompile Plots: 138s

julia> @time Plots.scatter(data.eq_site_limit, data.hu_site_limit) 5.461595 seconds

subsequent calls report significant speed ups compared to matplotlib, but render about as quickly

julia> @time Plots.histogram(data.eq_site_limit) 2.143983 seconds

Re: CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster than Python and R

#163

Post by StefanKarpinski »

mlthoughts2018 wrote: post #120
StefanKarpinski wrote: post #114

Earlier quoted context omitted.

Interestingly, CSV.jl can do all of those. This seems like a classic case of lack of composability in Python: you need a different CSV parser for each and every application even though the core logic of parsing the CSV format is the same. This is because anything fast has to be implemented in very concrete, specialized C code. In Julia, you can write something generic and let the compiler specialize it for you, so yo…

> “ Interestingly, CSV.jl can do all of those.“ That sounds like a poorly designed library to me. Instead of doing one thing well, it forces users to consume something meant as a jack of all trades, which may not meet their use case. I wish such a thing wasn’t forced in a standard package - rather split into different ones. Then I can easily compose them according to my needs, instead of having the stdlib force feed…

That you believe this violates the principle of "do one thing and do it well" indicates a certain "abstraction blindness". Do you really believe that "parse CSV to a data frame" and "parse CSV to a parquet file" and "parse CSV and apply a query to it in-place" are completely separate, unrelated things? It seems clear to me in any case that with the right abstraction, those are the same thing composed with different choices of what to do with the parsed data.

At a higher level, this seems like a classic Julia versus Python dynamic. In the Python ecosystem there are a dozen different versions of each thing that make different trade-offs and specialize in different ways. When you're picking one, you have to spend a week evaluating all the possible CSV parsers (for example).

In the Julia ecosystem, the ideal it to have one really high-quality, fast, flexible implementation that everyone collaborates on to make it great for all kinds of use cases. And the language design and the compiler make this possible, since you can write pretty abstract code and compose it with various types and get a really fast implementation. The annoying and tricky logic of parsing the CSV format only needs to be done once, and reused for all different situations.

This isn't done for "macho bragging rights" but because it is a waste of time and effort to implement a dozen half-baked versions of the same thing. When there are a dozen separate implementations of something like this, they also end up disagreeing on their behavior, which causes no end of headaches.

Re: CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster than Python and R

#164

Post by johnisgood »

bjoli wrote: post #100
ViralBShah wrote: post #54

Earlier quoted context omitted.

You can write slow programs in all languages. There are many cases where Julia's language features enable higher performance than in comparable C libraries. Have a look at Steve Johnson's keynote from JuliaCon 2019: https://www.youtube.com/watch?v=mSgXWpvQEHE

Pythons CSV reader is actually decently fast. It is just that speed wasnt a priority. I dislike the whole "faster than C" comparisons. Almost nothing is. If you want speed you chose C. If you want something that is plenty fast with a much better speed-to-effort ratio, Julia is a strong contender.

Exactly. Nothing is faster than C that is also high-level. If it is, then the C version is not equivalent to the code with what you are comparing. Or are there any examples where this is not the case? If you want performance in your language, you typically write those parts in C, and if it is in C and still not fast enough, you typically go for inline assembly.

Re: CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster than Python and R

#165

Post by StefanKarpinski »

dmw_ng wrote: post #108
StefanKarpinski wrote: post #99

Earlier quoted context omitted.

You may be missing the point of the comparison, which is this: to compare the performance of the de facto standard CSV parsers of each system when used the way that a typical user would use them, i.e. to parse a CSV file into a data frame. Regarding your specific points: 1. Yes, it's possible to rig up some way of parsing in parallel in Python, like splitting the CSV file into multiple files and then parsing them in…

Regarding 2), code like "for row in pandas.DataFrame(...).iterrows():" where the DataFrame was populated by some array-based reader is creating an indirection. Most good pandas code of course doesn't look like this, but it's regularly unavoidable, hence "There are use cases where either representation is preferred". Without specifying our use case, it is meaningless to talk about "faster" or "slower". For 3) there in…

> there inherently is no optimal solution to "a problem with large performance-correctness tradeoffs"

You seem to be under the misapprehension that mainstream CSV libraries parse float values with reduced precision in order to go faster. They do not. A decimal floating-point string represents a precise mathematical value and the only acceptable value for a CSV parser to produce is that value correctly rounded to a Float64 (or some other type if requested). Any CSV parser that does anything else is simply incorrect.

> We saw elsewhere in the thread that Julia's float parsing code at least presently handles certain syntaxes incorrectly.

You may be referring to a bug mentioned elsewhere that fields like `-123.` with a leading minus sign and a trailing decimal point are misinterpreted as strings. That is a bug, plain and simple. It has nothing to do with float parsing speed/accuracy tradeoffs.

Re: CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster than Python and R

#166

Post by tikej »

mlthoughts2018 wrote: post #158
cbkeller wrote: post #124

Earlier quoted context omitted.

Well, the promise of overhead-free multiple dispatch is that you don't have to choose between a general solution with poor performance and a specific solution with good performance. You can have all the specific solutions you need in one place and automatically dispatch to the right one without any extra effort. I know it sounds too good to be true, but my limited experience so far is that it really is true , with su…

Multiple dispatch based on static typing only solves a very narrow set of optimization problems - most complex trade offs that would lead to the need for eg multiple different third party csv readers are not addressable at all by multiple dispatch. I’d also point out that with fused typing in Cython, it’s also trivially easy to get overhead-free multiple dispatch in Python too, and unlike Julia, this had the benefit…

> Multiple dispatch based on static typing only solves a very narrow set of optimization problems - most complex trade offs that would lead to the need for eg multiple different third party csv readers are not addressable at all by multiple dispatch.

How about multiple dispatch with dynamic (runtime) typing?

Re: CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster than Python and R

#167

Post by mlthoughts2018 »

coliveira wrote: post #134
drej wrote: post #117

Earlier quoted context omitted.

Lack of composability? What? Of course you can use one parser for everything, what’s stopping you from doing that. Separating IO from business logic and composing the two is a pretty standard practice. Not sure I follow the argument here.

The argument is simple: Python is by design made so that if you want things to go fast then you need to implement it in C. This creates a constant need to use specialized libraries (such as pandas), where in other languages they can simply do a better job in optimizing the standard library.

This is untrue, you can compartmentalize your C implementations using tools like Cython.

The fact that you write these as isolated special implementations is a good thing and not a deficiency of generics.

This way it can be gradual. You only target certain functions or modules for optimized C implementation, you don’t waste static typing overhead or compliance to a generic interface on the 99% of the code that will have no material gain from any optimization.

It’s similar with gradual typing in Python as well. Annotate what you need, omit annotations for what you don’t. Implement an extension module or multiple dispatch for what you need, use plain Python for everything else.

Re: CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster than Python and R

#168

Post by mlthoughts2018 »

StefanKarpinski wrote: post #163
mlthoughts2018 wrote: post #120

Earlier quoted context omitted.

> “ Interestingly, CSV.jl can do all of those.“ That sounds like a poorly designed library to me. Instead of doing one thing well, it forces users to consume something meant as a jack of all trades, which may not meet their use case. I wish such a thing wasn’t forced in a standard package - rather split into different ones. Then I can easily compose them according to my needs, instead of having the stdlib force feed…

That you believe this violates the principle of "do one thing and do it well" indicates a certain "abstraction blindness". Do you really believe that "parse CSV to a data frame" and "parse CSV to a parquet file" and "parse CSV and apply a query to it in-place" are completely separate, unrelated things? It seems clear to me in any case that with the right abstraction, those are the same thing composed with different c…

> “ Do you really believe that "parse CSV to a data frame" and "parse CSV to a parquet file" and "parse CSV and apply a query to it in-place" are completely separate, unrelated things?“

Yes, absolutely. Very high level design goals are completely different between those use cases and you would be forced to make mutually exclusive trade offs to solve any one of those problems - which will directly lead to concessions in terms of the other use cases. No one generic csv parser can be optimal to all three.

Re: CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster than Python and R

#169

Post by StefanKarpinski »

mlthoughts2018 wrote: post #168
StefanKarpinski wrote: post #163

Earlier quoted context omitted.

That you believe this violates the principle of "do one thing and do it well" indicates a certain "abstraction blindness". Do you really believe that "parse CSV to a data frame" and "parse CSV to a parquet file" and "parse CSV and apply a query to it in-place" are completely separate, unrelated things? It seems clear to me in any case that with the right abstraction, those are the same thing composed with different c…

> “ Do you really believe that "parse CSV to a data frame" and "parse CSV to a parquet file" and "parse CSV and apply a query to it in-place" are completely separate, unrelated things?“ Yes, absolutely. Very high level design goals are completely different between those use cases and you would be forced to make mutually exclusive trade offs to solve any one of those problems - which will directly lead to concessions…

CSV.jl seems to be proof that you're wrong about that. More concretely, the only difference between parsing to a data frame, parsing to a parquet file, and executing a query over data is a) where you put the data in memory and b) optionally calling some code to operate on the data as you go. It's hard to see how those applications require radically different designs.

Re: CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster than Python and R

#170

Post by fermienrico »

ViralBShah wrote: post #160
orbifold wrote: post #155

Earlier quoted context omitted.

So I want to plot the data I just loaded. Since I wrote the comment I had closed the Julia window (not to make a point, btw.) julia> import CSV (takes ~1-2 seconds) julia> data = CSV.File("/User/orbifold/Downloads/FL_insurance_sample.csv) (takes seconds again...) I want to explore the data. I vaguely remember there is Gadfly to do that. For some reason it depends on FFTW and a whole bunch of other packages, but the d…

Yes, Gadfly does have a large compile time. Reducing the time to first plot (essentially compilation time) has been a major focus for the 1.5 release. People often use Plots.jl or PyPlot which have significantly lesser compile times.

@ViralBShah and @StefanKarpinski

I've had similar problems and so have many of my colleagues in silicon valley. No matter how you spin it and whatever benchmarks you show - Julia is a very slow fast language. :)

The UX of Julia needs major work - everything is just slow. If you've used Python for 10 years and it is like running through molasses. Benchmarks are meaningless for the most part. This is what bothers me about Julia's marketing - it claims to be fast, but in reality, precompilation time alone is a no go unless you're doing high-compute large batch processing. Julia cannot be come a general purpose language unless you fix these issues.

> These "conflations" sound suspiciously like excuses.

I think you're the one conflating total end-to-end time spent between given a task and going home early to see my kids if I used Python.

Post reply on HN