Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster than Python and R

juliacomputing.com

Post reply on HN
Print view View original

201–210 of 236 posts

Re: CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster than Python and R

#201

Post by jakobnissen »

ku-man wrote: post #66

I'm glad Julia does exist. It's great to have a free and open-source programming language focused on scientific computing. What I dislike from the Julia community is the unhealthy fixation they have with Python and R. Virtually, no comment or blog-post is written without mentioning how slow, inefficient, inappropriate, inelegant is Python or R. Somehow, the Julia community convinced itself that best way to attract mo…

Unfortunately, you are right - but mostly when discussing programming languages on the Internet. In forums like this, we (Julians) often want to make a case for Julia. Since most soon-to-be Julians come from Python, R or Matlab, it's natural to argue for Julia by arguing why Python and R is deficient.

It's similar to how it's hard to argue for Rust without mentioning that C is unsafe. If C users kept insisting that the unsafe behaviour of C was not a real issue, Rustaceans would seem a lot more negative towards C.

Re: CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster than Python and R

#202

Post by drej »

StefanKarpinski wrote: post #163
mlthoughts2018 wrote: post #120

Earlier quoted context omitted.

> “ Interestingly, CSV.jl can do all of those.“ That sounds like a poorly designed library to me. Instead of doing one thing well, it forces users to consume something meant as a jack of all trades, which may not meet their use case. I wish such a thing wasn’t forced in a standard package - rather split into different ones. Then I can easily compose them according to my needs, instead of having the stdlib force feed…

That you believe this violates the principle of "do one thing and do it well" indicates a certain "abstraction blindness". Do you really believe that "parse CSV to a data frame" and "parse CSV to a parquet file" and "parse CSV and apply a query to it in-place" are completely separate, unrelated things? It seems clear to me in any case that with the right abstraction, those are the same thing composed with different c…

> In the Python ecosystem there are a dozen different versions of each thing that make different trade-offs and specialize in different ways. When you're picking one, you have to spend a week evaluating all the possible CSV parsers (for example).

It saddens me to read this weird statement. It saddens me as I was a rather early adopter in Julia (v0.3.2, if I remember correctly) and really believed its mission. I don't quite understand why various ecosystems cannot peacefully coexist without these untrue and completely avoidable statements.

Re: CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster than Python and R

#203

Post by Mikhail_K »

johnisgood wrote: post #164
bjoli wrote: post #100

Earlier quoted context omitted.

Pythons CSV reader is actually decently fast. It is just that speed wasnt a priority. I dislike the whole "faster than C" comparisons. Almost nothing is. If you want speed you chose C. If you want something that is plenty fast with a much better speed-to-effort ratio, Julia is a strong contender.

Exactly. Nothing is faster than C that is also high-level. If it is, then the C version is not equivalent to the code with what you are comparing. Or are there any examples where this is not the case? If you want performance in your language, you typically write those parts in C, and if it is in C and still not fast enough, you typically go for inline assembly.

> Nothing is faster than C that is also high-level.

That is myth. C is not necessarily faster on modern hardware, because it does not represent its structure correctly

[C Is Not a Low-level Language. Your computer is not a fast PDP-11.](https://queue.acm.org/detail.cfm?id=3212479)

Re: CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster than Python and R

#204

Post by qaq »

deathtrader666 wrote: post #189
qaq wrote: post #2

Julia is a hidden gem Once ecosystem for web dev matures Julia will be the killer lang for building web apps

Genie.jl exists for web apps.

In no way did I mean to imply there are no packages, I just noted they are not yet as mature as compared to web frameworks/packages in other langs.

Re: CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster than Python and R

#205

Post by DNF2 »

CJefferson wrote: post #83
ViralBShah wrote: post #48

Earlier quoted context omitted.

Wouldn't this statement be true for any new technology? I don't doubt what you say about being burnt when adopting in the early days. However, it is simultaneously true that people have successfully used Julia in several commercial applications and significant research codebases for many years now. I routinely point people to the Julia Computing case studies for this reason: https://juliacomputing.com/case-studies/ W…

I've generally found the Julia community less welcoming than some (Rust, Python), although much better than the "classics" (C,C++,Java and friends). For example, picking the first question I can see on discourse (which isn't a clear technical question, it was number 3) is "What is the status of debugger?". The first answer is "I've used Julia for 3 years and I don't need a debugger" whereas searching for similar ques…

The second answer in that thread, however, are links directly docs, repository and a blogpost describing the debugger. I would say the question was very well answered.

Your post gives the impression that the question was dismissed, maybe even giving the impression that there is no work on a debugger, which is quite misleading. There are several debuggers, and lots of work.

There is some controversy in the community about the _need_ for a debugger, with people with strong opinions on both sides.

As for the general tone, I don't know what to say, except that I find it quite helpful, with lots of people putting in a lot of effort to help whomsoever comes along with questions.

Re: CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster than Python and R

#206

Post by StefanKarpinski »

ggrothendieck wrote: post #79

The vroom R package is likely the fastest R package in read speed, not the ones used in this article. vroom was 13.1x faster than R's fread/data.table based on the read performance benchmark in https://cran.r-project.org/web/packages/vroom/vignettes/benc... so may be similar or faster in read speed vs julia.

Vroom is lazy though: it doesn’t parse until you access the data. That’s a very different approach, which is why it’s no direct comparison. You’ll note that data.table, which is compared in the article, is faster than vroom’s default configuration when it comes to whole table scans, which would be the appropriate comparison. If the data is mostly numeric, then data.table is faster for all configurations. That said, if you don’t plan to access most of the data in a data set, then vrooms lazy approach can be as big win.

Re: CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster than Python and R

#207

Post by elcomet »

johnmyleswhite wrote: post #137
elcomet wrote: post #69

Earlier quoted context omitted.

It's a bit more complex. Both python and julia are compiled to bytecode. Then python bytecode gets interpreted, but I think that julia bytecode is actually compiled with a JIT compiler. So julia is not really interpreted.

What do you mean when you use the word "bytecode" in reference to Julia?

I'm not very familiar with julia so it might be wrong. It might be just an AST. The point is that julia is JIT compiled.

Re: CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster than Python and R

#208

Post by ddragon »

elcomet wrote: post #207
johnmyleswhite wrote: post #137

Earlier quoted context omitted.

What do you mean when you use the word "bytecode" in reference to Julia?

I'm not very familiar with julia so it might be wrong. It might be just an AST. The point is that julia is JIT compiled.

Julia has multi-stage compilation: first it's lowered to the AST (all macros are resolved), then it is lowered to an IR, then types are solved, then it's lowered to an SSA form IR, then to LLVM IR and finally machine code [1] (and it's JIT will try to pre-compile as much code as it can, in some situations it might even compile the entire program in one go, which is the cause of the delay when starting the program). Everything that runs is always machine code as there is no interpreter or VM (though you might say that Julia's bytecode is the LLVM IR).

[1] https://blog.rogerluo.me/images/julia-compile-diagram.png

Re: CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster than Python and R

#209

Post by StefanKarpinski »

nnm wrote: post #113

> The tools used for benchmarking were BenchmarkTools.jl for Julia, microbenchmark for R, and timeit for Python. Is this a meaningful comparison? These benchmarks repeatedly call the function (with same input file) many times and then take the average of the time spent. However, in real world, we only read a specific file ONCE -- if we call the csv reader several times, it is almost always for different files.

There are two potential issues that this might disregard:

1. cold file cache

2. JIT compile time

The cold/hot file cache issue affects all languages equally, so it doesn’t invalidate the comparison. It is possible that CSV file is not in cache and a system's disk I/O is slow enough that it becomes a bottleneck, which would make all parsers equally slow. However, this is not very realistic because CSV files—especially large enough ones where you care about speed—are almost always compressed—So you're not going to be bottlenecked on disk read. Assuming that compressed disk read + decompression is fast enough to keep the data flow high, you're back to CSV parsing being the bottleneck.

The JIT compile cache only affects Julia. The reason is it not included in the benchmark results is because it is a small, fixed overhead: it is only paid on the first CSV file read (no matter the size), and if you read a larger data set, the compile time does not increase. Since the point of benchmarks is typically to project from a smaller case how long it would take to do even larger tasks, you don't want to include small, fixed overheads.

For some concrete numbers, I just timed reading a tiny CSV file on my MacBook Pro 2018 and the first read, including compile time took 4 seconds. The second read took 0.000347 seconds. So that's the fixed overhead we're talking about here for reading a CSV file: about 4 seconds on the very first CSV file you read. People can be the judge of whether that's a showstopper for them or not.

Re: CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster than Python and R

#210

Post by StefanKarpinski »

bachmeier wrote: post #130
ku-man wrote: post #66

I'm glad Julia does exist. It's great to have a free and open-source programming language focused on scientific computing. What I dislike from the Julia community is the unhealthy fixation they have with Python and R. Virtually, no comment or blog-post is written without mentioning how slow, inefficient, inappropriate, inelegant is Python or R. Somehow, the Julia community convinced itself that best way to attract mo…

Completely agree. I was turned off by two things when Julia started getting hyped. The first was the benchmarks they reported, which were, shall I say, carefully selected. This is a perfect example. As someone else pointed out, R has much faster ways to read csv files than what they used. The thing is, most of the time it doesn't make any difference at all. The other was the nonstop attacks on other languages, many o…

For the historical record, when we picked those benchmarks [1] we were terrible at them. Like 10,000x slower than C. We picked them because they were representative of the kinds of things that we felt a language should be good at: iteration, recursion, shuffling data in arrays, floating-point operations, complex number arithmetic, string operations, small matrix operations and large matrix operations. Then we worked on the language until it was no worse than 2x slower than C. So claiming that we cherry-picked those benchmarks because they made Julia look good simply isn't historically accurate: we picked them because they made Julia look bad, and then we improved Julia until it didn't look bad anymore.

[1] https://julialang.org/benchmarks/

The only faster way to read CSV files in R that I'm aware of is vroom, which is faster because it's lazy: it doesn't parse until you actually access the data. According to vroom's own benchmarks, if you access all the data, it is slower than data.table, which is compared here. So yes, if you want to load a CSV file and not access most of the data, then vroom is faster; if you want to access all of the data (which seems pretty typical), it is not faster.

Regarding "nonstop attacks on other languages", honestly, we don't care and would be happy to stop talking about other languages. However, there is a constant barrage of people making comparisons to other languages and demanding that we justify Julia's existence. Like it is literally offensive to people that Julia exists — "why didn't you just use Python/R/Matlab?!??!" I cannot express how continual this is. So many people have asked to see these exact benchmarks comparing CSV.jl to read.table and Pandas' csv reader. So here are the benchmarks. If you see a mildly worded benchmark comparison showing that one CSV library is faster than some others as "an attack", then you are too good for the internet.

Post reply on HN