Viewing profile — mrocklin

Blas and faer are used only for small corners of the API (linalg and fft) which is exactly what numpy does. I encourage you to follow your own advice and look more closely at the i…

Yup.

One can have fun with all manner of things. Take wood-working for example. One can have fun with a handsaw. One can also have fun with a table saw. They're both fun, just different…

If you read on in the post you might be interested in the section titled Drop Python: Use Rust and Typescript https://matthewrocklin.com/ai-zealotry/#big-idea-drop-python...

Original author of the post here. Just to give credit where it's due, that was a quote from this other excellent article written by someone else: https://www.stochasticlifestyle.co…

I mean, Dask doesn't have money. We're definitely not in a place to pay them a bounty. I imagine this is just marketing on their part, or driving up some metric to show customers. …

I'm not convinced that human judgement was ever applied during this situation.

I help maintain Dask, an OSS project in the Python data space. Dask helps users run arbitrary Python code on distributed clusters of computers. Yesterday, I and several users got a…

Hey folks, original author here. It seems like people here are really connecting with the specifics of the code review example. The main point of the article is really "what we lea…

Parallel for loops are the new black I guess

$25 to process 250TiB seems cheaper than what I would expect. I've had (very good) beers that cost that much :)

See http://dask.pydata.org/en/latest/ghost.html

Thanks for the comments. I wrote most of that document so I'll try to explain my reasoning in line. > First and foremost, it would make more sense to compare against the DataFrame …

Flat CSV or JSON files are hard to parse. Fast CSV parsers and gzip decompression both run at around 100MB/s. If you want to get faster than this you'll need to use better (ideally…

Interesting. What are your thoughts on the costs of dependencies? What about toolz makes you hesitate to depend on it?

http://toolz.readthedocs.org/en/latest/

You can manage this by composing merge and merge_with In [1]: data = [{'a': {'b': 1}}, {'a': {'c': 2}}] In [2]: from toolz import merge, merge_with In [3]: merge_with(merge, data) …