Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pyston v2: Faster Python

blog.pyston.org

Post reply on HN
Print view View original

191–200 of 211 posts

Re: Pyston v2: Faster Python

#191

Post by disgruntledphd2 »

newen wrote: post #117
sk2020 wrote: post #101

Earlier quoted context omitted.

I like R, but there is nothing elegant or consistent about its standard library. I don’t think you’re making a good-faith argument.

Don’t just randomly accuse people of making bad faith arguments. I’ve use R a lot and while it has its issues, it has a much better interface than numpy.

Well they're different, right?

R provides an interface to dataframes and matrices, whereas numpy is just for matrices (and their generalisations, arrrays). I think the appropriate comparison is between base R and Numpy + pandas. (FWIW, I agree with your major point, but then I learned R first so that may be biasing me).

Re: Pyston v2: Faster Python

#192

Post by pjmlp »

throwaway894345 wrote: post #185
pjmlp wrote: post #176

Earlier quoted context omitted.

All dynamic languages are generic by default.

If that's how you define "generic", then Go is also generic by virtue of `interface{}`.

No, because interface{} in an empty type that needs to be type cast to the actual type before use.

Dynamic languages do that implicitly.

Implement max in Go with interface{} without casts and reflection:

   def max(a, b):
     if a >= b:
        a
     else:
        b

Re: Pyston v2: Faster Python

#193

Post by rwallace »

semi-extrinsic wrote: post #172
xvilka wrote: post #160

Earlier quoted context omitted.

Julia covers your use cases and overwhelmingly fast.

My gripe with Julia etc. as replacements, is that Python is duct tape. I don't need fast duct tape, I need duct tape that is understood and used by essentially everyone I work with, and that has native, fast handling of large amounts of data (NumPy, Pandas). Good user experience as duct tape. From my perspective Julia is sacrifising some amount of "duct tape UX" to gain speed, and that's the wrong direction. Whenever…

> From my perspective Julia is sacrifising some amount of "duct tape UX" to gain speed, and that's the wrong direction.

What particular language features of Julia make that trade-off? (Not a rhetorical question, I'm not disagreeing with you, just curious; I'm familiar with Python, not really familiar with Julia.)

Re: Pyston v2: Faster Python

#195

Post by sk2020 »

newen wrote: post #117
sk2020 wrote: post #101

Earlier quoted context omitted.

I like R, but there is nothing elegant or consistent about its standard library. I don’t think you’re making a good-faith argument.

Don’t just randomly accuse people of making bad faith arguments. I’ve use R a lot and while it has its issues, it has a much better interface than numpy.

It’s hardly random. R functions could be noun-adjective or adjective-noun or underscored or camelCase or dotted ... and that’s just naming conventions. If someone, like you, says that Numpy sucks, but R is the real masterpiece, then that’s totally insincere. The good thing about R is that it’s free, the C FFI is OK and R-studio is decent. It’s API is a patchwork. It’s cool to say Java sucks or Python sucks on the orange site because they’re popular and if you say something popular sucks, well you must be a pretty cool guy who is smarter than all those rubes out there. The arguments are almost always total nonsense, though.

Re: Pyston v2: Faster Python

#196

Post by sk2020 »

throwaway894345 wrote: post #164
sk2020 wrote: post #71

Earlier quoted context omitted.

So who does it right? If all these APIs suck compared to an imaginary perfect library, then that isn’t a useful comparison. Also, if an endpoint is spending minutes to respond, then I would think actually profiling the application would be a good start. Maybe researching prior art in the problem domain would be good too. If nobody can be bothered to explore the several solutions to distributing pandas computations ov…

That’s quite a rant with a lot of assumptions. Just about every library has a better API than matplotlib or pandas. Requests has a pretty good API IMO. The team who was responsible for the slow endpoint did investigate dask and alternatives, and they probably will end up on something like spark because they didn’t feel like they have better options. Maybe our team is just stupid and Python isn’t for mere mortals, I d…

I would certainly hope a minimal HTTP library would be simpler than a suite of functions to manipulate and plot tabular data.

“My application is slow, the language sucks!” Doesn’t indicate a very serious investigation into the problem.

Re: Pyston v2: Faster Python

#197

Post by throwaway894345 »

pjmlp wrote: post #192
throwaway894345 wrote: post #185

Earlier quoted context omitted.

If that's how you define "generic", then Go is also generic by virtue of `interface{}`.

No, because interface{} in an empty type that needs to be type cast to the actual type before use. Dynamic languages do that implicitly. Implement max in Go with interface{} without casts and reflection: def max(a, b): if a >= b: a else: b

I'm not aware of any definition of 'generic' that prohibits casting. It certainly seems like a very arbitrary condition. Basically I'm familiar with two definitions:

1. The abstract idea of writing an algorithm that supports a variety of types. This allows for casting, reflection, dynamic typing, etc.

2. The specific idea of a type system that allows for parameterized types (aka "typesafe generics"). This definition excludes castng, reflection, and dynamic typing.

Typically "typesafe generics" is what people talk about when they discuss "generics", but since you chose to pick the "dynamically typed languages are generic" nit, I assumed you were talking about (1).

Re: Pyston v2: Faster Python

#198

Post by [deleted] »

disgruntledphd2 wrote: post #187
semi-extrinsic wrote: post #172

Earlier quoted context omitted.

My gripe with Julia etc. as replacements, is that Python is duct tape. I don't need fast duct tape, I need duct tape that is understood and used by essentially everyone I work with, and that has native, fast handling of large amounts of data (NumPy, Pandas). Good user experience as duct tape. From my perspective Julia is sacrifising some amount of "duct tape UX" to gain speed, and that's the wrong direction. Whenever…

I agree with you right now, but think that Julia will end up dominating over the long-term, because dropping down into another language absolutely sucks for data scientists without engineering support. Interestingly, R is probably a better UX for statisticians/data scientists than Python is (almost all the good parts of Numpy/Pandas were in R first), but it really suffers from not being well known by developers. To b…

[deleted]

Re: Pyston v2: Faster Python

#199

Post by theamk »

globular-toast wrote: post #179
theamk wrote: post #127

Wanted to see redistribution rules, and was surprised to see there is no license anywhere for the binaries... The closest thing I found is "copyright" file inside .deb: Copyright: 2020 The Pyston Team License: Closed source, all rights reserved. I guess it means no one should be touching the file, as they haven't even granted access to run it.

> I guess it means no one should be touching the file, as they haven't even granted access to run it. Since when does someone need to explicitly grant you permission to run a program on your own computer?

Running is probably ok, but giving to others? I am not sure.

Those limitations existed since before the computer time, when the copyright law was passed. For example, even you own a book, there are certain things you can not do, like duplicate it and sell copies.

In case of software, here is how the law works [0]

> When you make a creative work (which includes code), the work is under exclusive copyright by default. Unless you include a license that specifies otherwise, nobody else can copy, distribute, or modify your work without being at risk of take-downs, shake-downs, or litigation.

There are "fair use" terms, which allow some things without permissions -- but things like "copying the binary to company-internal repo so CI runners can pick it" really need explicit permissions if you want to be above the board.

[0] https://choosealicense.com/no-permission/

Re: Pyston v2: Faster Python

#200

Post by willseth »

ihnorton wrote: post #155
willseth wrote: post #137

Earlier quoted context omitted.

Pandas is not some mysterious black box. If you need predictable runtime performance or bounded memory usage, you have to figure it out. Pandas doesn't inherently have a staggering or unpredictable amount of overhead, given that it's a statistical analysis package. There are ways to mitigate Pandas memory usage (10x is a sign that something has gone very horribly wrong), and sometimes Pandas is simply the wrong tool…

10x reflects both experience and expert recommendations. You may recognize the author [1]: > Nowadays, my rule of thumb for pandas is that you should have 5 to 10 times as much RAM as the size of your dataset [1] https://wesmckinney.com/blog/apache-arrow-pandas-internals/

I don't doubt Wes' upper bound for Pandas OOTB, without optimization. The context was web applications. If you're seeing 10x on a web app, either something is wrong or you probably shouldn't be using Pandas.
Post reply on HN