PyPy 4.0.0 Released – A Jit with SIMD Vectorization and More • Hacker News Board

Re: PyPy 4.0.0 Released – A Jit with SIMD Vectorization and More

#131

Post by oberstein » Fri, Oct 30, 2015, 2:00 AM UTC

vegabook wrote: post #117

oberstein wrote: post #114

Earlier quoted context omitted.

> Let me warn you. If you persist with your propaganda on Python3, then GOLANG will be the future of Python. Does that register with you guys? This abstract future of Python has already been ceded to something that is not Python, just as once the abstract future of Perl was Python. Ruby once tried to take the mantle but failed. Go is now taking a lot of mindshare (from 2 or 3 more I wonder?) and I agree the Python 3…

I'm currently registered on the Ocaml mooc ;-) Very intrigued by Nim too. Yes I find Golang to be dry and uninspiring, but as per one of my other comments, I get 20x the performance, and the concurrency model beats Asyncio hands down. Honestly I needed to get 360 gigabytes of JSON into Cassandra and Python was going to take 6 days (there is some light conditionality on each datapoint preventing a raw dump). I took 1.…

It will be interesting to see what the final winner of GPU programming looks like. There's also the possibility of FPGAs making their way into consumer hardware, and right now the pure-Python library MyHDL is a great choice for those. There are a few attempts at abstracting over OpenCL like https://github.com/eholk/harlan, and some intriguing domain specific attempts like http://halide-lang.org/ but I think it will be a while yet before something takes over. I also don't think whatever it is can succeed without corporate backing.

Re: PyPy 4.0.0 Released – A Jit with SIMD Vectorization and More

#132

Post by camaraj » Fri, Oct 30, 2015, 2:07 AM UTC

lqdc13 wrote: post #90

dr_zoidberg wrote: post #17

Earlier quoted context omitted.

I think you guys are saying the same thing. Ie, because they don't need to switch languages, and PyPy is a JIT engine for Python, these tons of Python code would benefit from if the devs were aware of PyPy .

Devs are already aware of PyPy. If it was as easy as being aware, everyone would switch tomorrow. The problem is that tons of code uses C extensions created with swig or python boost and those take a lot of time to port to PyPy. Another problem is that PyPy is sometimes slower/less memory efficient than CPython when you wrote the code that is optimized for CPython. That is abusing dictionaries and very limited OO use…

PyPy has higher memory overhead for very small units of work but usually once you do more than a couple of seconds of work it usually uses significantly less memory than CPython. As far as I'm concern who cares if a Hello World program uses more memory or some small trivial few lines of code. I'm more interested in real world code which my experience has been that PyPy uses significantly less memory than CPython. If you know of a case where this is not true report it as I'm sure the PyPy devs will consider it a bug.

PyPy also has a slower startup time/overhead so very small amounts of work may end up slower than CPython but in general larger chunks of work will be faster if not its a bug and should be reported.

There is currently some work going on to improve the C extension issue. It's currently in its beginning stages and its goals are to provide both a significant amount of compatibility with existing extensions and a significant reduction in the overhead to call out to C extensions over what is currently implemented in PyPy. On the performance side the new approach was said to remove about 40% of the existing overhead. There are tons of corner cases that still need to be worked on for the new approach to become production worthy and there is no guarantee that there will be no show stoppers that come up or that the tail of corner cases that need to be dealt with becomes a nightmare and the approach is abandoned. If this works succeeds, it will be a game changer as suddenly the majority of libraries would then become compatible with PyPy.

Re: PyPy 4.0.0 Released – A Jit with SIMD Vectorization and More

#133

Post by vegabook » Fri, Oct 30, 2015, 6:50 AM UTC

andrepd wrote: post #115

vegabook wrote: post #103

Earlier quoted context omitted.

That is exactly what I am suggesting. There is no urgent necessity to add any new core language features to 2.7. That's what you 3 guys just don't get. I certainly don't need asyncio as much as I need vectorised numerics, and if I needed asyncio, I had at least 5 libraries already. I most definitely will never be type annotating Python. Cloud cuckoo land. Let me warn you. If you persist with your propaganda on Python…

I would suggest toning down the arrogance. Just because you don't personally need some features does not mean that people who want the language to progress and not stagnate are idiots.

That's rich when it is the 2.7 users like myself who are constantly bombarded with arrogance and condescension. Nobody wants Python to stagnate. The argument is which direction it should take.

Re: PyPy 4.0.0 Released – A Jit with SIMD Vectorization and More

#134

Post by pjmlp » Fri, Oct 30, 2015, 7:46 AM UTC

nickpsecurity wrote: post #93

pjmlp wrote: post #55

Earlier quoted context omitted.

I don't understand numerical analysis as business applications. For me business applications are what you would use Cobol, Clipper, Java, Delphi, C#, Eifel, .... Of course Fortran makes sense for number crushing, but that is the language domain, not doing CRUD, ETL or distributed computing stuff (not counting MPI here).

Accounting, spreadsheets, transaction processing, business intelligence?

Of course those are business applications as well.

Do you intend to refer to a product done in those areas Fortran in modern days, maybe with Fortran 2008?

Outside scientific computing and heavy number crushing algorithms and libraries, I don't see a use for it.

Re: PyPy 4.0.0 Released – A Jit with SIMD Vectorization and More

#135

Post by MatthewWilkes » Fri, Oct 30, 2015, 10:02 AM UTC

JulianWasTaken wrote: post #77

MatthewWilkes wrote: post #9

> We would also like to thank our contributors (7 new ones since PyPy 2.6.0) and encourage new people to join the project. PyPy has many layers and we need help with all of them This doesn't match my experience of the PyPy project. I found a tiny bug in the stdlib matching against CPython, went into IRC to ask a question about test running to be sure I got it right and was quickly engaged in conversation about why I…

This is very very disingenuous. Here are the public IRC logs for what I assume you're referring to (which is from July 2014 by the way): https://botbot.me/freenode/pypy/2014-07-02/?msg=17362202&pag... You found a bug in an old version of PyPy. A core developer told you you might want to try a new one. No one stole your shot out from under you. You would undoubtedly be welcomed to contribute. Don't post FUD. Disclaime…

I apologise that I mis-remembered and characterised intending to fix a bug, investigating, and finding out it was already fixed as just fixing a bug, however I believe the rest stands.

The fact that I wanted to run tests and couldn't, and the team went on to why I wanted to run tests and looked to solve my problem is my complaint.

Also, to be clear, I'm not trying to say the PyPy channel is unfriendly, it certainly isn't. It's just not set up to help potential new contributors get started.

Re: PyPy 4.0.0 Released – A Jit with SIMD Vectorization and More

#136

Post by trentnelson » Fri, Oct 30, 2015, 2:01 PM UTC

How up-to-date is this? http://doc.pypy.org/en/latest/windows.html#what-is-missing-f...

I've read that a few times and I always come away confused -- it sounds like a huge fundamental type change needs to be made by someone well versed in the PyPy internals. i.e. not something you'd typically defer to an external contributor.

I'd like to experiment with PyPy and PyParallel, but I'm basically exclusively 64-bit Windows, so it sounds like a non-starter.

Re: PyPy 4.0.0 Released – A Jit with SIMD Vectorization and More

#137

Post by CuriouslyC » Fri, Oct 30, 2015, 3:48 PM UTC

vegabook wrote: post #117

oberstein wrote: post #114

Earlier quoted context omitted.

> Let me warn you. If you persist with your propaganda on Python3, then GOLANG will be the future of Python. Does that register with you guys? This abstract future of Python has already been ceded to something that is not Python, just as once the abstract future of Perl was Python. Ruby once tried to take the mantle but failed. Go is now taking a lot of mindshare (from 2 or 3 more I wonder?) and I agree the Python 3…

I'm currently registered on the Ocaml mooc ;-) Very intrigued by Nim too. Yes I find Golang to be dry and uninspiring, but as per one of my other comments, I get 20x the performance, and the concurrency model beats Asyncio hands down. Honestly I needed to get 360 gigabytes of JSON into Cassandra and Python was going to take 6 days (there is some light conditionality on each datapoint preventing a raw dump). I took 1.…

You're looking at this from the SV startup echo-chamber perspective.

People use Python because the language gets out of the way fairly quickly to let you get things done fairly quickly. Sure, it isn't a rigorous engineering language, but the ratio of small projects that just need something good enough and easy to work with to large/complex projects requiring careful engineering is staggeringly large. While Python is definitely cooling off; people have been working with it for long enough to want something better - unfortunately there just isn't anything on the radar that is significantly enough better to overcome Python's third party library momentum at the moment.

Believe me, I'd love a language with a really flexible/optional functional type system, better metaprogramming facilities, a better concurrency design, slightly more consistent syntax, etc - just as long as it is still easy to think in and convenient to use like Python.

Re: PyPy 4.0.0 Released – A Jit with SIMD Vectorization and More

#138

Post by dragonwriter » Fri, Oct 30, 2015, 3:59 PM UTC

CuriouslyC wrote: post #137

vegabook wrote: post #117

Earlier quoted context omitted.

I'm currently registered on the Ocaml mooc ;-) Very intrigued by Nim too. Yes I find Golang to be dry and uninspiring, but as per one of my other comments, I get 20x the performance, and the concurrency model beats Asyncio hands down. Honestly I needed to get 360 gigabytes of JSON into Cassandra and Python was going to take 6 days (there is some light conditionality on each datapoint preventing a raw dump). I took 1.…

You're looking at this from the SV startup echo-chamber perspective. People use Python because the language gets out of the way fairly quickly to let you get things done fairly quickly. Sure, it isn't a rigorous engineering language, but the ratio of small projects that just need something good enough and easy to work with to large/complex projects requiring careful engineering is staggeringly large. While Python is…

> Believe me, I'd love a language with a really flexible/optional functional type system, better metaprogramming facilities, a better concurrency design, slightly more consistent syntax, etc - just as long as it is still easy to think in and convenient to use like Python.

As someone who, among dynamic languages, dismissed Perl quite a while back as not as useful as Python and Ruby, I'm starting to think Perl 6 may be the language that hits that spot.

Re: PyPy 4.0.0 Released – A Jit with SIMD Vectorization and More

#139

Post by nickpsecurity » Fri, Oct 30, 2015, 5:08 PM UTC

pjmlp wrote: post #134

nickpsecurity wrote: post #93

Earlier quoted context omitted.

Accounting, spreadsheets, transaction processing, business intelligence?

Of course those are business applications as well. Do you intend to refer to a product done in those areas Fortran in modern days, maybe with Fortran 2008? Outside scientific computing and heavy number crushing algorithms and libraries, I don't see a use for it.

No, I'm just saying they're business applications. A segment of business applications are numerically focused. Fortran handles them well. So, in that segment, it can be considered a language for business applications with numeric focus.

I'm not saying anyone should use it today lol. Julia and R are the best for that sort of stuff. Way the hell ahead of Fortran. Although I thought about making a nice web application in Fortran just to screw with people who eventually try to extend it and gasp in horror. ;)

http://www.aim.env.uea.ac.uk/aim/info/fortran.pdf

Re: PyPy 4.0.0 Released – A Jit with SIMD Vectorization and More

#140

Post by vegabook » Fri, Oct 30, 2015, 6:27 PM UTC

gvalkov wrote: post #126

vegabook wrote: post #103

Earlier quoted context omitted.

That is exactly what I am suggesting. There is no urgent necessity to add any new core language features to 2.7. That's what you 3 guys just don't get. I certainly don't need asyncio as much as I need vectorised numerics, and if I needed asyncio, I had at least 5 libraries already. I most definitely will never be type annotating Python. Cloud cuckoo land. Let me warn you. If you persist with your propaganda on Python…

You contribute the same borderline FUD in every thread that is related to Python. It's unfortunate that Python is not the language you want it to be, but consider just how dismissive you are of the needs of others - frankly, it comes across as arrogant.

FUD? I have >100k lines of highly sophisticated numerical analysis Numpy/Python code in fixed income which has cost literally millions of dollars. I'm not going to let that investment get toyed with by the webified 3.x crew without vigorously stating my point.