Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Socketify.py: Http/Https and WebSockets servers for PyPy3 and Python3

github.com

Post reply on HN
Print view View original

161–170 of 192 posts

Re: Show HN: Socketify.py: Http/Https and WebSockets servers for PyPy3 and Python3

#161

Post by qorrect »

throwaway894345 wrote: post #127
cirospaciari wrote: post #71

Earlier quoted context omitted.

PyPy work well with CPython modules but the performance is not that great, with CFFI performance is really good in PyPY, HPy project will change this https://hpyproject.org/

Last I checked, Pypy wasn’t compatible with many important packages in the Python ecosystem. As of a couple years ago, there was no trustworthy, supported package for talking to a Postgres database with Pypy because psycopg2 wasn’t supported.

When is the last time you checked ?

I have run it with pandas+numpy and postgres+timescale without issue for the last year.

Re: Show HN: Socketify.py: Http/Https and WebSockets servers for PyPy3 and Python3

#162

Post by dymk »

pdpi wrote: post #147
okaleniuk wrote: post #13

Good point! However, you bring in a common misconception I'm fighting with within my company for a long time. Python is not an interpreted language. There is no such thing as an "interpreted language", a language is just a set or rules and keywords. Everything you can fit into Backus–Naur form is already a language even if it doesn't have any implementation nor compiler neither interpreter. Just as a piece of evidenc…

> There is no such thing as an "interpreted language", a language is just a set or rules and keywords. Sure there is. Even if it is technically possible to write an interpreter for a "compiled" language, or a compiler for an "interpreted" language, language design really pushes you towards one or the other. "Technically possible" can also represent a very real challenge, as that set of rules and keywords might make a…

> Inversely, You can write an interpreter for C++, but features like `consteval` and `constexpr` are meaningless without ahead-of-time compilation.

No they're not, `constexpr` and `consteval` indicate where you can use an expression, not when in the compilation/interpretation pipeline they need to be evaluated. The spec does not say that C++ has to be compiled, nor does the spec say "A compiler must execute `constexpr` expressions prior to entering `main()`".

If you think this is just being pedantic, well, that's kind of the point of a spec.

Re: Show HN: Socketify.py: Http/Https and WebSockets servers for PyPy3 and Python3

#163

Post by jerf »

A good example of why web servers should be measured in seconds per request rather than requests per second. As you trim little tiny bits off the seconds per request, the requests per second go skyrocketing off to infinity, but unless you're doing zero work per request, that's a dubiously useful metric.

1,000,000 requests per second on what I think is an 8-core system (?) is 1/1,000,000 * 8 = 8us per request. 1,250,000 requests per second would be 6.4 us per request. Is this really your make or break issue? There's a set of people who can say "yes" to that. There's a set of people who think it is yes. The latter is much larger than the former.

(Although arguably the ones worst off are the ones for whom it is their make-or-break issue, but they don't realize it....)

Re: Show HN: Socketify.py: Http/Https and WebSockets servers for PyPy3 and Python3

#164

Post by Thaxll »

When I see that: https://github.com/cirospaciari/socketify.py/blob/main/bench...

It's kind of hopeless, Python still needs to fork per core to get any performance? So if you have 8 cores you're actually running 8 processes, so 8 DB pool etc ...

Re: Show HN: Socketify.py: Http/Https and WebSockets servers for PyPy3 and Python3

#165

Post by MobiusHorizons »

okaleniuk wrote: post #13

Good point! However, you bring in a common misconception I'm fighting with within my company for a long time. Python is not an interpreted language. There is no such thing as an "interpreted language", a language is just a set or rules and keywords. Everything you can fit into Backus–Naur form is already a language even if it doesn't have any implementation nor compiler neither interpreter. Just as a piece of evidenc…

There are language features of dynamic languages like python that prevent them from being compiled to machine code. That is to say, the compiler can’t know what assembly to output from the code itself, it also has to know some type information which isn’t available until runtime. I would encourage you not to fight the classification of interpreted language as your primary argument. You will likely lose credibility with the listener before you have a chance to make any good points. You could for instance say “just because the reference implementation is slow doesn’t mean there is no hope for the language becoming fast”. There is some truth to the argument. JavaScript has become a reasonably fast language with all the engineering effort spent on engines for it. But I would also spend some time learning about what makes dynamic languages hard to compile. I think you will find that some features of the language must be dropped to achieve fully aot compilation. The classic example would be eval() which invokes an interpreter from inside the language, but there are usually other features.

Re: Show HN: Socketify.py: Http/Https and WebSockets servers for PyPy3 and Python3

#166

Post by cirospaciari »

Thaxll wrote: post #164

When I see that: https://github.com/cirospaciari/socketify.py/blob/main/bench... It's kind of hopeless, Python still needs to fork per core to get any performance? So if you have 8 cores you're actually running 8 processes, so 8 DB pool etc ...

Python GIL is the problem for multithreading, but I think I have a solution to not need 8 DB pools, soon o will post about it. But yeah it's a waste

Re: Show HN: Socketify.py: Http/Https and WebSockets servers for PyPy3 and Python3

#167

Post by LtWorf »

no_circuit wrote: post #150
LtWorf wrote: post #37

Earlier quoted context omitted.

Author of typedload here. Pydantic is indeed really slow ( https://ltworf.github.io/typedload/performance.html ) typedload is faster and written in pure python (unlike pydantic). I know they are rewriting it in rust… but I don't know what will come out of it. After all my pure python implementation already manages to beat pydantic's binary, and when unions are in use, also apischema's binary.

The perf part of the tests just seems to be a microbenchmark for seeing how fast the various frameworks can parse a 30000x300 dict of strings representing numbers [1]. If that is all one's application does, and can use your library in their organization/team, that's great. However a 2-3x performance boost for the parsing stage for a use case like an API call might not matter when that could be overshadowed by validat…

> a microbenchmark for seeing how fast the various frameworks can parse a 30000x300 dict of strings representing numbers [1].

Did you somehow miss all the other tests, even thought they are higher on the page? The important test is shown first: loading objects.

I'm not trying to benchmark my wifi or my disk. The IO time is not included there on purpose. Of course a bigger application that does other things wouldn't see this huge difference in performance. But I'm testing the performance of a library here.

typedload can use validators, but since it doesn't reimplement attrs/dataclass, I had no interest in testing those… it'd be a race between libraries I didn't write.

typedload's exceptions contain enough detail to tell the end user what went wrong.

when encountering errors in lists, typedload slows down due to keeping compatibility with python 3.7. However in my test with loading objects, despite the slowdown it remains faster than pydantic.

Re: Show HN: Socketify.py: Http/Https and WebSockets servers for PyPy3 and Python3

#168

Post by simonw »

Thaxll wrote: post #164

When I see that: https://github.com/cirospaciari/socketify.py/blob/main/bench... It's kind of hopeless, Python still needs to fork per core to get any performance? So if you have 8 cores you're actually running 8 processes, so 8 DB pool etc ...

Why does that feel hopeless to you?

Running one process per core has been working well for scaling huge websites for decades at this point.

Re: Show HN: Socketify.py: Http/Https and WebSockets servers for PyPy3 and Python3

#169

Post by cirospaciari »

simonw wrote: post #168
Thaxll wrote: post #164

When I see that: https://github.com/cirospaciari/socketify.py/blob/main/bench... It's kind of hopeless, Python still needs to fork per core to get any performance? So if you have 8 cores you're actually running 8 processes, so 8 DB pool etc ...

Why does that feel hopeless to you? Running one process per core has been working well for scaling huge websites for decades at this point.

nodejs do this and actually Go fiber uses prefork to do this too

Re: Show HN: Socketify.py: Http/Https and WebSockets servers for PyPy3 and Python3

#170

Post by tomwojcik »

cirospaciari wrote: post #166
Thaxll wrote: post #164

When I see that: https://github.com/cirospaciari/socketify.py/blob/main/bench... It's kind of hopeless, Python still needs to fork per core to get any performance? So if you have 8 cores you're actually running 8 processes, so 8 DB pool etc ...

Python GIL is the problem for multithreading, but I think I have a solution to not need 8 DB pools, soon o will post about it. But yeah it's a waste

Oh, I have a pretty fresh news for you.

https://github.com/python/peps/pull/2955

Post reply on HN