451–460 of 460 posts
Earlier quoted context omitted.
Yeah but for JS style async you'd use probably use an event loop in Python, not multi processing.
Yes. But, frankly, async is also simpler in JS than in Python: e.g. no need to start a reactor loop.
Earlier quoted context omitted.
What has GIL to do with the thread model vs asyncio? asyncio is also single threaded, so cooperative (and even preemptive) green threads would have been a fully backward compatible option. JS never had an option as, as far as I understand, callback based async was already the norm, so async functions were an improvement over what came before. C# wants to be an high performance language, so using async to avoid alloca…
> What has GIL to do with the thread model vs asyncio? Obviously the fact that the GIL prevents effient use of threads, so asyncio becomes the way to get more load from a single CPU by taking advantage of the otherwise blocking time.
Earlier quoted context omitted.
Yeah I mean it's pretty bad, but the upside is that it will run everywhere without having to install anything and there are lots and lots of answered questions out there if you run into issues. Which I doubt is true for Nim. Using niche stuff means you're on your own in terms of support. And with how dependency happy everything is these days, I avoid trendy projects like the plague.
Nim is compiled, so a dependency on a runtime or a VM is not an issue there, at least.
Now if it got transpiled to bash for example, that might be actually practical. Cuts dev time, still enables multiplatform usage and random edits.
Earlier quoted context omitted.
I agree. The rule of thumb I follow is that if the list comp doesn't fit on one line, whatever I'm doing is probably too complicated for a list comp.
But why? The only reason is that error messages are awful and intermediate values can't be named. There is in principle no reason why list comprehension need to be worse than straight line for loops. The deciding factor to chose betwee the two should be whether I'm writing something for side effects as opposed to for its value.
Earlier quoted context omitted.
Cool, they should start now. As a python dev, pythons multiprocess/multithreading story is one the largest pain points in the language. Single threaded performance is not that useful while processors have been growing sideways for 10 years. I often look at elixir with jealousy.
> As a python dev, pythons multiprocess/multithreading story is one the largest pain points in the language. Hmm, how is that so? As a python dev as well, I don't have much complaint with multiprocessing. The API is simple, it works OK, the overall paradigm is simple to grok, you can share transparently with pickle, etc.
The ProcessPoolExecutor is nice but shouldn't be necessary.
Earlier quoted context omitted.
"Personally, I like that Python has kept the GIL so far because I would never run a 24/7 server in Python and I am happy to use it very frequently for single-threaded scripting tasks." Just as a side-note - my prior gig used Python on both the Server and Data Collection industrial systems. It was very much a 24x7x365 must-never-go-down type of industrial application, and, particularly when we had a lot of data-source…
I know that people do use python for 24/7 "must not fail" applications. I'm just not smart enough to write python that I would trust like that. Python comes with a tremendous number of foot guns and you have to find them because there is no compiler to help, and it can be a real pain to try to understand what is happening in large (>10,000 line) python programs.
You just have to isolate the complicated parts, define sensible interfaces for them, and make sure they are followed with type hints and a good type checker.
Earlier quoted context omitted.
If we’re going to leave Python as a scripting language (fine by me), can we get the machine learning community to swap to something better suited? It strikes me as a bit of a waste of resources to keep stapling engineering effort into the Python ML/data ecosystem when it’s basically a crippled language capable of either: mindlessly driving C binaries, or scripting simple tasks. What other performance, feature and tec…
From what I can tell, the ML community is moving toward Julia. I don't think anyone predicted that they would end up locked into Python so heavily.
Also, I hate the idea of re-building all the code when the program starts. Python's JIT can at least ignore the performance-critical code that's written in C++.
Earlier quoted context omitted.
There are a lot of dynamically typed languages that are significantly faster than python. Late binding issues can be effectively worked around.
Do you have an example of a dynamically typed language where, say, addition of two lists of doubles would be significantly faster than in Python?
Python version:
from random import randrange
from time import time
def main():
L = [float(randrange(2**52, 2**53)) for _ in range(20000000)]
M = [float(randrange(2**52, 2**53)) for _ in range(20000000)]
t0 = time()
N = [ x+y for x,y in zip(L, M) ]
print('Concluded in', round(1000*(time() - t0)), 'millisec.')
main()
Python 3.10.5 (main, Jun 9 2022, 00:00:00) [GCC 12.1.1 20220507 (Red Hat 12.1.1-1)] on linux
Type "help", "copyright", "credits" or "license()" for more information.
============ RESTART: /run/media/user/KINGSTON/benchmark_doubles.py ============
Concluded in 1904 millisec.
| L M N t0 |
Transcript clear.
L := (1 to: 2e7) collect:
[ :each | (( 2 raisedTo: 52 ) to: ( 2 raisedTo: 53 )) atRandom asFloat ].
M := (1 to: 2e7) collect:
[ :each | (( 2 raisedTo: 52 ) to: ( 2 raisedTo: 53 )) atRandom asFloat ].
t0 := DateAndTime now.
M := L with: M collect: [ :x :y | x + y ].
Transcript
show: 'Concluded in '
,
((DateAndTime now - t0) asMilliSeconds asInteger ) asFloat asString
, ' millisec.';
cr.
Concluded in 914.0 millisec.Earlier quoted context omitted.
"Personally, I like that Python has kept the GIL so far because I would never run a 24/7 server in Python and I am happy to use it very frequently for single-threaded scripting tasks." Just as a side-note - my prior gig used Python on both the Server and Data Collection industrial systems. It was very much a 24x7x365 must-never-go-down type of industrial application, and, particularly when we had a lot of data-source…
I know that people do use python for 24/7 "must not fail" applications. I'm just not smart enough to write python that I would trust like that. Python comes with a tremendous number of foot guns and you have to find them because there is no compiler to help, and it can be a real pain to try to understand what is happening in large (>10,000 line) python programs.
I don't think anybody expects a recent graduate from computing science, with maybe 4 or 5 classes that used python under their belt (and maybe a co-op or two) to be writing robust code (perhaps in any language).
But, after working with Sr. Engineers who do so, and understanding how to catch your exceptions and respond appropriately, how to fail (and restart) various modules in the face of unexpected issues (memory, disk failures, etc...) - then a python system is just as robust as any other language. I speak from the experience of running them in factories all over the planet and never once (outside of power outages - and even there it was just downtime, not data loss) in 2+ years seeing a single system go down or in any way lose or distort data. And if you want more performance? Make good use of numpy/pandas and throw more cores/processes at the problem.
Just being aware of every exception you can throw (and catching it) and making robust use of type hinting takes you a long way.
Also - and this may be more appropriate to Python than other languages that are a bit more stable, an insane amount of unit and regression testing helps defend quite a bit from underlying libraries like pandas changing the rules from under you. The test code on these project always seemed to outweigh the actual code by 3-5x. "Every line of code is a liability, Every test is a golden asset." was kind of the mantra.
I think that what makes python different from other languages, is that it doesn't enforce guardrails/type checking/etc... As a result, it makes it trivial for anyone who isn't an engineers to start blasting out code that does useful stuff. But, because those guardrails aren't enforced in the language, it's the responsibility of the engineer to add them in to ensure robustness of the developed system.
That's the tradeoff.
Earlier quoted context omitted.
Yes. But, frankly, async is also simpler in JS than in Python: e.g. no need to start a reactor loop.
Starting the event loop is no worse than any setup of a main function, it’s a oneliner: asyncio.get_event_loop().run_until_completion(my_async_main)