81–90 of 259 posts
Earlier quoted context omitted.
What sucked about FastAPI?
Not fast enough! I used it to call llama.cpp server but it would crash if requests were "too fast". Calling the llama.cpp server directly solved the issue.
Earlier quoted context omitted.
And I guess what I don't understand is why people choose Python for these use cases. I am not in the "Rustify" everything camp, but Go + C, Java + JNI, Rust, and C++ all seem like more suitable solutions.
Why do people use python for anything beyond glue code? Because it took off, and machine learning and data science now rely on it. I think Python is a terrible language that exemplifies the maxim "worse is better". https://en.wikipedia.org/wiki/Worse_is_better
Earlier quoted context omitted.
> `multiprocessing` works fine for serving HTTP requests Not if you use Windows, then it's a mess. I have a suspicion that people who say that the multiprocessing works just fine never had to seriously use Python on Windows.
Why is it a mess? What's wrong with it on Windows?
* All Python webservers that somewhat support multiprocessing on Windows disable the IOCP asyncio event loop when using more than one process (because it breaks in random ways), so you're left with the slower select() event loop which doesn't support more than 512 connections.
Earlier quoted context omitted.
What sucked about FastAPI?
Not fast enough! I used it to call llama.cpp server but it would crash if requests were "too fast". Calling the llama.cpp server directly solved the issue.
Earlier quoted context omitted.
The "ray" library makes running python code on multi core and clusters very easy.
Interesting - looking at their homepage they seem to lean heavily into the idea that it's for optimising AI/ML work, not multi-process generally.
You can do whatever you want in the workers, I parse JSONs and write to sqlite files.
Earlier quoted context omitted.
`multiprocessing` works fine for serving HTTP requests or do some other subset of embarrassingly-parallel problems.
> `multiprocessing` works fine for serving HTTP requests Not if you use Windows, then it's a mess. I have a suspicion that people who say that the multiprocessing works just fine never had to seriously use Python on Windows.
Earlier quoted context omitted.
"Ray" can share python objects memory between processes. It's also much easier to use than multi processing.
How does that work? I'm not familiar with Ray, but I'm assuming you might be referring to actors [1]? Isn't that basically the same idea as multiprocessing's Managers [2], which also allow client processes to manipulate a remote object through message-passing? (See also DCOM.) [1] https://docs.ray.io/en/latest/ray-core/walkthrough.html#call... [2] https://docs.python.org/3/library/multiprocessing.html#manag...
More than seeing it in main, I'm happy for the "python thread slow" meme officially going away now.