211–220 of 376 posts
Earlier quoted context omitted.
This is fine as long as you can convince management to spend the money to rewrite your software. That's usually a hard sell though. In my experience this plan usually ends up with a python monstrosity that everyone hates but is forced to deal with forever.
Type hints and dataclasses are a game-changer for Python. Much easier to reason about programs that use them.
At that point it makes more sense to use a statically typed language.
Earlier quoted context omitted.
Many other people "know" about the GIL, to the extent of believing there's no point using threads in python "because of the GIL". I had a funny such experience lately in a job interview. I told the interviewer his misconception could be falsified with ~10 LOC summing a list with 2 threads.
Ok, I see some comments (rightfully) asking for less talk and more code. # main.py import random from concurrent.futures import ThreadPoolExecutor as Pool items = [random.random() for _ in range(10 ** 7)] def run(items, n): step = len(items) // n with Pool(max_workers=n) as ex: res = [ex.submit(sum, items[i*step : (i+1)*step]) for i in range(n)] return sum(r.result() for r in res) if __name__ == '__main__': import ti…
(2) You're not measuring what you think you are measuring.
Take out the Pool altogether:
def run(items, n):
step = len(items) // n
res = [sum(items[i*step : (i+1)*step]) for i in range(n)]
return sum(r for r in res)
1 0.170
2 0.170
3 0.147
4 0.145
5 0.142
6 0.142
7 0.138
8 0.137
9 0.135
10 0.138
EDIT: As an aside, you can speed this up 20x by using Numpy:
items = numpy.array([random.random() for _ in range(10 ** 7)])
def run(items, n):
return numpy.sum(items)
1 0.007
Earlier quoted context omitted.
I semi-seriously think the entire modern shape of the cloud is a result of Ruby being really slow. Back when people were writing their backend business apps in C++, COBOL, Java, etc, if there was ever a performance problem, you could usually just get a slightly bigger machine and grow your thread pools a bit. But once the web took off and Ruby exploded onto it, you couldn't do that, because it's an order of magnitude…
The push for the need of scaling out started with Ruby and Python's lack of performance. The reason being pushed at the time was, "developer time was more expensive than hardware." Well, that didn't count the amortization of developer time over the lifetime of the product once the product was developed.
Yes, I’m always shocked by just how much performance overhead most languages have compared to C and similar lower level languages. It is a price worth paying for better language ergonomics, but I do wonder whether Rust might be able to give us the best of both worlds here.
The idea that GCed languages in general have Python-like performance is a dangerous myth. Languages that are managed but not interpreted (e.g. Java, OCaml, Haskell, C#, Swift) have performance characteristics that are much closer to C than to Python.
Carmack is moving to AI and inevitably has to deal with a lot of Python, which still bottlenecks process here an there despite all the effort to move computation to C extensions. I have a really high hope that he detours a bit and creates very-very good non-python tooling for ML.
Earlier quoted context omitted.
Yes, if by "optimizing programmers" you mean "optimizing the manager's corporate structure footprint and bonus incentives". If the choice is between hiring one good C++ programmer or 15 really dumb Python "backend engineers" (and a team of QA's and sysadmins to support them), what do you think your pointy-hared corporate boss would chose?
You never choose to hire dumb programmers. The choice is between 15 good cpp programmers and 4 good python programmers.
Last round, for the second best candidate who we hired at 130% the salary we initially offered, after the best candidate was snatched under our nose for what we were told was twice the salary we were offering:
>"Last question, I saw that you wrote a 100 line function here, is this because you ran out of time?"
>>"No. I don't like to break up my functions. Having many small functions is confusing and bad practice."
>What am I doing with my life?
You can choose a language that optimizes your hardware, or you can choose a language that optimizes your programmers. 99% of the time optimizing the programmers is the right call.
Yes, if by "optimizing programmers" you mean "optimizing the manager's corporate structure footprint and bonus incentives". If the choice is between hiring one good C++ programmer or 15 really dumb Python "backend engineers" (and a team of QA's and sysadmins to support them), what do you think your pointy-hared corporate boss would chose?
Personal productivity is generally going to be slower in C++ than it is in Python. In most situations you would gain more productivity out of a similarly skilled Python dev as you are a C++ dev, so you'd probably need to hire more C++ developers.
Unless your argument is "C++ devs are smart and Python devs are dumb", in which case, let's not start calling people names over the language they use.
Earlier quoted context omitted.
The Internet uses on the order of 10% of the world's electricity. A 10x difference is huge .
Out of curiosity - what's the source on that? Most figures I've seen put datacenter usage an order of magnitude lower: e.g. https://www.iea.org/reports/tracking-buildings/data-centres-... "Global data centre electricity demand in 2018 was an estimated 198 TWh, or almost 1% of global final demand for electricity (Masanet et al., 2018)."
Earlier quoted context omitted.
The point of the code is not to speedup the execution of summing a list of random number, but rather to speedup the acknowledgement of N random python developers that they have some misconceptions about the GIL. I think it does that pretty well but, well, that's just like my opinion.
I'm a casual Python programmer (just bit of scripting) and I had no particular misconception about the GIL; I don't care because I write single-threaded cookie cutter scripts. Your example I think is demonstrating the opposite of what you want to show. Those figures are atrocious.
I'm not even sure what the opposite of that is.