Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The computers are fast, but you don't know it

shvbsle.in

Post reply on HN
Print view View original

771–780 of 819 posts

Re: The computers are fast, but you don't know it

#771

Post by morelisp »

paskozdilar wrote: post #732
morelisp wrote: post #710

Earlier quoted context omitted.

The ability of read/write to communicate via errors as well as the actual data transferred is significant - there's a reason Go's i/o model is io.Reader/io.Writer and not chan []byte. You might as well explain channels in terms of any blocking operation if the bar for "isomorphic" (now backtracked to "intuitively" I guess) is that low.

I don't know what's the bar for "isomorphism", but I know that the word literally means "same shape", so just because some nerd that got killed in a duel over a girl used the same word in a mathematical context doesn't give him dibs over its general use. Unless, of course, system calls can be modeled as operators over sets. In which case, please tell me how.

In normal use of the term, a function's shape is its type signature; at the very least its in and out arity, and usually even more specific.

Re: The computers are fast, but you don't know it

#772

Post by necovek »

hinkley wrote: post #767
necovek wrote: post #558

Earlier quoted context omitted.

In a sense, knowing this can also hurt you. At all my recent jobs, I grow frustrated with how slow running a single unit test is locally on a codebase. We are talking 5+ seconds for even the most trivial of trivial unit tests (say, purely functional arithmetic unit test). And this is even with dynamic languages like Python (you see pytest reporting how your unit test completed in 0.00s, and wall time is 7s). And then…

How on earth are you getting 5 seconds for simple tests? Simple tests should be running in 8ms, and those are my 2015 numbers that I've been too lazy to update.

Have you worked on a recent idiomatic development setup (dockerised local development, top level imports of everything and plenty of setup at the top level too, people unfamiliar with how to manage .pyc files so they simply disable them...)?

Common libraries like requests or sqlalchemy take 300-500ms to import (eg. try `time python3 -c 'import requests'` and contrast just `time python3 -c ''` which is python startup overhead).

As I said, tests run in sub 10ms, but from issuing pytest to completion it's usually 5-15s.

Re: The computers are fast, but you don't know it

#773

Post by jcelerier »

throwaway894345 wrote: post #756
jcelerier wrote: post #735

Earlier quoted context omitted.

we're in 2022, gdb asks me "can i download missing symbols from the internet" when it loads a binary and does it

Glad to hear the 2022 C++ ecosystem is finally catching up on some regards, but how does it know which version of those dependencies to download, and how does it download closed source symbols?

I mean, if it's closed source you're not getting the debug symbols in any case lol, no matter which language.

It uses debuginfod for fetching symbols - here it "just works".

Re: The computers are fast, but you don't know it

#774

Post by pindab0ter »

hinkley wrote: post #766
pindab0ter wrote: post #487

Earlier quoted context omitted.

Bonus points if the 'speed is faster'.

Oh, but we let people say "acceleration is faster". It's like we've reserved 'faster' for a single derivative and banned it for all the others.

I don't think it's about banning words at all. It's about words making sense.

"What's cheaper? The price is." Now that just doesn't make any sense, since a price isn't cheap or expensive, it's high or low. The thing that is priced can be cheap or expensive, but that's not what's being said.

"What's faster? The speed is." Doesn't make sense either. Speed isn't fast, the speedy thing is. However, "What's faster? The acceleration is." is fine, because you can have slow or fast acceleration (I think?).

I'm an ESL speaker, so please do tell me if I'm wrong and how.

Re: The computers are fast, but you don't know it

#775

Post by paskozdilar »

morelisp wrote: post #771
paskozdilar wrote: post #732

Earlier quoted context omitted.

I don't know what's the bar for "isomorphism", but I know that the word literally means "same shape", so just because some nerd that got killed in a duel over a girl used the same word in a mathematical context doesn't give him dibs over its general use. Unless, of course, system calls can be modeled as operators over sets. In which case, please tell me how.

In normal use of the term, a function's shape is its type signature; at the very least its in and out arity, and usually even more specific.

Well, in my case I used it to signify same behavior in terms of process/goroutine communication. I think it's "specific enough" to warrant the use of word "isomorphic".

Re: The computers are fast, but you don't know it

#776

Post by xenadu02 »

von_lohengramm wrote: post #603
vishnugupta wrote: post #498

Earlier quoted context omitted.

For most of my work CPUs form the last decade will work just fine. It’s the memory and, especially, disk IO that kills the performance. SSDs have helped big time.

I'd argue that SSDs have done more harm than good. Since the worst-case is now far superior that it used to be (HDDs), most developers see no need to optimize any further. For example, plenty of video game engines will stream copious amounts of data from disk instead of optimizing memory usage, asset size, and in general more creative solutions (i.e. shader effects instead of GBs of redundant assets). If hitting the…

> I'd argue that SSDs have done more harm than good

I can't imagine the thought process that would lead to a statement like that. SSDs are the single best thing to happen to personal computing in the past 15 years. Absolutely no question and not even arguable.

Re: The computers are fast, but you don't know it

#777

Post by inkblotuniverse »

knorker wrote: post #611
inkblotuniverse wrote: post #593

Earlier quoted context omitted.

You should, however, not pessimize. People make cargo-cult architecture choices that bloat their codebase, make itnless readable, and make it 100x slower.

Maybe. Using actual numbers vetted by actual expenses in an actual company, if you can save 100 CPU cores by spending 3h a year keeping it optimized, then it is NOT worth it. It is cheaper to burn CPU, even if you could spend one day a year making it max out one CPU core instead of 100. It can be better for the business to cargo cult. Not always. But you should remember that the point of the code is to solve a proble…

I wasn't talking about optimization! I was talking about non-pessimization, which includes not prematurely abstracting/generalizing your code.

I've seen people making poor decisions at the outset, and having code philosophies that actively make new code 100x slower without any clear gain. Over-generalization, 100 classes and subclasses, everything is an overriden virtual method, dogmatic TDD (luckily, nobody followed that.)

The dogma was to make things more complicated and illegible, 'because SOLID'.

Re: The computers are fast, but you don't know it

#778

Post by knorker »

loup-vaillant wrote: post #750
knorker wrote: post #670

Earlier quoted context omitted.

I'm actually also talking about the second. > avoid having the CPU do useless work all the time It's not worth an engineer spending 1h a year even investigating this, if it's less than 20 CPU cores doing useless work. The break even for putting someone full time on this is if you can expect them to save about fourty thousand CPU cores. YMMV. Maybe you're a bank who has to have everything under physical control, and y…

In the specific case of batch processing, I hear you. Machine time is extremely cheap compared to engineer time. Then there are interactive programs. With a human potentially waiting on it. Someone's whose time may be just as valuable as the engineer's time (morally that's 1/1, but even financially the difference is rarely more than a single order of magnitude). If you have as few as 100 users, shaving off seconds of…

I'm up there on the mound preaching the same thing, trust me.

Re: The computers are fast, but you don't know it

#779

Post by necovek »

gpm wrote: post #719
necovek wrote: post #582

Earlier quoted context omitted.

That's definitely quite curious: I am sure pure Python could have been heavily optimized to reach 2 minutes as well, though. Random number generation in Python is C-based, so while the pseudo-random generators from Python's random module might be slow, it's not because of Python itself ( https://docs.python.org/3/library/random.html is a different implementation from https://man7.org/linux/man-pages/man3/random.3.htm…

I'm reasonably sure the PRNG being used in the python version came from numpy and was implemented in C (or other native code, not python). The problem was that the necessary control flow and varying parameters around it meant you had to call it once per value from python (and you had to generate a lot of values). And if I recall correctly there was no allocation in the hot loop, with a single large array being initia…

Sure, numpy introduces its own set of restrictions. I was mostly referring to taking a different approach before turning to numpy, but it could very well be true.

In essence, doing what you did is the way to get performance out of Python when nothing else works.

Re: The computers are fast, but you don't know it

#780

Post by tpoacher »

kortex wrote: post #685
tpoacher wrote: post #609

The point about pandas resonates with me. Don't get me wrong, pandas is a nice library ... but the odd thing is, numpy already has, like, 99% of that functionality built in in the form of structured arrays and records, is super-optimised under the hood, and it's just that nobody uses it or knows anything about it. Most people will have never heard of it. To me pandas seems to be the sort of library that because popul…

> numpy already has, like, 99% of that functionality built in in the form of structured arrays and records Respectfully, this is pretty wrong. Pandas does vastly more out of the box than numpy. Off the top of my head: I/O from over a dozen of data formats, joins/merges, sql queries directly to dataframes, sql-like queries on dataframes, index slicing by time, multi-indexes, much more ergonomic grouping/aggregation fu…

Like I said, I'm probably wrong :)
Post reply on HN