Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The computers are fast, but you don't know it

shvbsle.in

Post reply on HN
Print view View original

781–790 of 819 posts

Re: The computers are fast, but you don't know it

#781

Post by lumost »

closeparen wrote: post #390
judge2020 wrote: post #376

Earlier quoted context omitted.

I believe that's called a "monolith" and it's quite popular, especially in pretty much every traditional enterprising company.

Traditional enterprise companies are agglomerations of IT systems from a sprawling network of acquisitions, subsidiaries, and partners. They collect fad languages, architectures, and proprietary ecosystems from across decades of computing history. And then try to somehow make them all play with each other. At least in our world we have the source code to all the services. They have explicit and intentional APIs. They…

This is a big reason microservices are popular in enterprise. Performance rarely matters via minimizing development costs.

Re: The computers are fast, but you don't know it

#782

Post by goosedragons »

von_lohengramm wrote: post #603
vishnugupta wrote: post #498

Earlier quoted context omitted.

For most of my work CPUs form the last decade will work just fine. It’s the memory and, especially, disk IO that kills the performance. SSDs have helped big time.

I'd argue that SSDs have done more harm than good. Since the worst-case is now far superior that it used to be (HDDs), most developers see no need to optimize any further. For example, plenty of video game engines will stream copious amounts of data from disk instead of optimizing memory usage, asset size, and in general more creative solutions (i.e. shader effects instead of GBs of redundant assets). If hitting the…

Has that even happened yet? PS5 and Xbox Series are the first consoles to use SSDs instead of HDDs and they're only beginning to gain serious steam. PS4 games are still being released and they have to cater to that stock 5400 RPM HDD.

Re: The computers are fast, but you don't know it

#783

Post by booleandilemma »

generalizations wrote: post #108
forinti wrote: post #22

On a 3GHz CPU, one clock cycle is enough time for light to travel only 10cm. If you hold up a sign with, say, a multiplication, a CPU will produce the result before light reaches a person a few metres away.

I ran across an animation once that showed graphically the time it takes light to travel between the planets and the sun. It's weird, but light doesn't seem that fast anymore.

I saw that exact animation you're talking about and my first thought was "there's no way that's the fastest thing in the universe."

Re: The computers are fast, but you don't know it

#784

Post by igouy »

morelisp wrote: post #763
igouy wrote: post #749

Earlier quoted context omitted.

And then the Java etc programs are re-written to do the same and the test value is increased (to compensate for the reduced memory allocation) and we're back where we started? Seems like moving the furniture around. No doubt I've misunderstood.

A null value in Java has no methods / is not an object. I didn’t precompute anything.

afaict you're making a distinction without a difference.

afaict the Java etc programs could be re-written using null where you used nil — please explain why you think that isn't correct.

Re: The computers are fast, but you don't know it

#785

Post by infra_ »

tpoacher wrote: post #609

The point about pandas resonates with me. Don't get me wrong, pandas is a nice library ... but the odd thing is, numpy already has, like, 99% of that functionality built in in the form of structured arrays and records, is super-optimised under the hood, and it's just that nobody uses it or knows anything about it. Most people will have never heard of it. To me pandas seems to be the sort of library that because popul…

Pandas does a lot, and often times most of it isn’t needed. Basic functionality like Map, Reduce, GroupBy, InnerJoin, LeftJoin, CrossJoin, row or column generators, and transformations between columnar and row based data structures, are often needed but come with a heavy weight library that is not performant when it counts.

Because I needed these operations, I wanted to work with Numpy directly, and didn’t want to write custom implementations each time, I created a library to do it. It also has constructor methods for Python Dicts, any kind of Iterable, CSV, SQL query, pandas DataFrames and Series, or otherwise. As well as destructor methods to generate whatever you need when done. It tries its best to maintain the types you specify, and offers a means to cast as easily as possible. All functions return a single type to allow static type checking. And for performance, there is a “trust me I know what I’m doing” mode for extremely fast access to the data which achieves about a 10x speed up by skipping all data validation steps.

Everything it does outperforms pandas, except for the Joins. It does allow inequality joins and multiple join conditions, but the general solution used isn’t very fast. Anyone reading this who would be interested in improving these component would be welcome to contribute!

https://tafra.readthedocs.io/en/latest/

Re: The computers are fast, but you don't know it

#786

Post by colejohnson66 »

meshaneian wrote: post #706
winternett wrote: post #698

Earlier quoted context omitted.

I was working at my UNI library around 97 when a fresh t3 (~45MBPS) line was just installed... We also got brand new top of the line Micron computers as well there. I was the first person to test the connection and after years of working on 56k modems I couldn't believe how everything I clicked suddenly worked at the speed of light. Videos I clicked on (on MTVs web site back then) loaded instantly, almost felt as if…

Just like freeways - add more traffic lanes, get more traffic.

Theres actually a name for that phenomenon: Braess’ Paradox https://en.wikipedia.org/wiki/Braess%27s_paradox

Re: The computers are fast, but you don't know it

#787

Post by MyNameIsFred »

blub wrote: post #480
wildrhythms wrote: post #80

Earlier quoted context omitted.

I agree... for a business "fast" means shipping a feature quickly. I have personally seen the convos from upper management where they handwave away or even justify making the application slower or unusable for certain users (usually people in developing countries with crappy devices). Oh it will cost +500KB per page load, but we can ship it in 2 weeks? Sounds good!

Lots of businesses have nearly zero engineering in them and cobble together libraries and frameworks that they sell or rent as software. On the other end of the spectrum you have companies hiring specialists at all points of the stack to squeeze out the last drops of performance, dedicated perf teams, etc. The latter also typically produce the tools that enable the former to function.

You are completely correct and I really wish you weren't.

Re: The computers are fast, but you don't know it

#788

Post by morelisp »

igouy wrote: post #784
morelisp wrote: post #763

Earlier quoted context omitted.

A null value in Java has no methods / is not an object. I didn’t precompute anything.

afaict you're making a distinction without a difference. afaict the Java etc programs could be re-written using null where you used nil — please explain why you think that isn't correct.

You cannot call a method on null. Meaning:

- Practically, Java would rather have to `return 3` when it detects a null child, effectively precomputing the penultimate level.

- Semantically, Java could no longer distinguish between an absent child and a child with no children.

Honestly, I have other variants that still don't use pooling but are less idiomatic; I find this exercise is begging the question hard. Any tools, however idiomatic, the language is giving you to reduce the effects of allocation seem to be off-limits for GCd languages. Whereas then e.g. C can just throw them all in a third-party pool library. And JIT languages are presumably allowed to fuse anything they want.

Re: The computers are fast, but you don't know it

#789

Post by hinkley »

necovek wrote: post #772
hinkley wrote: post #767

Earlier quoted context omitted.

How on earth are you getting 5 seconds for simple tests? Simple tests should be running in 8ms, and those are my 2015 numbers that I've been too lazy to update.

Have you worked on a recent idiomatic development setup (dockerised local development, top level imports of everything and plenty of setup at the top level too, people unfamiliar with how to manage .pyc files so they simply disable them...)? Common libraries like requests or sqlalchemy take 300-500ms to import (eg. try `time python3 -c 'import requests'` and contrast just `time python3 -c ''` which is python startup…

Ah, I see, so the setup time is very slow. I don't work in python much but I've worked in a few other languages with slow startup, and amortization is your friend. It's hard though when you have a small module with 'only' 300 tests and your test is 6ms of code that works out to 40ms once setup and teardown are included. I haven't had many opportunities to have the "well maybe you should be making bigger modules" conversation but I am ready for that moment to arise.

This is usually the point at which I pull out a 'watch' implementation, since the 5 seconds it's going to take me to switch windows and hit 'up' the right number of times counts too, if we're comparing apples to apples.

That said, one of the last times I had a unit testing mentor, I walked into a project that ran 3800 tests in about 7 seconds, and then started poking around trying to figure out who was materially responsible. (He didn't know much more than me from an implementation standpoint, but boy was he good at selling people on test quality.) If that had been 20 seconds it would have still been lovely, but it wouldn't have grabbed my attention quite as much.

Re: The computers are fast, but you don't know it

#790

Post by igouy »

morelisp wrote: post #788
igouy wrote: post #784

Earlier quoted context omitted.

afaict you're making a distinction without a difference. afaict the Java etc programs could be re-written using null where you used nil — please explain why you think that isn't correct.

You cannot call a method on null. Meaning: - Practically, Java would rather have to `return 3` when it detects a null child, effectively precomputing the penultimate level. - Semantically, Java could no longer distinguish between an absent child and a child with no children. Honestly, I have other variants that still don't use pooling but are less idiomatic; I find this exercise is begging the question hard. Any tool…

I think you just said the Java etc programs could be re-written in a similar way?

And we could add some more rules ("distinguish between an absent child and a child with no children") to reject those Java etc programs.

Post reply on HN