81–90 of 366 posts
[0] https://github.com/gunnarmorling/1brc/blob/main/src/main/jav...
Earlier quoted context omitted.
This is done to simulate real-world performance. Your binary is not the only binary in the system and other services may be running as well. So fastest time is the happiest path and slowest is the unluckiest. The range of remaining three is what you expect to get 99% of the time on a real world system.
> Your binary is not the only binary in the system and other services may be running as well. Technically yes, but these days most of my machines are single purpose VMs; database/load balancer/app server/etc, so it still seems weird not to take the fastest.
There's logrotate and other cleanup tasks, monitoring, dns, a firewall, and many more stuff running on that server. No matter how much you offload to the host (or forego), there's always a kernel and supporting deamons running alongside or under your app.
You had 32GB more than expected 16GB RAM but for Java I doubt that 32GB is fine enough. Java is aboslutely good on older days but nowadays there is lot of opportunities than that poor guy.
but… can I use pandas?
BTW asking ChatGPT to utilize all cores did not yield anything working in reasonable time.
> The slowest and the fastest runs are discarded. The mean value of the remaining three runs is the result for that contender and will be added to the leaderboard. I think it's better to discard the two slowest, or simply accept the fastest as the correct. There's (in my opinion) no good reason to discard the best runs.
The rule lawyer in me wants to spend the first run spinning up a background daemon that loads everything into memory, pins it there, and maybe even prefetches everything into cache as the subsequent runs perform basically a linear scan (you never have to pagemiss if you have an oracle!). > write a Java program for retrieving temperature measurement values from a text file and calculating the min, mean, and max temper…
This might become a contest of judging what is fair pre-computing and what is not.
That's why machine learning contests don't let participants see the final data.
As far as I see the currently best performing solution [0] does not account for hash collisions and therefore probably generates wrong results if enough different cities are in the dataset. Or am I missing something? [0] https://github.com/gunnarmorling/1brc/blob/main/src/main/jav...
[0] https://twitter.com/mtopolnik/status/1742652716919251052
Anyone up to also do this in Rust ?