Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Djbsort: A new software library for sorting arrays of integers

sorting.cr.yp.to

Post reply on HN
Print view View original

61–70 of 158 posts

Re: Djbsort: A new software library for sorting arrays of integers

#61

Post by thenrich99 »

tazjin wrote: post #26
throwaway77384 wrote: post #10

Earlier quoted context omitted.

Care to elaborate? I am also looking into it, so would be interested to hear what the issues are.

There's an indexed list of posts about Go's flaws here: https://github.com/ksimka/go-is-not-good In my opinion, don't use Go at all if you can avoid it - it may be acceptable for a tiny CLI project but anything of significant complexity needs a language that can scale.

I'd hardly call Kubernetes, Docker daemon and tooling, etcd, CockroachDB, geth, and nsq tiny CLI projects.

If anyone has any reservations about learning Go, don't judge the language based on a list of flaws written by some programmers who used it for a few months, became frustrated, and wrote a blog post.

Go has tradeoffs just like any other language and plenty of programmers leverage it for its positives: https://github.com/avelino/awesome-go

Re: Djbsort: A new software library for sorting arrays of integers

#62

Post by jedisct1 »

dilap wrote: post #59
nrclark wrote: post #41

My job involves a lot of packaging/cross-compilation, and djb's libraries always seem consistently hostile to the lowly packaging engineer. Would it really be all that much work to package in autotools or CMake? Why do I need his special-snowflake build system with its hard-coded assumptions about system paths? I know that the cult of djb will downvote this into oblivion, but seriously, what is the rationale for a bu…

as a user, i'd be pretty fucking happy if ./configure disappeared off the face of the earth.

As a user, I'm super happy with `./configure`, that works the same way across all applications using it, and has been for the past 20 years.

You don't want to look at the actual script, but as a user, it is very convenient. I don't have to read installation guides to know how to change compilation options, compilers, cross-compile, installation paths, run the test suite, etc.

Re: Djbsort: A new software library for sorting arrays of integers

#63

Post by kbenson »

geocar wrote: post #53
nrclark wrote: post #48

Earlier quoted context omitted.

FWIW, I read through each of your links. They don't address cross-compilation at all, or the needs of software packagers. I can 100% promise you that somebody packaging this library for any Linux distro (or for a Yocto/Buildroot system) would grind their teeth in frustration at everything in the those links. The solution to having inconsistent packaging paths isn't to introduce _yet another_ packaging path system, bu…

> or the needs of software packagers. They address the needs of users. I don't care about software packagers at all, except insofar as they can address the needs of users. > The solution to having inconsistent packaging paths isn't to introduce _yet another_ packaging path system, but this one specific to djb stuff. It's to use a standard build system with overrideable paths, and not to assume the author knows better…

You example script with python2 is not about pathing, but a separate problem or multiple binaries with the same name, and is exactly why you want to let package managers deal with this.

As a developer, you can't know all the different setups you users will have, or for example what the default python will be. As a package manager, you know Exactly this for the systems you are packaging for. If needed, they will patch your library to fix the python binary name, or supply the correct build arg if you support it. It's their job to make sure all the users in their little slice of the Linux ecosystem can install and use your software reliably and easily, so working against them is user hostile, and using example of exactly the problems they solve as evidence for why we shouldn't make their lives easier makes no sense.

Re: Djbsort: A new software library for sorting arrays of integers

#64

Post by herf »

Ono-Sendai wrote: post #7

The median sort time for sorting 1048576 elements wth djbsort is 61467822 cycles: https://sorting.cr.yp.to/speed.html On a say 3.6 Ghz processor that would be around 17ms. So the number of elements sorted per second would be around 61.4 M elements/s. My parallel radix sort can sort floats (a little harder than integers) at around 165 M elements/s: http://forwardscattering.org/post/34 A serial radix sort should still…

My single-threaded floating point sort is also twice as fast as djbsort:

http://stereopsis.com/radix.html

This one uses 11-bit radix to save memory bandwidth. Without all the floating-point stuff it would be faster.

Re: Djbsort: A new software library for sorting arrays of integers

#65

Post by bluetech »

I liked this bit, using the fastest compiler for each primitive:

> ./do tries a list of compilers in compilers/c, keeping the fastest working implementation of each primitive. Before running ./do you can edit compilers/c to adjust compiler options or to try additional compilers.

Re: Djbsort: A new software library for sorting arrays of integers

#66

Post by mwcampbell »

nrclark wrote: post #41

My job involves a lot of packaging/cross-compilation, and djb's libraries always seem consistently hostile to the lowly packaging engineer. Would it really be all that much work to package in autotools or CMake? Why do I need his special-snowflake build system with its hard-coded assumptions about system paths? I know that the cult of djb will downvote this into oblivion, but seriously, what is the rationale for a bu…

There's also a world beyond Unix. Prof. Bernstein may not be interested in targeting Windows. But for library developers in general, I'd recommend CMake. Microsoft has adopted CMake as its standard cross-platform C/C++ build system, and has built a cross-platform library package manager [1] on top of it.

[1]: https://github.com/Microsoft/vcpkg

Re: Djbsort: A new software library for sorting arrays of integers

#67

Post by dilap »

jedisct1 wrote: post #62
dilap wrote: post #59

Earlier quoted context omitted.

as a user, i'd be pretty fucking happy if ./configure disappeared off the face of the earth.

As a user, I'm super happy with `./configure`, that works the same way across all applications using it, and has been for the past 20 years. You don't want to look at the actual script, but as a user , it is very convenient. I don't have to read installation guides to know how to change compilation options, compilers, cross-compile, installation paths, run the test suite, etc.

(i guess i should say i'm a dev as well as a user. but my exposure to configure is as a user.)

so here's why i'm not a fan of configure: (1) it's slow (2) if something goes wrong, it's a fucking nightmare to try to diagnose -- it's just an endless barage of crap that doesn't actually have anything to do with the app at hand, and the actual build process is like 2 or 3 layers away from running the configure script. (or for extra fun, maybe you didn't have a configure script, you had the other script that generates the configure script.)

i'd much rather see just a simple makefile with well documented dependencies, e.g., like redis does it.

(slowly this problem is going away as building becomes an integrated part of the language -- e.g., rust, go, ...)

(and to be fair, a large part of it is just an aesthetic objection -- the sheer mass of unnecessary code and checks that configure does & generates is really ugly to me. i want beautiful software!)

Re: Djbsort: A new software library for sorting arrays of integers

#68

Post by ur-whale »

nrclark wrote: post #41

My job involves a lot of packaging/cross-compilation, and djb's libraries always seem consistently hostile to the lowly packaging engineer. Would it really be all that much work to package in autotools or CMake? Why do I need his special-snowflake build system with its hard-coded assumptions about system paths? I know that the cult of djb will downvote this into oblivion, but seriously, what is the rationale for a bu…

1. You're assuming the software is intended to be deployed

2. Writing anything sane with autotools or CMakes is an exercise in frustration that borders on torture. Not to mention how either of these systems chafes on one's sense of aesthetics.

3. I've installed a few of djb's package over the years. While indeed non standard, unlike either of the solutions you've mentioned, djb's stuff: a) works b) is usually very simple to understand and change when needed (unlike the autotools steaming monstrosity which in 20 years of using opensource software I've never dared to touch)

Re: Djbsort: A new software library for sorting arrays of integers

#69

Post by JdeBP »

geocar wrote: post #46
nrclark wrote: post #41

My job involves a lot of packaging/cross-compilation, and djb's libraries always seem consistently hostile to the lowly packaging engineer. Would it really be all that much work to package in autotools or CMake? Why do I need his special-snowflake build system with its hard-coded assumptions about system paths? I know that the cult of djb will downvote this into oblivion, but seriously, what is the rationale for a bu…

> Why do I need ... assumptions about system paths? > what is the rationale for a build flow that involves: It solves problems. https://cr.yp.to/compatibility.html https://cr.yp.to/slashpackage/studies.html https://cr.yp.to/slashpackage/finding.html https://cr.yp.to/slashpackage/sharability.html > Would it really be that much harder to give us a git repo and a ./configure or a CMakeLists.txt? Yes.

Bear in mind that steps 1, 3, 4, 6, and 7 are not related to slashpackage. nrclark has got step #2 wrong, as all that it yields is the current source package version number not a URL and there's no real parsing going on (except for the shell stripping the trailing newline); and xyr step #8 is xyr own invention and not actually in the build instructions as given.

That leaves step #5. What that does is both fairly obvious and documented in the instructions. It tries out a whole bunch of compilers, compiler options, and implementations in turn to see which results in the best code. Ironically, it is probably the sort of thing that autotools might be persuaded to do. But in practice no-one would, or it would be buried under reams of copied-and-pasted cargo cult tests for things that the actual source at hand cannot in fact do without in the first place.

This is of course a long-recognized fly in any let's-just-use-autotools ointment, discussed over many years. (-:

* https://news.ycombinator.com/item?id=5276876

* https://queue.acm.org/detail.cfm?id=2349257

* http://freshmeat.sourceforge.net/articles/stop-the-autoconf-...

* https://news.ycombinator.com/item?id=1499738

* https://news.ycombinator.com/item?id=4407486

Re: Djbsort: A new software library for sorting arrays of integers

#70

Post by jeffreyrogers »

puzzle wrote: post #58
yoklov wrote: post #13

The problem with AVX2 accelerated code (and much of AVX) is that unless you have a lot of it to run you end up with a substantial speed hit that comes from the cost of switching to a different power bin (which often takes 1 or 2ms!) and then running at a lower clock speed. This often still ends up being an improvement over scalar code (at the cost of higher power usage), but for occasional workloads that don't need t…

You also have penalties from context switches, although you can reduce their impact by performing them in a lazy fashion. AVX512 is even worse, of course.

You need a context switch to use AVX?
Post reply on HN