Live data from Hacker News

The eigenvector of “Why we moved from language X to language Y”

erikbern.com

31–40 of 193 posts

Re: The eigenvector of “Why we moved from language X to language Y”

#31
post #4

Well the fact that C is going so strong guarantees we'll be dealing with easily preventable bugs for the next 100 years

C is a tadpole in the ocean of easily preventable bugs.

And when it grows up, it becomes the toad that is C++?

I'm not sure I get your analogy.

Besides, Real Programmers™ eat mutable state and NULL for breakfast.

Re: The eigenvector of “Why we moved from language X to language Y”

#32
Only relatively small project can afford a rewrite. So, this is statistic among projects that can affod a swith. And as far as I can see, this is eigenvector of trend. First derivative of actual state of things. More informative of the state of fashion today

Re: The eigenvector of “Why we moved from language X to language Y”

#33

Earlier quoted context omitted.

They could be. But you'd need a reason in order to be able to fund it (or maybe you just want to scratch an itch). But the codebase numbers in the 100's of millions of LOC, and replacing that with Rust (or anything else for that matter) will come with a number of requirements: - it really needs to be better in terms of bugs - it needs to be about as fast or faster - it would have to come with similar start-up times f…

The Linux ecosystem was created with the userland already having being rewritten from scratch, by the GNU Project. Then a Finnish student came along and wrote a kernel. So you can probably start anywhere, but maybe starting with a working kernel would be better. Linux doesn't really resemble UNIX anymore. It's starting to look a little like Plan 9, to be honest... give it 20 more years.

> Linux doesn't really resemble UNIX anymore. It's starting to look a little like Plan 9, to be honest... give it 20 more years.

Hm. I'm not sure I see the similarities there. plan9: small, elegant, really an improvement on Unix in many respects but unfortunately somewhat theoretical rather than practically oriented.

Linux: bloated, blunt, practically oriented, gets the job done but it never feels like it's the shortest path.

Re: The eigenvector of “Why we moved from language X to language Y”

#34

The research methodology in this blog post is fundamentally flawed. The author only counts how many people move from X to Y, but he doesn't count how many of them do not move at all. The whole diagonal of his (sample) transition matrix are actually missing values, but he treats them as zeroes. This greatly distorts the equilibrium distribution. As a result, he misinterprets each equilibrium probability as the "future…

I wonder if the data could be normalized against usage statistics somehow. Maybe not perfectly effectively, but at least better than as-is, precisely because the number of people "switching" is so much smaller than the number using a given language.

Re: The eigenvector of “Why we moved from language X to language Y”

#36
post #30

Earlier quoted context omitted.

Really? Why can't the C bits be replaced with e.g. Rust?

No because UNIX requires C semantics, so even if someone writes a UNIX like OS in Rust, Ada whatever language it might be, for compatibility with UNIX software it would require a POSIX API to be available. POSIX is defined in terms of C semantics, which includes C unsafety, like managing pointers and the respective length as separate entities, using null terminated strings or casting void* to specific data structures…

There used to be POSIX standards for both Ada and Fortran as well a C (POSIX.5 and POSIX.12). Though I don't think they can be described as successful.

If I remember right, the Fortran one was defined in terms of the C one, but the Ada one was written as if it was an independent specification.

I suppose this was only possible because POSIX misses out a lot of the fiddlier bits of Unix anyway (and the Ada one specified a spawn to use instead of fork+exec).

Re: The eigenvector of “Why we moved from language X to language Y”

#37
post #13

I appreciate that the author wanted to implement their own eigen vector/value method, but really they should use: numpy.linalg.eig(x)[0] numpy.linalg.eigvals(x)[0] Numerical stability can be hard to get right...

Yes, but... The matrix has all non-negative entries, and the author is after the highest eigenvalue/vector so I think this means stability is just not an issue. The only possible issue is time to convergence.

The nice thing about the power method is its conceptual simplicity. In cases like this, it's quite hard to screw it up. And it will scale far beyond those numpy functions (not that this is needed for this example.)

Also, did anyone mention PageRank yet?

Re: The eigenvector of “Why we moved from language X to language Y”

#38
post #34

The research methodology in this blog post is fundamentally flawed. The author only counts how many people move from X to Y, but he doesn't count how many of them do not move at all. The whole diagonal of his (sample) transition matrix are actually missing values, but he treats them as zeroes. This greatly distorts the equilibrium distribution. As a result, he misinterprets each equilibrium probability as the "future…

I wonder if the data could be normalized against usage statistics somehow. Maybe not perfectly effectively, but at least better than as-is, precisely because the number of people "switching" is so much smaller than the number using a given language.

Presumably also on a time series. People used to switch from Ruby to Node.js, now they switch to ???

Re: The eigenvector of “Why we moved from language X to language Y”

#39
post #34

The research methodology in this blog post is fundamentally flawed. The author only counts how many people move from X to Y, but he doesn't count how many of them do not move at all. The whole diagonal of his (sample) transition matrix are actually missing values, but he treats them as zeroes. This greatly distorts the equilibrium distribution. As a result, he misinterprets each equilibrium probability as the "future…

I wonder if the data could be normalized against usage statistics somehow. Maybe not perfectly effectively, but at least better than as-is, precisely because the number of people "switching" is so much smaller than the number using a given language.

If you mean to account for the larger variances of rarer events, that would be difficult. Perhaps some bootstrap sampling methods may help, but we need a true statistician here.

Re: The eigenvector of “Why we moved from language X to language Y”

#40
post #34

Earlier quoted context omitted.

I wonder if the data could be normalized against usage statistics somehow. Maybe not perfectly effectively, but at least better than as-is, precisely because the number of people "switching" is so much smaller than the number using a given language.

Presumably also on a time series. People used to switch from Ruby to Node.js, now they switch to ???

Agree. Depending on what one wants to measure, giving every sample equal weight can be potentially wrong.
Post reply on HN