Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The eigenvector of “Why we moved from language X to language Y”

erikbern.com

Post reply on HN
Print view View original

131–140 of 193 posts

Re: The eigenvector of “Why we moved from language X to language Y”

#131

Post by mhermher »

sirbranedamuj wrote: post #82
hasenj wrote: post #81

Earlier quoted context omitted.

This is similar to people measuring popularity of a library or language by stackoverflow question counts. Sure, you can't get a lot of questions for an obscure library that on one uses, but the mere fact that tons of people post questions about it doesn't mean it's good or popular. It might mean lots of beginners want to use it because it's been marketed and hyped as easy to use. (Angular comes to mind).

It might also mean that it's confusing and poorly designed, leading many, many of its users to ask questions about it.

Or that the official documentation is poor.

Re: The eigenvector of “Why we moved from language X to language Y”

#132

Post by sidlls »

lmm wrote: post #59
peterwwillis wrote: post #20

Earlier quoted context omitted.

C is a tadpole in the ocean of easily preventable bugs.

Then why do we see a major internet security bug that would be simply impossible in any other language every couple of months?

You need to compare bugs on a per-usage basis (e.g. lines of code, programs written, some other similar metric), not a raw total. You're comparing numerators, not ratios.

Re: The eigenvector of “Why we moved from language X to language Y”

#133

Post by j2kun »

Certhas wrote: post #106
omginternets wrote: post #101

Earlier quoted context omitted.

I wouldn't call that a "flaw" so much as an "entirely different question".

No it's definitely a flaw. The analysis doesn't model what he thinks it models. He claims that it doesn't matter by (correctly) pointing out that adding a diagonal doesn't change anything, and that hides the modelling assumption he makes: That there is the same absolute number of people choosing to stay within every language studied. His analysis would only be correct if the number of people who choose to stay with J…

Doesn't it make sense if you ask: Given that someone will change the language they're using (and writes a public blog post about it), which are they most likely to change to?

It's not unreasonable to use that as a proxy for industry trends. I recall reading about manufacturing jobs which, sure, might have lots of factories not changing, but the ones that do change _definitely_ opt for more automation with fewer workers. That's still a trend worth thinking about.

Re: The eigenvector of “Why we moved from language X to language Y”

#134

Post by patmcguire »

eeZah7Ux wrote: post #65
The_suffocated wrote: post #29

The research methodology in this blog post is fundamentally flawed. The author only counts how many people move from X to Y, but he doesn't count how many of them do not move at all. The whole diagonal of his (sample) transition matrix are actually missing values, but he treats them as zeroes. This greatly distorts the equilibrium distribution. As a result, he misinterprets each equilibrium probability as the "future…

Also, the likelihood of people blogging about a language change is not the same across languages. Some crowds (Go and nodejs enthusiasts) are notoriously vocal due to the hype. Finally, it's common practice for companies to have their marketing department to pay "media marketing specialist" to advertise for products (their language) by posting on forums.

Yeah, many languages will be almost entirely silent in transition. COBOL, FORTRAN, any proprietary language. And I would guess those transitions are usually to the "boring" languages: Java, C#, etc.

Re: The eigenvector of “Why we moved from language X to language Y”

#135

Post by j2kun »

erikbern wrote: post #125
The_suffocated wrote: post #29

The research methodology in this blog post is fundamentally flawed. The author only counts how many people move from X to Y, but he doesn't count how many of them do not move at all. The whole diagonal of his (sample) transition matrix are actually missing values, but he treats them as zeroes. This greatly distorts the equilibrium distribution. As a result, he misinterprets each equilibrium probability as the "future…

Author here. You are absolutely right. As I mentioned in the notes, I think this matters a bit less than it might seem like (the stationary distribution does not change if you add a diagonal matrix) but clearly some languages will have a higher propensity for people to stay. I think this flaw is even smaller than the issue of using Google statistics to infer transition probabilities. It's just a shitty proxy, at best…

The thing I like most about this post is that it's falsifiable. We will know in ten years whether C and Java are still popular, and whether Go succeeds in the sense this data suggests. So thank you for being concrete and clear, even if it's all in fun and other people don't like it :)

Re: The eigenvector of “Why we moved from language X to language Y”

#136

Post by erikbern »

j2kun wrote: post #135
erikbern wrote: post #125

Earlier quoted context omitted.

Author here. You are absolutely right. As I mentioned in the notes, I think this matters a bit less than it might seem like (the stationary distribution does not change if you add a diagonal matrix) but clearly some languages will have a higher propensity for people to stay. I think this flaw is even smaller than the issue of using Google statistics to infer transition probabilities. It's just a shitty proxy, at best…

The thing I like most about this post is that it's falsifiable. We will know in ten years whether C and Java are still popular, and whether Go succeeds in the sense this data suggests. So thank you for being concrete and clear, even if it's all in fun and other people don't like it :)

This is a great point. It's a model that predicts something about the future. I could have backtested it on historical Google stats to figure out if it's a good model :)

Re: The eigenvector of “Why we moved from language X to language Y”

#137

Post by j2kun »

howeman wrote: post #119
mathgenius wrote: post #37

Earlier quoted context omitted.

Yes, but... The matrix has all non-negative entries, and the author is after the highest eigenvalue/vector so I think this means stability is just not an issue. The only possible issue is time to convergence. The nice thing about the power method is its conceptual simplicity. In cases like this, it's quite hard to screw it up. And it will scale far beyond those numpy functions (not that this is needed for this exampl…

Why do you say it will scale far beyond? Mat mul is N^3 as is eigenvalue solving. It's actually the second highest eigenvalue. The highest eigenvalue is always 1 for stochastic matrices.

Power method is not matrix-matrix multiplication (which is not N^3, BTW [1]), but rather matrix-vector multiplication. So the power method is N^2*k where k is the number of iterations required to reach precision (usually polylogarithmic).

All this being said, scalability is _obviously_ a non-issue when talking about a matrix of programming languages. All methods are constant time.

[1]: https://en.wikipedia.org/wiki/Matrix_multiplication_algorith... Interesting tidbit: nobody can even prove it's not N^2 :)

Re: The eigenvector of “Why we moved from language X to language Y”

#138

Post by striking »

rocqua wrote: post #54
pjmlp wrote: post #11

Earlier quoted context omitted.

Sadly getting rid of C means getting rid of UNIX, as they are symbiotic and UNIX vendors will surely never rewrite them in anything else or replace POSiX standard.

C and say, rust or c++ can interface. You don't need to rewrite, just stop writing extra stuff in C, maybe when you do a really big refactor in C, port it. In the end, we can migrate away from C gradually. That makes me wonder, is there any chance in hell we get some RUST in the Linux source code?

No way will Torvalds allow modifying the build process to include Rust.

Write your own out-of-tree Rust modules if you like, but they'll never be merged in.

Re: The eigenvector of “Why we moved from language X to language Y”

#139

Post by j2kun »

erikbern wrote: post #136
j2kun wrote: post #135

Earlier quoted context omitted.

The thing I like most about this post is that it's falsifiable. We will know in ten years whether C and Java are still popular, and whether Go succeeds in the sense this data suggests. So thank you for being concrete and clear, even if it's all in fun and other people don't like it :)

This is a great point. It's a model that predicts something about the future. I could have backtested it on historical Google stats to figure out if it's a good model :)

In an unrelated note: interested in doing a guest post over at Math ∩ Programming? I see you've got lots of cool stuff with high-dimensional NN :)

Re: The eigenvector of “Why we moved from language X to language Y”

#140

Post by pmarreck »

erikbern wrote: post #125
The_suffocated wrote: post #29

The research methodology in this blog post is fundamentally flawed. The author only counts how many people move from X to Y, but he doesn't count how many of them do not move at all. The whole diagonal of his (sample) transition matrix are actually missing values, but he treats them as zeroes. This greatly distorts the equilibrium distribution. As a result, he misinterprets each equilibrium probability as the "future…

Author here. You are absolutely right. As I mentioned in the notes, I think this matters a bit less than it might seem like (the stationary distribution does not change if you add a diagonal matrix) but clearly some languages will have a higher propensity for people to stay. I think this flaw is even smaller than the issue of using Google statistics to infer transition probabilities. It's just a shitty proxy, at best…

And I share my Go reservations with you, given the whole error-handling (or lack thereof) philosophy as well as information emerging that it may require 100 lines of Go to do roughly the same amount of work as 20 lines of Elixir or Haskell, according to one example at https://medium.com/unbabel-dev/a-tale-of-three-kings-e0be17a...

(Although I concluded the Haskell-Elixir equivalency myself based on functional semantics)

Post reply on HN