Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The eigenvector of “Why we moved from language X to language Y”

erikbern.com

Post reply on HN
Print view View original

141–150 of 193 posts

Re: The eigenvector of “Why we moved from language X to language Y”

#141

Post by pmontra »

eeZah7Ux wrote: post #65
The_suffocated wrote: post #29

The research methodology in this blog post is fundamentally flawed. The author only counts how many people move from X to Y, but he doesn't count how many of them do not move at all. The whole diagonal of his (sample) transition matrix are actually missing values, but he treats them as zeroes. This greatly distorts the equilibrium distribution. As a result, he misinterprets each equilibrium probability as the "future…

Also, the likelihood of people blogging about a language change is not the same across languages. Some crowds (Go and nodejs enthusiasts) are notoriously vocal due to the hype. Finally, it's common practice for companies to have their marketing department to pay "media marketing specialist" to advertise for products (their language) by posting on forums.

Some languages and frameworks are simply too new to have a significant number of people moving away from them. Go and Vue fit that case. MySQL looks genuine, even if surprising.

Re: The eigenvector of “Why we moved from language X to language Y”

#142

Post by jrocketfingers »

PDoyle wrote: post #64

> Surprisingly, (to me, at least) Go is the big winner here. There’s a ton of search results for people moving from X to Go. You mean ... Google search results? I'm not trying to suggest that Google's search engine is intentionally biased toward Google projects, but I think it's reasonable to assume that their own projects wouldn't fall into whatever unintentional blind spots their search engine may have.

They didn't exactly make it search engine friendly either, given that the name is quite generic/short.

Re: The eigenvector of “Why we moved from language X to language Y”

#143

Post by unpythonic »

erikbern wrote: post #129
fishnchips wrote: post #100

Am I reading this incorrectly, or there is more movement from Swift to Objective-C than the other way around? Do I sense a methodological error?

Author here. Yes you are reading this incorrectly. Look at the contingency table. objective c to swift: 5216 swift to objective c: 1639 (sorry about the small font size though. had to squint really hard)

The contingency table shows these very clearly, but I believe the question is about the future popularity table.

The future probability table shows that Swift is to the left of (smaller future probability) of Objective C. The coloring of the chart shows a much darker square for Swift -> Objective C than for Objective C -> Swift.

It seems surprising given your contingency table that your analysis would show that Objective C is going to be the more popular of the two.

Re: The eigenvector of “Why we moved from language X to language Y”

#144

Post by omaranto »

erikbern wrote: post #125
The_suffocated wrote: post #29

The research methodology in this blog post is fundamentally flawed. The author only counts how many people move from X to Y, but he doesn't count how many of them do not move at all. The whole diagonal of his (sample) transition matrix are actually missing values, but he treats them as zeroes. This greatly distorts the equilibrium distribution. As a result, he misinterprets each equilibrium probability as the "future…

Author here. You are absolutely right. As I mentioned in the notes, I think this matters a bit less than it might seem like (the stationary distribution does not change if you add a diagonal matrix) but clearly some languages will have a higher propensity for people to stay. I think this flaw is even smaller than the issue of using Google statistics to infer transition probabilities. It's just a shitty proxy, at best…

That bit about the stationary distribution not changing if you add a diagonal matrix sounds completely wrong to me. Let me see if I understand what you mean. Given a matrix M with non-negative entries (and no row of just zeros), let S(M) denote the stochastic matrix you get by normalizing each row of M. You are saying that if M is any matrix and D is a diagonal matrix with non-negative entries then S(M) and S(M+D) have the same stationary distribution?

Re: The eigenvector of “Why we moved from language X to language Y”

#146

Post by ArneBab »

This looks pretty interesting. Most striking to me is that Go is taking from other 'target' languages. You can see the 5x5 block of the other strongest target languages giving to Go, but not taking from it. To make what the Eigenvector says explicit:

Top 5 giving to Go directly: C, Python, Java, Ruby, Scala

Top 5 giving to C: C#, R, Java', C++, Fortran

Top 5 giving to Python: C', Perl, Java', C#, C++

Top 5 giving to Java: C', C++, PHP, Python', C#

Top 5 giving to Ruby: Python', PHP, Perl, Java', (C++ — only 215)

Top 5 giving to Scala: Java', Ruby, (Python', C#, PHP — only 100, 17, 16)

': language also in top 5 givers to Go.

The other top languages take from each other (there is migration in both directions), but currently Go mostly takes here. However it does lose people to Rust — which is actually the strongest go-to language from Go. And C++ does not give to go.

This might point to a discrepancy between Go marketing and reality (efficiency and replacing C++).

It would be great if you could repeat this exercise next year to see how things changed.

(besides: the script is nice and concise!)

Re: The eigenvector of “Why we moved from language X to language Y”

#147

Post by schoen »

jeremyjh wrote: post #58

I don't think the Google queries measure what the author thinks it does. I noticed it lists 14 results from Haskell to Erlang, which I was skeptical of. When I google "move from Haskell to Erlang" or "switch from Haskell to Erlang" I do find results (such as quora questions, versus questions,lecture notes) but none of those results are the type of article we're looking for. If they really want to do this, I think the…

I'm also concerned that for some blogs, a single post might appear several times in Google's estimated search results (due to crawling the same blog under different hostnames or the same post under different paths, or because of syndication or posts to link aggregators). So maybe some individual posts reflecting particular teams' decisions are reflected 5, 10, or 100 times in the Google result count.

Re: The eigenvector of “Why we moved from language X to language Y”

#148

Post by ArneBab »

unpythonic wrote: post #143
erikbern wrote: post #129

Earlier quoted context omitted.

Author here. Yes you are reading this incorrectly. Look at the contingency table. objective c to swift: 5216 swift to objective c: 1639 (sorry about the small font size though. had to squint really hard)

The contingency table shows these very clearly, but I believe the question is about the future popularity table. The future probability table shows that Swift is to the left of (smaller future probability) of Objective C. The coloring of the chart shows a much darker square for Swift -> Objective C than for Objective C -> Swift. It seems surprising given your contingency table that your analysis would show that Objec…

if I understand it correctly that means that most people who move away from swift move to Objective C, but people moving away from Objective C also move to C# and Java.

Re: The eigenvector of “Why we moved from language X to language Y”

#149

Post by pavanky »

ArneBab wrote: post #146

This looks pretty interesting. Most striking to me is that Go is taking from other 'target' languages. You can see the 5x5 block of the other strongest target languages giving to Go, but not taking from it. To make what the Eigenvector says explicit: Top 5 giving to Go directly: C, Python, Java, Ruby, Scala Top 5 giving to C: C#, R, Java', C++, Fortran Top 5 giving to Python: C', Perl, Java', C#, C++ Top 5 giving to…

Or it could be that Go is a fairly new language and has so few users (relative to other top languages), the search queries will show up in only one direction. You should note down what is happening in the other "new" languages.

Re: The eigenvector of “Why we moved from language X to language Y”

#150

Post by jabot »

erikbern wrote: post #125
The_suffocated wrote: post #29

The research methodology in this blog post is fundamentally flawed. The author only counts how many people move from X to Y, but he doesn't count how many of them do not move at all. The whole diagonal of his (sample) transition matrix are actually missing values, but he treats them as zeroes. This greatly distorts the equilibrium distribution. As a result, he misinterprets each equilibrium probability as the "future…

Author here. You are absolutely right. As I mentioned in the notes, I think this matters a bit less than it might seem like (the stationary distribution does not change if you add a diagonal matrix) but clearly some languages will have a higher propensity for people to stay. I think this flaw is even smaller than the issue of using Google statistics to infer transition probabilities. It's just a shitty proxy, at best…

Another thing which seems at least as important:

How many projects are _started_ in a language, and how many _die_?

Given p.e. Java - It may seem that there is a huge flow to go. However, if there are enough new projects started in java, then the number of java projects might still rise faster than the number of go projects.

Post reply on HN