Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Relating natural language aptitude to differences in learning programming

nature.com

Post reply on HN
Print view View original

11–20 of 50 posts

Re: Relating natural language aptitude to differences in learning programming

#11

Post by bonoboTP »

dhosek wrote: post #7
bonoboTP wrote: post #4

The scatter plots seem quite spread out. If you removed one outlier person from the numeracy plot it would be much more correlated. Maybe that person is a special case somehow. I wonder if there is any formalized way in statistical hypothesis testing to quantify how much the conclusions could be changed by removing at most k (e.g. k=1) number of datapoints.

That's actually pretty typical for behavioral science scatter plots. Getting p>.95 on a regression line is not something that is obvious from looking at the raw data.

I think eyeballing a scatterplot is often more useful than just looking at the p-value.

Re: Relating natural language aptitude to differences in learning programming

#13

Post by dhosek »

bonoboTP wrote: post #11
dhosek wrote: post #7

Earlier quoted context omitted.

That's actually pretty typical for behavioral science scatter plots. Getting p>.95 on a regression line is not something that is obvious from looking at the raw data.

I think eyeballing a scatterplot is often more useful than just looking at the p-value.

One of the big takeaways from my stats class is that eyeballing a scatterplot is much less useful than you think it is.

Re: Relating natural language aptitude to differences in learning programming

#14

Post by throwaway468883 »

skybrian wrote: post #12

I wonder if this predicts any better than a generic IQ test? The idea behind IQ is that lots of different cognitive tasks are correlated.

That would throw under the bus a lot of race-and-IQ suppositions seeing how Africans commonly speak 3 or 4 languages, sometimes more, and a large majority of white people and Asians are monolingual.

Re: Relating natural language aptitude to differences in learning programming

#16

Post by gryson »

username90 wrote: post #10

> one participant was excluded because he was an extreme outlier in learning rate (>3 sd away from the mean). Can anyone explain why did they do this? Was it just to prove their hypothesis? I bet this guy had really high numeracy scores as well. Edit: They also botched some of the numbers: > (Fig. 1A: mean learning rate = 1.25, range = 0.81–2.0, sd = 0.24) The 2.0 point is not included in Fig. 1A, the highest learnin…

> Can anyone explain why did they do this? Was it just to prove their hypothesis? I bet this guy had really high numeracy scores as well.

It's likely a predetermined exclusion criterion (although they should explicitly state that). An extreme outlier on the primary measure may suggest that the subject was not doing the task correctly (resulting in an extremely low score) or it may suggest that the subject misreported their lack of programming knowledge (resulting in an extremely high score).

Depending on what you're looking at, it can be standard procedure to exclude extreme outliers, but it should be announced as predetermined and be principled.

Re: Relating natural language aptitude to differences in learning programming

#17

Post by username90 »

throwaway468883 wrote: post #14
skybrian wrote: post #12

I wonder if this predicts any better than a generic IQ test? The idea behind IQ is that lots of different cognitive tasks are correlated.

That would throw under the bus a lot of race-and-IQ suppositions seeing how Africans commonly speak 3 or 4 languages, sometimes more, and a large majority of white people and Asians are monolingual.

A majority of Europeans speak 2 languages, a quarter speaks 3 or more. So it isn't a white thing, it is an American thing. And it isn't because Americans are dumber than Europeans, it is just because learning new languages makes more sense for Europeans than for Americans due to those languages being smaller and foreign languages being physically closer.

Also mastering a single language is harder than learning to get by in multiple languages, as even though a majority of Europeans can speak multiple languages a majority of them are not really all that good even in their mother tongue as we can see from standardized testing. They aren't worse than Americans, but they aren't better either, they get roughly the same results.

Re: Relating natural language aptitude to differences in learning programming

#18

Post by nazgul17 »

username90 wrote: post #10

> one participant was excluded because he was an extreme outlier in learning rate (>3 sd away from the mean). Can anyone explain why did they do this? Was it just to prove their hypothesis? I bet this guy had really high numeracy scores as well. Edit: They also botched some of the numbers: > (Fig. 1A: mean learning rate = 1.25, range = 0.81–2.0, sd = 0.24) The 2.0 point is not included in Fig. 1A, the highest learnin…

Removing outliers is a common practice in data science. A quick search on "why removing outliers" returns a lot of discussion on the topic of when it should be done and when it should not. I encourage you to read some to get familiar with the arguments.

In this specific case, I suppose they could have added some motivation for their decision, but I don't think they did it in bad faith. They only had 36 samples, and a 3sd event occurs once in ~300 cases: this hints to the fact that that data point might have been erroneous, or that there could have been some other factor (e.g. earlier exposure to programming concepts - even disguised as something else).

If you are interested in the study, you can find the data and the script in the linked repository: https://github.com/UWCCDL/ComputerWhisperers The range for the learning rate seems wrong. If you check the data, there is no data point with 2.0: https://github.com/UWCCDL/ComputerWhisperers/blob/master/Com...

Re: Relating natural language aptitude to differences in learning programming

#19

Post by oh_sigh »

throwaway468883 wrote: post #14
skybrian wrote: post #12

I wonder if this predicts any better than a generic IQ test? The idea behind IQ is that lots of different cognitive tasks are correlated.

That would throw under the bus a lot of race-and-IQ suppositions seeing how Africans commonly speak 3 or 4 languages, sometimes more, and a large majority of white people and Asians are monolingual.

Number of language spoken is largely a function of the environment and doesn't have much to do with intelligence. If you take a highly multilingual African and drop them in America, in two generations you will likely have mostly monolingual descendants.

Re: Relating natural language aptitude to differences in learning programming

#20

Post by ltbarcly3 »

username90 wrote: post #10

> one participant was excluded because he was an extreme outlier in learning rate (>3 sd away from the mean). Can anyone explain why did they do this? Was it just to prove their hypothesis? I bet this guy had really high numeracy scores as well. Edit: They also botched some of the numbers: > (Fig. 1A: mean learning rate = 1.25, range = 0.81–2.0, sd = 0.24) The 2.0 point is not included in Fig. 1A, the highest learnin…

> Was it just to prove their hypothesis? Yes

Removing outliers is common, but extremely sketchy. The existence of outliers in the dataset at all is most likely because they are assuming a probability distribution that does not hold. If they are using the wrong distribution, none of their results mean anything.

Post reply on HN