Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

To the brain, reading computer code is not the same as reading language

news.mit.edu

Post reply on HN
Print view View original

151–160 of 232 posts

Re: To the brain, reading computer code is not the same as reading language

#151

Post by tremon »

pgt wrote: post #96
User23 wrote: post #28

As someone who is fluently bilingual and speaks a third language and is also proficient or better in a half dozen programming languages along with having written code in many more, this is completely obvious to me. Learning a programming language is much closer to learning a complex board game than it is a natural language.

Your comment is insightful and suggests that perhaps we should try to design a programming language that feels more like a natural language.

No, not at all. Natural language is imprecise, ambiguous and heavily contextual in both its structure and semantic meaning. All those qualities are sources of bugs or undefined behaviour in programming.

As others have said in this thread: programming "languages" are not like natural languages for a reason, and conciseness is a big part of that. The two fill a completely different role, and it should be obvious that the brain structures involved are different too.

To illustrate that: the only reason people are surprised that programming languages and natural languages are dissimilar is because our natural language is ambiguous: it calls them both "languages" even though they serve entirely different functions.

Re: To the brain, reading computer code is not the same as reading language

#152

Post by nutanc »

magusdei wrote: post #145
nutanc wrote: post #124

So this may mean that language models like GPT-3 are not best suited for code generation tasks. And networks trained on languages may not learn the feature embeddings needed for programming languages.

Wouldn't the empirical success of GPT-3 in simple programming tasks itself be evidence against this interpretation? Furthermore, GPT-3 is only a language model because it is trained on textual data. Transformer architectures simply map sequences to other sequences. It doesn't particularly matter what those sequences represent. GPT-2 has been used to complete images, for example: https://openai.com/blog/image-gpt/

Empirical success shows that the GPT-3 model has seen the sequence before(maybe many times).

Transformer architectures do map sequences to sequences. What is not known is that the task of programming is a sequence problem. This experiment seems to suggest that maybe its not a sequence problem.

Re: To the brain, reading computer code is not the same as reading language

#153

Post by visarga »

coldtea wrote: post #138
traceroute66 wrote: post #128

Earlier quoted context omitted.

> obsessing over grammar is almost detrimental to making progress with a new language If you skip over the hard parts, then you're not exactly learning a language. You might as well just learn from one of those "Hello, my name is Bob, where's the toilet ? " cartoon books. Its a bit like those people who learn programming informally. Most of them end up writing spaghetti code for the rest of their lives. Its the same…

> If you skip over the hard parts, then you're not exactly learning a language. That's not true. Most native speaks have little knowledge of the grammar and syntax beyond intuitive. They have forgotten most of what they've been taught at school, plus, they could speak quite well before going to school and being taught grammar in the first place. We acquire language by osmosis, not by studying, and even less so by stu…

Apparently knowing grammar acts as a filter for bad speech, but it doesn't actually generate speech itself. Learning to speak is based on direct experience, not abstract rules.

Re: To the brain, reading computer code is not the same as reading language

#154

Post by nutanc »

This may be the reason why you can still learn programming even when you are 40 years old, but its very hard for you to learn a new language the older you get as you language acquisition skills fade away.

Also the experiment was "The researchers showed them snippets of code and asked them to predict what action the code would produce.".

This may not be the best way to prove the thesis conclusively. Because when I read code I check the syntax fast and then I try to figure out logic.

"The researchers saw little to no response to code in the language regions of the brain"

This may mean that the participants quickly scanned the sytntax(language part of brain is used) and then they tried to figure out the logic(multiple demand network of brain is used)

Re: To the brain, reading computer code is not the same as reading language

#156

Post by nutanc »

jlmorton wrote: post #100

When I'm coding, I often listen to music, and when I listen to music, I often end up singing along. One thing I've noticed is that I can type out code at the same time I'm singing a song aloud. If I tried to do that while writing English, I'd start typing whatever I'm singing.

Actually when I write blogs I listen to music. So I dont think it matters a lot whether we are writing code or English. The songs and music just sort of goes in the background.

Re: To the brain, reading computer code is not the same as reading language

#157

Post by usrusr »

vncecartersknee wrote: post #64
User23 wrote: post #28

As someone who is fluently bilingual and speaks a third language and is also proficient or better in a half dozen programming languages along with having written code in many more, this is completely obvious to me. Learning a programming language is much closer to learning a complex board game than it is a natural language.

I feel exactly the same, I've been learning a second language and it's nothing like learning programming, in fact arguably the opposite, I've found that obsessing over grammar is almost detrimental to making progress with a new language. The confident students who aren't concerned with getting all the specifics of grammar exactly perfect are the ones that are making the most progress toward being conversational, they…

Yes, it seems like spoken languages can have a rather pronounced Ballmer Peak when you are learning them and are not very good at it. And the peak is much further out than with programming languages.

Re: To the brain, reading computer code is not the same as reading language

#158

Post by quickthrower2 »

Mikhail_Edoshin wrote: post #115

Niklaus Wirth wrote that the term "language" is not too fitting to computer languages and that a more precise term would be "notation".

I prefer language to notation because language means to me syntax and semantics, both are important to differentiate say Go and Java

Re: To the brain, reading computer code is not the same as reading language

#159

Post by noneeeed »

User23 wrote: post #28

As someone who is fluently bilingual and speaks a third language and is also proficient or better in a half dozen programming languages along with having written code in many more, this is completely obvious to me. Learning a programming language is much closer to learning a complex board game than it is a natural language.

As a total monoglot who regrets not learning languages properly when he was a kid, this still seems pretty obvious to me that at least I don't read code in the way I read natural language, or maths.

However it's always good to have intuitions like this confirmed or refuted by actual investigation, and putting them on a more solid footing within the broader context.

My personal experience of reading code has always felt more like looking at a mechanical mechanism, or perhaps solving a paper maze where you see the whole thing at once but can focus on parts as well. It isn't something I look at one line at a time, I read it in structural blocks, almost a breadth-first reading. I see the if/else statement and structure before I see any of the details of each branch. It's a deeply non-linear process of going up and down levels of abstraction where necessary, and ignoring details at lower levels that are irrelevant to the task at hand.

Re: To the brain, reading computer code is not the same as reading language

#160

Post by tgv »

nutanc wrote: post #154

This may be the reason why you can still learn programming even when you are 40 years old, but its very hard for you to learn a new language the older you get as you language acquisition skills fade away. Also the experiment was "The researchers showed them snippets of code and asked them to predict what action the code would produce.". This may not be the best way to prove the thesis conclusively. Because when I rea…

First: that language acquisition at a later age is so much harder is not true. It takes a child 6 to 10 years to get fluent in their first language, and they don't spend 3 hours per week studying it, but 80. For adults, there are language schools with an immersive program. They are expensive, but successful. You probably won't get as good as when you'd have been immersed from day 1 in your life, but you can get quite far.

> This may not be the best way to prove the thesis conclusively.

It certainly isn't. For starters, it does not provide a comparison to normal language.

> The researchers saw little to no response to code in the language regions of the brain

And I say: screw that logic. It's faulty beyond comprehension. That a brain region doesn't light up doesn't mean anything. It could even mean their scanner was broken.

Post reply on HN