Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Researchers: It takes 1.5 MB of data to store language information

medicalxpress.com

Post reply on HN
Print view View original

31–40 of 59 posts

Re: Researchers: It takes 1.5 MB of data to store language information

#31

Post by MisterTea »

segfaultbuserr wrote: post #15
timw4mail wrote: post #12

Earlier quoted context omitted.

Not quite. Considering the most common floppies stored a bit less than 1.44MB.

One feature of floppy disks is that the recording technology and the format is independent from the physical medium, the limitations are mostly imposed by the disk drive, the encoding and the file system, not the medium itself. Well, this "feature" creates significant compatibility issues as all the vendors created their own proprietary formats, sometimes with patents. As a result, together with the economics of scal…

I always wanted to know how far you could push the capacity of an old standard density floppy drive if one hacked on modern control electronics. Things like adding more tracks and variable sectors per track. Though I think you'd be limited by the magnetic density of the media and physical head size. Still, it would be fun to see just how much more data you could squeeze onto an old 1.44MB floppy.

Re: Researchers: It takes 1.5 MB of data to store language information

#32

Post by madhadron »

coldtea wrote: post #24
MrQuincle wrote: post #11

This describes how to summarize a language in a computer system. I think the storage requirements for the semantics is in particular questionable. If you can refer to a particular concept somewhere in the brain you only need a pointer to it. However, single pointers might not be sufficient The apparatus to learn a language might also not so easily distinguished from the structure to store the language or the structur…

> This describes how to summarize a language in a computer system. It describes how to summarize a language from a information theory standpoint (which is something different). There's a good reason to believe evolution would use something close to the most information theoretic efficient implementation (and even if not, it's a good lower bound)

> There's a good reason to believe evolution would use something close to the most information theoretic efficient implementation

Do you have a citation? I would be very interested to see it.

I hate to make analogies between computing and biological systems, but I immediately think of denormalization in databases, error correcting codes, and database replication as examples of optimizations that take you farther from information theoretic efficiency.

Information applies very straightforwardly to the study of a spike train of a synapse (source, channel, and receiver). I remain unconvinced that it systems like memory make sense as information channels.

Re: Researchers: It takes 1.5 MB of data to store language information

#34

Post by nbabitskiy »

simias wrote: post #9
LKAndrew wrote: post #4

How would it be possible that 40,000 words translates to 400,000 bits?! Am I missing something here?

Compression I suppose. Like storing the word "compression" is easier if you already know the familiar "com-" prefix as well as the noun "pression". Which is itself easier to remember if you know the verb "to press". I've been actively learning Portuguese and Russian lately, it's impressive how much faster I can pick up Portuguese vocabulary vs. Russian. And that's even for words that don't have an obvious cognate in…

English cognate to "atreve" is "attribute".

~ов is an iterative suffix - like ~le or ~er in gamble and chatter. It's useful to know, because you can rationalise why it is always dropped in the present tense - you can't be iterative at the moment, unless you're an Englishman)

Re: Researchers: It takes 1.5 MB of data to store language information

#35

Post by mrec »

coldtea wrote: post #25
mrec wrote: post #18

Earlier quoted context omitted.

"Compression" applies very much to grammatical variants of words. It's why the only bizarre irregular verbs in a language tend to be the ones that get used all the time - be, do, go etc - because for anything more obscure the brain just forgets the special case and applies the general rule. Steven Pinker's book Words and Rules is a great layman-oriented read if this sort of thing interests you.

It also makes sense why the most common verbs are irregular in most languages: to have us pick the direct word quickly, instead of the slower way of deriving it from a rule. So, they're like "constants" vs calling a function to calculate a value.

Not sure I follow. If (big if) there were really a measureable advantage to having us "pick the direct word quickly, instead of the slower way of deriving it from a rule", it doesn't follow that irregularity makes that easier. I could memorize "goed" just as easily as I can memorize "went".

Re: Researchers: It takes 1.5 MB of data to store language information

#36

Post by simias »

nbabitskiy wrote: post #34
simias wrote: post #9

Earlier quoted context omitted.

Compression I suppose. Like storing the word "compression" is easier if you already know the familiar "com-" prefix as well as the noun "pression". Which is itself easier to remember if you know the verb "to press". I've been actively learning Portuguese and Russian lately, it's impressive how much faster I can pick up Portuguese vocabulary vs. Russian. And that's even for words that don't have an obvious cognate in…

English cognate to "atreve" is "attribute". ~ов is an iterative suffix - like ~le or ~er in gamble and chatter. It's useful to know, because you can rationalise why it is always dropped in the present tense - you can't be iterative at the moment, unless you're an Englishman)

>English cognate to "atreve" is "attribute".

I didn't know that, but you'll grant me that it's not a very useful cognate (either in spelling or in meaning).

>~ов is an iterative suffix - like ~le or ~er in gamble and chatter. It's useful to know, because you can rationalise why it is always dropped in the present tense - you can't be iterative at the moment, unless you're an Englishman)

Very interesting, thanks.

Re: Researchers: It takes 1.5 MB of data to store language information

#37

Post by randyrand »

rob74 wrote: post #27
wrinkl3 wrote: post #14

I haven't read the paper itself, but the article makes it sound like they simply counted the minimum number of bits required to represent the English language. It mentions no neuroscience-specific insights, so the "brain" part of the title is quite misleading.

Aside from that - is there really a way to express neural connections in the brain in terms of bits and bytes?

most definitely. ANN's do just that.

That said, they assume a computer. and a program to run it. Similarly, the brain probably has many complex lower structures that compress language storage, but the compression software is not trivial to store itself.

Re: Researchers: It takes 1.5 MB of data to store language information

#38

Post by simias »

mrec wrote: post #35
coldtea wrote: post #25

Earlier quoted context omitted.

It also makes sense why the most common verbs are irregular in most languages: to have us pick the direct word quickly, instead of the slower way of deriving it from a rule. So, they're like "constants" vs calling a function to calculate a value.

Not sure I follow. If (big if) there were really a measureable advantage to having us "pick the direct word quickly, instead of the slower way of deriving it from a rule" , it doesn't follow that irregularity makes that easier. I could memorize "goed" just as easily as I can memorize "went".

I took it the other way around: it's easy to memorize "went" because you use it all the time. If on the other hand a much less common verb like "to satiate" had a very irregular conjugation then it would regularize pretty quickly because nobody but ultra-pedants would bother to remember the exception.

I think a decent real example of that is fiancé/fiancée, those are french borrowings and have, at least originally, kept the French grammatical gender inflection. However nowadays I often see people using either spelling in a gender-neutral way since most people don't bother to learn French grammar for this one word.

Re: Researchers: It takes 1.5 MB of data to store language information

#39

Post by mannykannot »

coldtea wrote: post #24
MrQuincle wrote: post #11

This describes how to summarize a language in a computer system. I think the storage requirements for the semantics is in particular questionable. If you can refer to a particular concept somewhere in the brain you only need a pointer to it. However, single pointers might not be sufficient The apparatus to learn a language might also not so easily distinguished from the structure to store the language or the structur…

> This describes how to summarize a language in a computer system. It describes how to summarize a language from a information theory standpoint (which is something different). There's a good reason to believe evolution would use something close to the most information theoretic efficient implementation (and even if not, it's a good lower bound)

> There's a good reason to believe evolution would use something close to the most information theoretic efficient implementation.

People used to think that about genomes, until we got to know in some detail what's in them.

Re: Researchers: It takes 1.5 MB of data to store language information

#40

Post by coldtea »

mrec wrote: post #35
coldtea wrote: post #25

Earlier quoted context omitted.

It also makes sense why the most common verbs are irregular in most languages: to have us pick the direct word quickly, instead of the slower way of deriving it from a rule. So, they're like "constants" vs calling a function to calculate a value.

Not sure I follow. If (big if) there were really a measureable advantage to having us "pick the direct word quickly, instead of the slower way of deriving it from a rule" , it doesn't follow that irregularity makes that easier. I could memorize "goed" just as easily as I can memorize "went".

>I could memorize "goed" just as easily as I can memorize "went".

It wouldn't have anything distinctive for people to latch on to, so they would be constantly trying to derive it from the general rules for regular verbs.

E.g.

(a) all verbs regular -> instinctively go to (slower) rule derivation instead of memorization of all verbs, even for the most common ones.

(b) most frequent verbs being irregular -> instinctively retrieve them from the (faster) "lookup table" of memorizations, and bypass the rule based derivation for them.

I.e having the clear distinction of irregularity makes it faster to go directly to that kind of "constant" memory.

That said, this is not my theory, read it years ago in a cognitive/linguistic pop science article. This seems to say more or less the same thing:

https://en.wikipedia.org/wiki/Regular_and_irregular_verbs#Li...

In studies of first language acquisition (where the aim is to establish how the human brain processes its native language), one debate among 20th-century linguists revolved around whether small children learn all verb forms as separate pieces of vocabulary or whether they deduce forms by the application of rules. Since a child can hear a regular verb for the first time and immediately reuse it correctly in a different conjugated form which he or she has never heard, it is clear that the brain does work with rules; but irregular verbs must be processed differently.

Post reply on HN