31–38 of 38 posts
From what I understand, before Shannon showed up no one knew how to calculate the Channel Capacity of a medium and how to deal with noise in the medium. So people just yelled louder or yelled repeatedly to get things across with better reliability. And when channel capacity was exceeded with everyone yelling too much, no one knew it had exceeded and lot of energy and time was wasted with increasing errors in the syst…
Curiosity Stream has a recent video centering around a fictionalized interview with Claude Shannon: https://curiositystream.com/video/3831 If you sign up through a YouTube promotional link, you also get access to Nebula for free: https://curiositystream.com/legaleagle/index.html
Although not devoted to Claude Shannon, Jim Al-Khalili's documentary "The Story of Information" does discuss Shannon's significant contributions and is how I first became aware of Claude Shannon.
Earlier quoted context omitted.
One key insight of Shannon's is that to study information as he did, you need to remove "meaning". You could measure the information content of anything and it doesn't matter what the symbols mean. Paradoxically, a random sequence of characters of the same length as this message would contain more information than this message that I am writing. --- The connection you draw to social media is cute, and I think it coul…
Now we need an analog of Shannon in social sciences to offer a Theory of Meaning. Then perhaps a century or so where there are still incompatibilities with the theory of information that prevent a bigger picture from emerging, and eventually an all-encompassing theory ushering humanity into an Era of Meaning. Ted Chiang could probably write a plausible future with that as a writing prompt.
That's called semiotics and it was first developed in the late 19th century by Ferdinand de Saussure.
> Saussure approaches theory of language from two different perspectives. On the one hand, language is a system of signs. That is, a semiotic system; or a semiological system as he himself calls it. On the other hand, a language is also a social phenomenon: a product of the language community.
> One of Saussure's key contributions to semiotics lies in what he called semiology, the concept of the bilateral (two-sided) sign which consists of 'the signifier' (a linguistic form, e.g. a word) and 'the signified' (the meaning of the form). Saussure supported the argument for the arbitrariness of the sign although he did not deny the fact that some words are onomatopoeic, or claim that picture-like symbols are fully arbitrary. Saussure also did not consider the linguistic sign as random, but as historically cemented.[a] All in all, he did not invent the philosophy of arbitrariness, but made a very influential contribution to it.
> After his death, structural and functional linguists applied Saussure's concept to the analysis of the linguistic form as motivated by meaning. The opposite direction of the linguistic expressions as giving rise to the conceptual system, on the other hand, became the foundation of the post-Second World War structuralists who adopted Saussure's concept of structural linguistics as the model for all human sciences as the study of how language shapes our concepts of the world. Thus, Saussure's model became important not only for linguistics, but for humanities and social sciences as a whole.
> Information Theory was created by C.E.Shannon in the late 1940s. The management of Bell Telephone Labs wanted him to call it “Communication Theory” as that is a far more accurate name, but for obvious publicity reasons “Information Theory” has a much greater impact—thus Shannon chose and so it is known to this day.
[1] https://press.stripe.com/#the-art-of-doing-science-and-engin... [2] http://worrydream.com/refs/Hamming-TheArtOfDoingScienceAndEn...
Shannon was cursed by being the inventor of information theory. It seems like this wonderful thing; who wouldn’t want to invent information theory? But as an inventor, you’ll never top it. You’ll forever live in your own shadow. And for someone that enjoys inventing things, that’s... not a happy thing.
Nothing I've read about Shannon suggests that he felt "cursed" in his later life. Is there some reason to think otherwise, or is this just assuming that anybody who does something big early feels sad about it later?
https://en.wikipedia.org/wiki/A_Symbolic_Analysis_of_Relay_a...
Earlier quoted context omitted.
The alleged origins of the naming of entropy: When Shannon first derived his famous formula for information, he asked von Neumann what he should call it and von Neumann replied “You should call it entropy for two reasons: first because that is what the formula is in statistical mechanises but second and more important, as nobody knows what entropy is, whenever you use the term you will always be at an advantage! From…
Yeah, I would characterize it as a mathematical theory of -symbols- and that information has different definitions. However, the argument can be made that symbolic representation is a principle of reality.
Earlier quoted context omitted.
The timing is a little less funny if you know that I listened to the podcast shortly before finding the IEEE article. The podcast was great! I would think that many on HN would find it interesting.
Cautionary Tales is my favourite podcast. Most episodes concern either interesting (mis)use of statistics or cognitive biases, both of which are topics I find inherently interesting, but Tim Harford and team do a brilliant job of the storytelling too. Tim Harford's other podcasts are great too - More or Less, and 50 Things Which Made the Modern Economy. Does anyone have any recommendations along similar lines?
Funny timing. Tim Harford's Cautionary Tales podcast had an episode about Shannon. He sounds like a fun guy to have been around. https://timharford.com/2021/05/cautionary-tales-fritterin-aw...