Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Can text be made to sound more than just its words? (2022)

arxiv.org

Post reply on HN
Print view View original

11–20 of 32 posts

Re: Can text be made to sound more than just its words? (2022)

#11

Post by Isognoviastoma »

foofoo12 wrote: post #9

Very interesting idea. I remember reading that in visual spoken communications, only 20% is the actual words. The rest is tone of voice, body language, context, emphasis, expressions, ... all that stuff. I don't know if 20% is correct, but I feel it's very close to it. I also think a lot of internet arguments happen as a direct result of miscommunication. Emojis are great, but they get abused to the point that HN fil…

Easy to check: try to speak with someone talking foreign language you don't know and estimate what percentage of what they said you understood from tone of voice etc. I would guess it's less than 80%.

Re: Can text be made to sound more than just its words? (2022)

#14

Post by [deleted] »

foofoo12 wrote: post #9

Very interesting idea. I remember reading that in visual spoken communications, only 20% is the actual words. The rest is tone of voice, body language, context, emphasis, expressions, ... all that stuff. I don't know if 20% is correct, but I feel it's very close to it. I also think a lot of internet arguments happen as a direct result of miscommunication. Emojis are great, but they get abused to the point that HN fil…

[deleted]

Re: Can text be made to sound more than just its words? (2022)

#16

Post by cenamus »

Isognoviastoma wrote: post #11
foofoo12 wrote: post #9

Very interesting idea. I remember reading that in visual spoken communications, only 20% is the actual words. The rest is tone of voice, body language, context, emphasis, expressions, ... all that stuff. I don't know if 20% is correct, but I feel it's very close to it. I also think a lot of internet arguments happen as a direct result of miscommunication. Emojis are great, but they get abused to the point that HN fil…

Easy to check: try to speak with someone talking foreign language you don't know and estimate what percentage of what they said you understood from tone of voice etc. I would guess it's less than 80%.

Maybe also control for cultural similarity, but I definitely agree

Re: Can text be made to sound more than just its words? (2022)

#17

Post by andai »

Many moons ago I became quite obsessed with analyzing spectrograms on my computer.

I would load up audio files in Audacity and look at them to see how the audio "looked", as a function of how intense each frequency is over time.

You can even set a track to spectrogram while recording which allowed you to see the sound in real time.

Music also tends to be very beautiful in the spectrogram! And birdsong also. Sometimes I would see a bird first, and only afterwards notice it in my field of hearing.

I noticed while analyzing a podcast that I began to recognize common words like "you." I also noticed that I was able to easily distinguish between different people's voices.

I had to wonder if I were deaf, or if I become deaf, I would suddenly have a strong motivation to learn how to read these things. To develop some kind of device which would show them to me 24 hours a day.

I have not done this, but the project has remained in the back of my mind for over a decade.

Does anyone else know more about this? Does such a device exist?

I think that only some linguists learn how to read spectrograms. But it seems like something that might be extremely useful to any hearing impaired person?

Relating to the article, I think one could quickly learn to read them fluently (e.g. as subtitles, perhaps overlaid on real life), and of course you get the tonal information built in for free—that's what a spectrogram is!

Re: Can text be made to sound more than just its words? (2022)

#19

Post by AndrewOMartin »

andai wrote: post #17

Many moons ago I became quite obsessed with analyzing spectrograms on my computer. I would load up audio files in Audacity and look at them to see how the audio "looked", as a function of how intense each frequency is over time. You can even set a track to spectrogram while recording which allowed you to see the sound in real time. Music also tends to be very beautiful in the spectrogram! And birdsong also. Sometimes…

You're on the fringe of an area which in academia is called Sensory Substitution. A simplification of which is experiencing one of the five senses using different sense organs than usual. Classic examples of this are video cameras which represent their image as a matrix of vibrations on the subjects skin or as a sound.

Re: Can text be made to sound more than just its words? (2022)

#20

Post by foofoo12 »

Isognoviastoma wrote: post #11
foofoo12 wrote: post #9

Very interesting idea. I remember reading that in visual spoken communications, only 20% is the actual words. The rest is tone of voice, body language, context, emphasis, expressions, ... all that stuff. I don't know if 20% is correct, but I feel it's very close to it. I also think a lot of internet arguments happen as a direct result of miscommunication. Emojis are great, but they get abused to the point that HN fil…

Easy to check: try to speak with someone talking foreign language you don't know and estimate what percentage of what they said you understood from tone of voice etc. I would guess it's less than 80%.

That's very easy and very wrong. Let's say you have a 100 page book. Page 1 contains fundamental knowledge that allows you to understand the rest of it. If you skip page 1 then you won't understand the other 99.

How much of the book will you understand if you only read page 1?

Post reply on HN