Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Live-caption glasses let deaf people read conversations [video]

youtube.com

Post reply on HN
Print view View original

21–30 of 181 posts

Re: Live-caption glasses let deaf people read conversations [video]

#21

Post by elliekelly »

generalizations wrote: post #16
giraffe_lady wrote: post #2

Very cool, especially for people who lose their hearing later in life. For other deaf people it's important to remember that written english is not a form of their native sign language¹, so this would be like (because it is) reading captions in a second language. Still potentially useful but with more limitations. Not that there's necessarily a technological way around those limitations either. ¹ Afaik this applies t…

The problem with ASL compared to lip reading is that it's a form of self-segregation, limiting the deaf person to primarily communicating only with other people who know ASL. If these glasses are effective, it could help bridge that gap.

I suspect (and hope) you had no ill intent but this comment is really ignorant. There is a terrible history of Deaf people being discriminated against and forced to “lip read” rather than communicate through ASL. And by “forced” I mean they were more or less mentally and physically tortured into compliance.

Your comment not only perpetuates this totally false narrative that there’s a “problem” with ASL but it makes it sound like Deaf people have chosen only to socialize among themselves when the reality is that we have built a world that makes communication difficult for Deaf people. It doesn’t have to be that way: https://icyseas.org/2014/01/12/marthas-vineyard-deaf-people-...

You might find the Deaf mythology(?) of Eyeth interesting: https://www.nytimes.com/2021/10/10/opinion/deaf-population-i...

Re: Live-caption glasses let deaf people read conversations [video]

#22

Post by elliekelly »

TehShrike wrote: post #19
giraffe_lady wrote: post #17

Earlier quoted context omitted.

Why would you compare an entire language to a single technique like that? There is no "the problem" with ASL any more than there is "the problem" with english or any other language. Yes communicating across the barrier can be a challenge but that's just the nature of having more than one language in use.

Only being able to see ASL is different from speaking a different language – it's still English. Not being able to hear is more like not being able to read. It's the same language you already know, but you're missing out on one of the primary ways people use it.

No, ASL is not English. ASL is a complete and distinct natural language in its own right.

Re: Live-caption glasses let deaf people read conversations [video]

#23

Post by DrKeithDuggar »

quantumquetzal wrote: post #15

Background: researched this space for a graduate degree. There are a few issues that are unanswered by this video (which isn't intended to be a technical deep dive, but I don't see any related links in the video description): 1. How do these glasses handle multiple simultaneous speakers? Based on the display I saw, it shows the speakers' words sequentially, which starts to fall apart in real-world environments, espec…

Thank you for looking at XRAI Glass!

1. For multiple simultaneous speakers of comparable volume, it’s only as good as the underlying speech-to-text engines we’ve implemented/integrated, which is currently not very good. It’s active area of research and engineering for us and we believe we’ll make strides to improve things; but, as you rightly point out, solving the crosstalk problem is very difficult. For the more general so-called "cocktail party problem", we can do a good job of filtering out more distant/lower volume voices and other environmental noise. Choosing the right microphone can improve things further, for example by pairing a noise canceling Bluetooth lapel mic.

2. We allow one to project the subtitles at varying depth, within the capabilities of glasses. We're seeing an effective focal depth range for fixed apparent size of about 0.5m to 3m. If one also allows change in apparent size, to simulate perspective scaling, the range is higher.

Re: Live-caption glasses let deaf people read conversations [video]

#26

Post by giraffe_lady »

TehShrike wrote: post #19
giraffe_lady wrote: post #17

Earlier quoted context omitted.

Why would you compare an entire language to a single technique like that? There is no "the problem" with ASL any more than there is "the problem" with english or any other language. Yes communicating across the barrier can be a challenge but that's just the nature of having more than one language in use.

Only being able to see ASL is different from speaking a different language – it's still English. Not being able to hear is more like not being able to read. It's the same language you already know, but you're missing out on one of the primary ways people use it.

Oh you should definitely look it up then because this is completely incorrect. Sign languages are fully distinct languages with their own histories and influences.

For example the sign languages spoken in the US and in the UK have different ancestries and are not mutually comprehensible, despite both countries using english as spoken languages.

Re: Live-caption glasses let deaf people read conversations [video]

#27

Post by ogig »

danscarfe wrote: post #20

Thanks for the shout-out! To try and answer your questions: 1) The cocktail party problem is still a WIP. This is a very hard problem to solve. 2) These are not 'viewer' glasses they are 3DOF glasses which support moving and pinning the subtitles in 3D space 3) Whilst we targeted the Deaf and HOF to begin, we see broad applicability beyond this 4) You don't need glasses to test it, just an Android 12+ phone. Download…

Hi there: Some feedback from a sing language interpreter, my wife, as I showed her this.

> 4) You don't need glasses to test it, just an Android 12+ phone.

This exactly pointed her. Some deaf people will use any dictation software on the phone, and look at the phone when needed. This glasses instead will cover some field of view. Note that view for deaf people is more important and used than for the rest of us. She couldn't see the improvement of using bulky glasses instead of lowering your eyes to the phone.

Personally I think the endeavor is admirable and wish you best of luck. Also, as other comments say, I think this product might be more desirable for HOF and late in life hearing loss sectors than born deaf people.

Re: Live-caption glasses let deaf people read conversations [video]

#28

Post by elil17 »

I'm a hearing person and I've spent a summer interning in a 50/50 mixed Deaf and hearing research group.

My take is that this is a huge UI improvement for AI speech to text, which a lot of Deaf people are already using to listen to conversations. It seems particularity great because it allows this technology to provide situational awareness while, for example, walking.

It's important to remember, though that, for conversations where you're trying to include a Deaf person who isn't good at speaking or chooses not to speak, speech to text is a fundamentally unequal communication modality. They will be able to "receive" but they won't be able to "transmit", which makes for extremely lopsided conversations. There is no substitute for taking the time to learn a sign language or to have conversations via writing (no substitute for sign language as it requires a lot of patience from both parties).

Re: Live-caption glasses let deaf people read conversations [video]

#29

Post by gumby »

quantumquetzal wrote: post #15

Background: researched this space for a graduate degree. There are a few issues that are unanswered by this video (which isn't intended to be a technical deep dive, but I don't see any related links in the video description): 1. How do these glasses handle multiple simultaneous speakers? Based on the display I saw, it shows the speakers' words sequentially, which starts to fall apart in real-world environments, espec…

I have a different use case, involving wearing them in the house: listen to what my girlfriend says, use some ML to analyze whether I need to know it and if so, put it on the display.

She has a different "speech mode" than I: she speaks while I'm reading or washing the dishes or whatever, and sometimes it's to herself, sometimes to Siri, and sometimes it's something she wants me to know.

Re: Live-caption glasses let deaf people read conversations [video]

#30

Post by generalizations »

giraffe_lady wrote: post #17
generalizations wrote: post #16

Earlier quoted context omitted.

The problem with ASL compared to lip reading is that it's a form of self-segregation, limiting the deaf person to primarily communicating only with other people who know ASL. If these glasses are effective, it could help bridge that gap.

Why would you compare an entire language to a single technique like that? There is no "the problem" with ASL any more than there is "the problem" with english or any other language. Yes communicating across the barrier can be a challenge but that's just the nature of having more than one language in use.

> that's just the nature of having more than one language in use

Well, yeah. I guess my point is that ASL is pretty much a foreign language.

I'd compare it to the situation with the english language worldwide - since english is the lingua franca, so to speak, many countries around the world teach it as a second language. If you don't learn english, then (generally speaking) you're at a disadvantage because you can only communicate with a subset of your population.

I'm not saying there's a problem with deaf people, any more than there's a problem with anyone else who simply doesn't happen to know the languages of some of the people around them.

Post reply on HN