Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Live-caption glasses let deaf people read conversations [video]

youtube.com

Post reply on HN
Print view View original

11–20 of 181 posts

Re: Live-caption glasses let deaf people read conversations [video]

#11

Post by flanbiscuit »

eclipxe wrote: post #4

https://www.youtube.com/watch?v=SQd394a4qEo

Anyone know the status of these Google AR glasses?

The form factor of Google's AR glasses look much much closer to a normal pair of glasses than the glasses in the top video (which look like heavy sunglasses with a wire connecting to your phone)

You posted a CNET link, here's a direct link to Google's: https://www.youtube.com/watch?v=lj0bFX9HXeE

In the description it says: "This device has not been authorized as required by the rules of the Federal Communications Commission. This device is not, and may not be, offered for sale or lease, or sold or leased, until authorization is obtained."

So I guess they might be waiting for that?

Re: Live-caption glasses let deaf people read conversations [video]

#12

Post by kiba »

spullara wrote: post #3
giraffe_lady wrote: post #2

Very cool, especially for people who lose their hearing later in life. For other deaf people it's important to remember that written english is not a form of their native sign language¹, so this would be like (because it is) reading captions in a second language. Still potentially useful but with more limitations. Not that there's necessarily a technological way around those limitations either. ¹ Afaik this applies t…

Folks that use ASL use fingerspelling which is of course just written english, no? https://www.lifeprint.com/asl101/fingerspelling/fingerspelli...

No. Fingerspelled words are not written English, but literally spelled out English words.

For example, it would be like me saying, "'h', 'e', 'l', 'l', 'o'" instead of whatever translated 'hello' in ASL.

Re: Live-caption glasses let deaf people read conversations [video]

#13

Post by Tarragon »

spullara wrote: post #3
giraffe_lady wrote: post #2

Very cool, especially for people who lose their hearing later in life. For other deaf people it's important to remember that written english is not a form of their native sign language¹, so this would be like (because it is) reading captions in a second language. Still potentially useful but with more limitations. Not that there's necessarily a technological way around those limitations either. ¹ Afaik this applies t…

Folks that use ASL use fingerspelling which is of course just written english, no? https://www.lifeprint.com/asl101/fingerspelling/fingerspelli...

Nope. https://www.signingsavvy.com/article/45/The+difference+betwe...

Here's a fun example. ASL allows, maybe even requires, negation after the statement. An interpreter friend of mine was interpreting Wayne's World in a mixed crowd. The whole "... NOT!" joke gets laughs from the hearing audience and the Deaf audience doesn't understand why.

Re: Live-caption glasses let deaf people read conversations [video]

#14

Post by blamestross »

norgie wrote: post #5
giraffe_lady wrote: post #2

Very cool, especially for people who lose their hearing later in life. For other deaf people it's important to remember that written english is not a form of their native sign language¹, so this would be like (because it is) reading captions in a second language. Still potentially useful but with more limitations. Not that there's necessarily a technological way around those limitations either. ¹ Afaik this applies t…

It is true and something more people should understand that ASL is not a signed version English, but most ASL speakers are pretty much bilingual. They are taught to read English, and most places also encourage learning to speak English, generally with speech-language pathologists, though some in the community are understandably reluctant.

"most" is an interesting choice of word here.

It's true. 70% of deaf children learn to read English [1]. But I don't think that's enough to consider it a safe assumption.

[1] https://www.handsandvoices.org/articles/education/advocacy/w...

Re: Live-caption glasses let deaf people read conversations [video]

#15

Post by quantumquetzal »

Background: researched this space for a graduate degree.

There are a few issues that are unanswered by this video (which isn't intended to be a technical deep dive, but I don't see any related links in the video description):

1. How do these glasses handle multiple simultaneous speakers? Based on the display I saw, it shows the speakers' words sequentially, which starts to fall apart in real-world environments, especially group conversations. This is a big problem, and wider adoption is contingent on handling this elegantly.

2. These appear to be the classic "smart glasses" display style that's pervasive in consumer head-worn displays today, where content is projected at a fixed depth in front of the wearer. Because the captions aren't anchored at the same focal distance as the speaker, the wearer's eyes will swap between the captions and the speaker's faces, which is a tiring activity, and can make the wearer feel like they're not part of the conversation or being rude.

3. As mentioned by another commenter, this is a useful idea for people who lose their hearing later in life. That said, this is less (although certainly still) useful for people who have congenital hearing loss and primarily communicate via ASL.

All in all, it's exciting to see growing interest in this space, as it's easily extendable to people learning a new language or navigating a foreign country. I think offloading the speech-to-text to a tethered mobile device is a good choice (though it would be nice to do low-latency wireless transmission).

Re: Live-caption glasses let deaf people read conversations [video]

#16

Post by generalizations »

giraffe_lady wrote: post #2

Very cool, especially for people who lose their hearing later in life. For other deaf people it's important to remember that written english is not a form of their native sign language¹, so this would be like (because it is) reading captions in a second language. Still potentially useful but with more limitations. Not that there's necessarily a technological way around those limitations either. ¹ Afaik this applies t…

The problem with ASL compared to lip reading is that it's a form of self-segregation, limiting the deaf person to primarily communicating only with other people who know ASL. If these glasses are effective, it could help bridge that gap.

Re: Live-caption glasses let deaf people read conversations [video]

#17

Post by giraffe_lady »

generalizations wrote: post #16
giraffe_lady wrote: post #2

Very cool, especially for people who lose their hearing later in life. For other deaf people it's important to remember that written english is not a form of their native sign language¹, so this would be like (because it is) reading captions in a second language. Still potentially useful but with more limitations. Not that there's necessarily a technological way around those limitations either. ¹ Afaik this applies t…

The problem with ASL compared to lip reading is that it's a form of self-segregation, limiting the deaf person to primarily communicating only with other people who know ASL. If these glasses are effective, it could help bridge that gap.

Why would you compare an entire language to a single technique like that? There is no "the problem" with ASL any more than there is "the problem" with english or any other language. Yes communicating across the barrier can be a challenge but that's just the nature of having more than one language in use.

Re: Live-caption glasses let deaf people read conversations [video]

#18

Post by aeturnum »

quantumquetzal wrote: post #15

Background: researched this space for a graduate degree. There are a few issues that are unanswered by this video (which isn't intended to be a technical deep dive, but I don't see any related links in the video description): 1. How do these glasses handle multiple simultaneous speakers? Based on the display I saw, it shows the speakers' words sequentially, which starts to fall apart in real-world environments, espec…

This is a classic curbcut in the sense that it will help those with heading as much (if not more than) the D/deaf community. Still very excited for it - agree with all your questions and concerns.

As usual, this marketing seems most directed at normate ideas about what disabled people want / need, but the tech seems very cool and there does seem to be potential. Without looking into the product deeply it seems like there are D/deaf people on the team, which gives me hope.

I do wish that we would just embrace the idea that using machines to make information available in many mediums is something all people can use and appreciate.

Re: Live-caption glasses let deaf people read conversations [video]

#19

Post by TehShrike »

giraffe_lady wrote: post #17
generalizations wrote: post #16

Earlier quoted context omitted.

The problem with ASL compared to lip reading is that it's a form of self-segregation, limiting the deaf person to primarily communicating only with other people who know ASL. If these glasses are effective, it could help bridge that gap.

Why would you compare an entire language to a single technique like that? There is no "the problem" with ASL any more than there is "the problem" with english or any other language. Yes communicating across the barrier can be a challenge but that's just the nature of having more than one language in use.

Only being able to see ASL is different from speaking a different language – it's still English. Not being able to hear is more like not being able to read.

It's the same language you already know, but you're missing out on one of the primary ways people use it.

Re: Live-caption glasses let deaf people read conversations [video]

#20

Post by danscarfe »

Thanks for the shout-out! To try and answer your questions:

1) The cocktail party problem is still a WIP. This is a very hard problem to solve.

2) These are not 'viewer' glasses they are 3DOF glasses which support moving and pinning the subtitles in 3D space

3) Whilst we targeted the Deaf and HOF to begin, we see broad applicability beyond this

4) You don't need glasses to test it, just an Android 12+ phone. Download and try it. We'd love your feedback https://link.xrai.glass/app

Post reply on HN