[deleted]
Live-caption glasses let deaf people read conversations [video]
131–140 of 181 posts
Re: Live-caption glasses let deaf people read conversations [video]
#132I'm a hearing person and I've spent a summer interning in a 50/50 mixed Deaf and hearing research group. My take is that this is a huge UI improvement for AI speech to text, which a lot of Deaf people are already using to listen to conversations. It seems particularity great because it allows this technology to provide situational awareness while, for example, walking. It's important to remember, though that, for con…
Why can’t they also make glasses that translate sign language into English audio or text then?
Microsoft Research had early implementation for this kind of visual sign language translation with Kinect & ASL recognition in 2013 or so. I expect that with the death of Kinect in the market it stayed lab-bound.
Re: Live-caption glasses let deaf people read conversations [video]
#133Earlier quoted context omitted.
Sign language is a complex language, so something has to learn yet another language. Perhaps made all the more complex as it's spatial and visual instead of verbal or written. Not every deaf person uses sign language and there are many different sign languages in the world. American Sign Language(ASL) is but one of these languages.
> Sign language is a complex language Specifically, sign languages are not visual representations of existing languages (e.g. ASL and English) but completely different languages altogether.
I was waiting for someone to say this. I have taught Humanities in a program which included a significant number of deaf students. Communication was enabled via translators. The difference in language culture plus the fact that everything was mediated through a third party made the whole experience profoundly unsatisfactory. Nuance was completely lost. Perhaps if I had recovered training in sign language things world have been different.
Re: Live-caption glasses let deaf people read conversations [video]
#134So I dug around a little bit and figured I'd just ask. As just some guy on the internet, can I buy one of these and write a hello world to have text show up in front of my eyes of my own choosing? Does it have an API or will it?
You can indeed. It's called the Nreal sdk
Re: Live-caption glasses let deaf people read conversations [video]
#135Earlier quoted context omitted.
You can see how it appears in our promo video: https://www.youtube.com/watch?v=iilN0368vQU
The video gives the general idea, but it only shows a graphical simulation of what the wearer sees. I'm curious to see an actual photo or video of the text appearing on the lens of the glasses, in reality, with no special effects.
It doesn't appear on the glasses. It appears in 3D space infront of you. It's almost impossible to show via video.
Re: Live-caption glasses let deaf people read conversations [video]
#136Re: Live-caption glasses let deaf people read conversations [video]
#137Earlier quoted context omitted.
We've taken a privacy-first approach here. All data is only ever stored on the device, owned by the user, inaccessible to us. It's only ever transcribing when a user asks it to and only stored if the user asks it to be. We don't censor anything. We are soon to release purely on-device transcription, but the quality of this is still not as good as the cloud providers offer. The app itself is what powers the glasses, t…
> We don't censor anything. I hadn't thought that you would be, but I don't know anyone who hasn't had at least one service they used change their terms for the worse over time, especially once it becomes successful or ownership/management changes. Better to safeguard against such problems before they are problems than come to depend on a device like this only to find yourself stuck when the rug gets pulled out from…
We are soon releasing versions for iOS, Mac and Windows. We are also hoping to release an extensibility model to allow others to build skins / modules. The glasses will only work properly on Android though.
Re: Live-caption glasses let deaf people read conversations [video]
#138Earlier quoted context omitted.
The video gives the general idea, but it only shows a graphical simulation of what the wearer sees. I'm curious to see an actual photo or video of the text appearing on the lens of the glasses, in reality, with no special effects.
It doesn't appear on the glasses. It appears in 3D space infront of you. It's almost impossible to show via video.
Oh, interesting. It's not being projected onto the lenses, but straight into the eye or something?
Re: Live-caption glasses let deaf people read conversations [video]
#139Background: researched this space for a graduate degree. There are a few issues that are unanswered by this video (which isn't intended to be a technical deep dive, but I don't see any related links in the video description): 1. How do these glasses handle multiple simultaneous speakers? Based on the display I saw, it shows the speakers' words sequentially, which starts to fall apart in real-world environments, espec…
Is multiple simultaneous speakers at the same volume/distance actually an important problem to solve? I already can't have a conversation if that's happening and my hearing is fine.
Let’s say you’re in a meeting with colleagues who are having a side discussion about something, which you can contribute to. In this current setup, that conversation would not be displayed, while as a hearing person, you could context switch on a dime.
It’s a decently-common occurrence. There’s a significant amount of literature pointing to the fact that people who are DHH avoid group conversations because they can’t follow along (see references). This sort of falls hand-in-hand with speech localization, which is another unsolved problem (and likely is unsolvable without true AR).
https://journals.sagepub.com/doi/10.1177/1084713807301322
Re: Live-caption glasses let deaf people read conversations [video]
#140Earlier quoted context omitted.
>3. As mentioned by another commenter, this is a useful idea for people who lose their hearing later in life. That said, this is less (although certainly still) useful for people who have congenital hearing loss and primarily communicate via ASL.> Someone primarily communicates with ASL and then there's me that doesn't know ASL. I can speak to them, and they can read what I've spoken. That works pretty well. They com…
ASL is heavily inspired by English, so it's usually very easy for people to become basically conversant in it rather quickly. For signs you don't know, there's literal "finger spelling" that's part of the language, so conversational learning is greatly aided by this. Which.. aside from that, you could do this anyways. When I first started living with a deaf person, I just wrote things down on paper, and they wrote ba…
I would disagree! It takes time and resources to learn ASL or even fingerspelling. This would allow people without the time/resources to communicate effortlessly, which I would say is a marked improvement. That said, you could argue it would be a one-way conversation, which is a fair point.