Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple unveils new accessibility features

apple.com

Post reply on HN
Print view View original

101–110 of 400 posts

Re: Apple unveils new accessibility features

#101

Post by jjice »

terabytest wrote: post #78
postalcoder wrote: post #64

One thing Apple really needs to get right is speech to text transcription. They've nailed accessibility in so many ways and yet it feels like they're a decade behind on properly transcribing voices. At least half a decade. Input on the iPhone is so dreadful nowadays. Their palm rejection is definitely worse than before, so mistyping is more frequent. Their text-correction algorithm for typing is worse than before, an…

Wispr Flow is a masterclass in STT. Apple's solution feels like it's from the last century in comparison. Same applies with Apple's TTS when you have ElevenLabs and OpenAI running laps around it. All I need is for my iPhone to do those things natively at the same quality level (because in Apple's walled garden that's the only way to get them usable everywhere).

But Apple's uses so few system resources and runs fully on device on newer iPhone models (16+ I believe). It's so efficient. I really enjoy using Handy with Parakeet as the model, but the system resource usage is a monster compared to Apple's (although very good).

Looks like Wispr Flow uses a cloud model [0]:

> Cloud based speech processing infrastructure for 1B users

It gets to be a messy comparison because my iPhone can do STT with no latency pretty well fully on device, but Wispr Flow requires a cloud model, but to be fair, older Apple devices do as well. It's not an apples and oranges comparison, but I think those technical details make this a non direct comparison in a few ways.

For on-device with low system resource usage, Apple's is pretty damn good.

[0] https://wisprflow.ai/post/technical-challenges

Re: Apple unveils new accessibility features

#102

Post by jkman »

mistersquid wrote: post #46

> A new power wheelchair control feature leverages the precision eye-tracking system on Apple Vision Pro to offer a responsive input method for compatible alternative drive systems. [0] The above caption for Apple Vision Pro is for a video that to me, as an Apple Vision Pro user, is discomforting. More questions are raised than are answered by the short video: Is the user able to fit the Apple Vision Pro by him/herse…

Your concerns are completely nonsensical. It's clearly being marketed as a healthcare tool for people with debilitating injuries that preclude the use of hand-powered wheelchair controls, severe situations where there's no neck-down control and users would be limited to controls like head-tilt or mouth actuated systems. These people obviously require daily care to simply get them out of bed and into the chair and back again every single day - their nurse could just put on their Vision Pro for them! This seems like an incredible leap forward for people in this situation, if they iterate on this and it gets better then this could be a very viable wheelchair control system in the future.

Re: Apple unveils new accessibility features

#104

Post by dagmx »

dgllghr wrote: post #73

Putting aside the fact that no company should have direct access to anyone's brain, how cool would it be to be building toward VISOR (from TNG) instead of this. If we could translate sensor signals to the neural circuitry of the brain directly, we wouldn't even need an LLM in the mix. But to have it as an overlay, as supplementary data! With the ability to turn it off of course. (Would a person even be able to turn i…

Brain Control Interface support was already announced last year and afaik is part of iOS already.

https://developer.apple.com/documentation/accessibility/brai...

Re: Apple unveils new accessibility features

#105

Post by lern_too_spel »

tiffanyh wrote: post #87
nechuchelo wrote: post #4

This looks like a genuinely useful application of LLMs. I wish more companies focused on how they can help humans instead of replacing us or squeezing us as hard as possible in the name of productivity.

This is what Apple does best. They treat new industry advancements as technology , not products itself. AI will be a feature to improve the customer experience, not the product itself.

These features have existed on Android devices for years. What Apple does best is marketing.

https://blog.google/products-and-platforms/platforms/android...

https://android-developers.googleblog.com/2024/09/talkback-u...

Re: Apple unveils new accessibility features

#106

Post by kridsdale1 »

isityettime wrote: post #51
RobMurray wrote: post #41

Earlier quoted context omitted.

I know plenty of blind people who have their voice speed unbearably slow and barely scratch the surface of what technology can do for them. I think an interface where you can tell your phone what to do in natural language will really help a lot of less technical people. I'm not getting my hopes up though given apple's history with Siri, which is truly awful.

Whenever my sister (blind) and I (visually impaired) visit my mom (blind) we secretly turn up the reading speed on her TV just a little because we can't stand how unbearably slow she keeps it, but if we turn it up quickly, she'll freak out. After a few more years of Thanksgivings and Christmases and Mothers' Days, we'll finally train her up to a reasonable speed lmao.

This is heartwarming. The audio equivalent to the practice of sighted people fixing the bad default settings on boomers’ televisions each Thanksgiving.

Re: Apple unveils new accessibility features

#107

Post by miki123211 »

throwatdem12311 wrote: post #67
mohsen1 wrote: post #6

Fun fact: This video was made accessible to sighted people because no blind person would ever listen to voice at that speed. Honestly if you ever observe a blind person using computers you'd impressed how they can listen to audio at unimaginable speeds.

I did IT for a community Center way back in the day and the director was blind. I was blown away by how fast his screen reader read things out to him - completely incomprehensible to me - and his efficiency with keyboard shortcuts would put even vim/emacs elitists to shame.

The way (Windows) screen readers handle web navigation is basically Vim in disguise.

You have two modes: "focus mode", where you can edit text in text fields and keys are passed straight to the browser, and "browse mode", where keys move a virtual cursor around the page.

In browse mode, navigating with just arrow keys all the time would be just as slow as you might imagine, so you use single-key keyboard shortcuts to move by role, E.G. to the next heading, button, table or unvisited link.

The keyboard layout is optimized for memorizability and not efficiency, you use the actual arrow keys instead of hjkl for example, but the concepts are eerily similar.

There are a couple of other approaches to solve this problem, Mac OS's Voice Over is much more Emacs-like for example, and each approach has its own pros and cons, but that's definitely one way to do it.

Re: Apple unveils new accessibility features

#108

Post by lern_too_spel »

bilbo0s wrote: post #49
JimDabell wrote: post #38

Earlier quoted context omitted.

Tim Cook has been pretty clear where he stands: > “When we work on making our devices accessible by the blind,” he said, “I don’t consider the bloody ROI.” It was the same thing for environmental issues, worker safety, and other areas that don’t have an immediate profit. The company does “a lot of things for reasons besides profit motive. We want to leave the world better than we found it.” — https://www.forbes.com/s…

Again, it's absolutely great that Apple does these things! I was just answering the question of why other corporations don't. Money. There's relatively little money in helping the visually impaired. You have to do it because you want to do it. Not because you're going to get rich.

Apple's competitors have had these features for years (Android for 7, Windows for 1), so it's really an indictment of Apple. They give lip service to helping the visually impaired, and this press release is good marketing for the non-visually impaired people who don't know this.

Re: Apple unveils new accessibility features

#109

Post by kridsdale1 »

jchw wrote: post #29
embedding-shape wrote: post #18

Earlier quoted context omitted.

> Honestly if you ever observe a blind person using computers you'd impressed how they can listen to audio at unimaginable speeds. Even better, fire up Orca (or whatever screenreader application your OS comes with) yourself and try to use your computer while shutting your eyes, kind of eye-opening (no pun intended) what kind of experience these sort of users typically get. And also, you quickly start to understand wh…

When I was at Google, I'd periodically test our (internal-only) app with Chromevox with the display off. It's not that it sounded like it would be easy, but it really is a challenge, and I can only imagine the muscle memory built up over time of trying to work around accessibility bugs and strange behaviors. Unfortunately it seems impossible to get all that much funding for accessibility work :/ I wonder what ever ha…

I’ve worked at Apple Facebook and Google. Apple was the only one that made a11y bugs and a face to face consultation with a blind developer to show you how your app sucked, mandatory before you could launch.

Re: Apple unveils new accessibility features

#110

Post by gruez »

WarmWash wrote: post #65
Almondsetat wrote: post #24

I have difficulty trusting this. There are plenty of videos online of LLMs making up stuff like "I just ate a hot dog, is there mustard around my mouth?" "No, everything is clean" while there is a big yellow stain om the guy's face

The problem is using a language model to assess images. Probably 80% of "LLM's are below expectation" complaints (from the general population) involves some form of image analyses. Image tokenization is hard because unlike language tokenization, where every token is extremely dense with meaning, image tokens tends to be meaningless or irrelevant but are processed all the same. Give an SOTA LLM a picture of toothpicks…

I thought all the recent models are "multimodal"? Is the image part just sticking an image recognizer in front of the text model?
Post reply on HN