I spent quite some effort to _completely_ get rid of mouse usage in my computer workflows and I believe it paid off.
Reimagining the mouse pointer for the AI era
61–70 of 223 posts
Re: Reimagining the mouse pointer for the AI era
#62I don't understand why we need to move from an explicit operation like, say, circling something, to a fuzzy one where you have to hope the machine understands what you're pointing at.
I also don't think people want to constantly talk to their computers.
Re: Reimagining the mouse pointer for the AI era
#63you can really tell the people building these tools spend a lot of time alone. I work from a home office 90% of the time and I wouldn't want this to be my workflow. I don't want to talk to my computer, I want to listen to music while I work, and I want to not sound deranged and disturb everyone around me when I am working from a coffeeshop or the open-plan office or the airport or the train or whatever.
and that's aside from the obvious privacy problems.
Re: Reimagining the mouse pointer for the AI era
#64My first impression coming away from this is skepticism. Anything with voice controls for routine use is a pretty tough sell. Doing this when you're not completely alone would be annoying to everyone around you. Most of their examples seem like they could have been done with a right click drop down menu so they don't really need to "re-invent the mouse pointer". So is this thing talking to Google's servers all the ti…
The second half of your comment is a go-to-market concern but doesn't feel so relevant for a research prototype. It could be done with a private local model too, maybe not by Google.
But I don't think the voice problem is surmountable. I closed their image editing demo when I saw it required a mic.
It would be appealing as a Spotlight-like text pop-up interface where you type instructions, which would work in social/office environments, but that might only appeal to power users.
Re: Reimagining the mouse pointer for the AI era
#65My first impression coming away from this is skepticism. Anything with voice controls for routine use is a pretty tough sell. Doing this when you're not completely alone would be annoying to everyone around you. Most of their examples seem like they could have been done with a right click drop down menu so they don't really need to "re-invent the mouse pointer". So is this thing talking to Google's servers all the ti…
The "Edit an Image" Demo at the bottom is pretty fun. Maybe this is just Google flexing their LLM inference capacity.
That demo was an absolute disaster for me on Firefox on mac. It just fundamentally didn't work - the voice wasway behind my pointer, there were multiple agents speaking over each other saying conflicting things, and it couldn't even move the crab to the bottom right of the image. Embarassingly bad I would say!
Re: Reimagining the mouse pointer for the AI era
#66please leave the pointer alone. Hes been with us so long without enshittification.
Re: Reimagining the mouse pointer for the AI era
#67I've been iterating on some 3D models for various wacky garage projects I have. It's fun. I've often wished I could click on an arbitrary place and say "add an eye bolt here" or somesuch. Of course learning proper cad software is probably the right thing here, but having Claude write python scripts which generate HTML files which reference three.js to provide a 3d view has gotten me surprisingly far. If something cou…
Hah, Ive been thinking the same thing. Recently I prototyped a 2d paint app to validate the idea using chrome's Prompt API: https://arjo129.github.io/apps/voice_paint.html
Honestly what'd be epic is if this could be made to work with a XR headset. Imagine using the headset to capture the piece you are working on and generating saying "hey can you drill some holes over here"?
Re: Reimagining the mouse pointer for the AI era
#68Nice, cute, silly little feel-good demo so that we can all pretend like we’re all going to be making decisions and micro-managing AIs by pointing at things in 5 years. It’s going to be great! The future is bright!
Re: Reimagining the mouse pointer for the AI era
#69Its like watching a demo from the old Xerox PARC, except everybody has only bad ideas. Like an opposite Xerox PARC.
Re: Reimagining the mouse pointer for the AI era
#70you can really tell the people building these tools spend a lot of time alone. I work from a home office 90% of the time and I wouldn't want this to be my workflow. I don't want to talk to my computer, I want to listen to music while I work, and I want to not sound deranged and disturb everyone around me when I am working from a coffeeshop or the open-plan office or the airport or the train or whatever. and that's as…
yeah, i understand the frustration of needing to do all the communication through typing and clicking and that it can feel limiting - but i want the computer to be less demanding of my physical reality not more. i want to be able to talk to someone on the phone, work on something ith my hands, and still successfully manage my compute tasks. improvement can only be made by requiring less attention to hte screen and less hand movements, not adding in anything new like voice