Just seven hours ago there was a plea on HN [0] to please not do this. Seriously, what are they smoking at Google right now?
Reimagining the mouse pointer for the AI era
31–40 of 223 posts
Re: Reimagining the mouse pointer for the AI era
#32Maybe I'm misunderstanding, but what is new about the pointer itself? Seems to be functionally the same as selecting + tooltips / context menus.
> but what is new about the pointer itself?
I'm hoping for a const-reference joke.
Re: Reimagining the mouse pointer for the AI era
#33There's already a product that does this lol
Aaaaand now I can't remember the name of it
Re: Reimagining the mouse pointer for the AI era
#34Don't build these things, instead build protocols and expose system level APIs for application developers to build things.
Re: Reimagining the mouse pointer for the AI era
#35Oh interesting, this is very cool. At first I thought it was just focus-follows-mouse but it's more interesting. You have certain keywords trigger "add to prompt". Ignoring the voice functionality (which is admittedly crucial currently because other inputs currently take over focus), I've often wanted to just have a continuous conversation with the LLM as I 'point and click' (or tab over and select) at various things. Might be neat to have text input focus continue to go to the LLM where I'm typing text etc.
Sometimes I go to a different page to take a screenshot and other times I'm browsing for a file, and other times I'm highlighting some log lines. Cursor did this well, with selecting text in the terminal auto-focusing the Cursor agent textbox so you could talk to the agent and then select some text and you didn't have to re-select the original agent textbox again. The agent is a top-level function in that system not "just another app I have to switch to" to take my context with.
I have some small amount of bias because I've always felt input-constrained on computers. I have to move my hands to go places and that's exasperating. I've tried head tracking, had a vim pedal for a while, and used tiling WMs, and things like this to aid but while my vim-fu is pretty good and I function inside things very well with it, my cross-application interface isn't.
In the end, perhaps we all have our home offices with our Apple Vision Pros and we talk to them like this to maneouvre faster through our machines and get our ideas into them.
Cool research. I wonder what we'll end up with.
Re: Reimagining the mouse pointer for the AI era
#36Please don't.
I like text selection exactly how it is. I want precise controls.
It's fine for a touch interface like a phone, but on a computer I expect precision. As much as I can get.
Re: Reimagining the mouse pointer for the AI era
#37My first impression coming away from this is skepticism. Anything with voice controls for routine use is a pretty tough sell. Doing this when you're not completely alone would be annoying to everyone around you. Most of their examples seem like they could have been done with a right click drop down menu so they don't really need to "re-invent the mouse pointer". So is this thing talking to Google's servers all the ti…
Right — it does seem cool but the voice is patching over a major gap. If I'm talking already, why wouldn't I just describe what I'm looking at and have the AI grab it for me?
Re: Reimagining the mouse pointer for the AI era
#38Both of the text based demos would have been simpler and faster with traditional mouse and keyboard interactions. What is the AI adding?
Hype-flavored surveillance!
Re: Reimagining the mouse pointer for the AI era
#39> We’ve been exploring new AI-powered capabilities to help the pointer not only understand what it’s pointing at, but also why it matters to the user. We couldn't quite track you well enough before. So we're fixing that under the guise of "AI powered capabilities."
[deleted]
Re: Reimagining the mouse pointer for the AI era
#40Google needs to beat OpenAI and Antropic in coding models because that's where the big money is going. I love using the Gemini pro model for quick questions, but that's not where I'm spending the real money.
They have so many great software engineers but unable to use them to speed up coding AI research. Hopefully with Sergey's focus it will get better.
This cursor thing is just another experiment nobody cares about.