Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Streaming AI agent desktops with gaming protocols

blog.helix.ml

Post reply on HN
Print view View original

31–40 of 52 posts

Re: Streaming AI agent desktops with gaming protocols

#31

Post by _ea1k »

vladgur wrote: post #24

I’m curious how far are we from giving coding agents access to these desktop agents so that when we are using say Claude Code to build a native desktop app, the coding agents can actually see and act on the desktop UI that it is building

For web apps, I'd guess that many of us already do that via Playwright or other MCPs. I'd bet there are people doing something similar with desktop apps too.

Re: Streaming AI agent desktops with gaming protocols

#33

Post by tomnipotent »

seemaze wrote: post #17
DrewADesign wrote: post #3

Earlier quoted context omitted.

A career change that left me as a recent graduate in a decimated marketplace missing the bottom ten rungs on the ladder and no interest in getting back into the software world has led me to advanced manufacturing as a metal worker. I code a little, move heavy steel pieces periodically which is a nice way to break up the standing/sitting but not nearly as much as a general laborer, solve lots of problems, keep my trig…

There’s 1,000 established industries that don’t offer the rapid growth and pay outs of the modern tech ecosystem. I’m excited to see some of the current industrial backwaters soak up technical talent freed up by the SV AI brain drain. To think we’ve handsomely paid our best and brightest the last few decades in pursuit of.. advertising?

> don’t offer the rapid growth

How are these industries going to absorb new headcount without the revenue to support it?

Re: Streaming AI agent desktops with gaming protocols

#34

Post by LunicLynx »

I find this very curious. I don’t think agents care about UI, humans do. So in the end the UI is not required. As soon as the ai can get into the physical world. The whole IT world is done for. All of this will be automated away. The IT and CS only ever started to make us more productive more connected to improve our physical well being. When we don’t need to touch computers anymore there is no need for …

Re: Streaming AI agent desktops with gaming protocols

#35

Post by lewq »

LunicLynx wrote: post #34

I find this very curious. I don’t think agents care about UI, humans do. So in the end the UI is not required. As soon as the ai can get into the physical world. The whole IT world is done for. All of this will be automated away. The IT and CS only ever started to make us more productive more connected to improve our physical well being. When we don’t need to touch computers anymore there is no need for …

Vision language models have been trained on how to operate human UIs though, so at least for a while, computer use will be an interesting area to explore. I think debugging web apps and building UIs is a particularly fruitful area for this

Re: Streaming AI agent desktops with gaming protocols

#36

Post by lewq »

vladgur wrote: post #32

Another question regarding Helix - its being built as a platform for private air-gap-ready ai agents that can work against private LLM models. Are there appliances or easy to deploy hardware that allow one to run these private models on-premise vs cloud

Hey! Yeah we are working with partners on fully integrated hardware+software stack for this. We particularly like the RTX 6000 Pro Blackwell chips for this

Re: Streaming AI agent desktops with gaming protocols

#37

Post by mxkopy »

I’ve also independently concluded Moonlight was the best way to go after trying my hand at a very similar task. I didn’t want to dig through moonlight’s source, but I’m sure if you’re dedicated enough it would pay dividends later on, it basically does everything you’d need for realtime control in the setting of simulating human input.

Re: Streaming AI agent desktops with gaming protocols

#39

Post by dude250711 »

_cenw wrote: post #2

> because we’re all going to become managers of coding agents whether we like it or not I will join the woodworking people before that happens, thanks.

Either the crap truly works and nobody is needed or it does not work. Where is this half-arsed human-agent hybrid vision coming from? The land of plateaued LLM gains?
Post reply on HN