Live data from Hacker News

Show HN: What should the GUI for AI agents look like?

marbleos.com

81–90 of 98 posts

Re: Show HN: What should the GUI for AI agents look like?

#81
Hear me out. Photoshop UX.

History panel to Undo/Redo.

Layers panel where each layer is a separate LLM context (conversation). Maybe call it Contexts.

Main view area is the current preview of what's being worked on - whether it's an image, writing, or software. Visually the main area is just a white square canvas with box shadow and rulers across the top and side until you start making stuff.

Unlike Photoshop have the ability to go multi-page (like a PDF), or multi-canvas. So I can write a book, design a comic, organize video shots, or organize software screens in the same canvas-based UX.

Contexts ends up being like Layers in that they're each manipulating the same shared canvas, but on isolated channels.

The engineering of this software should be so good that you can turn a context (eyeball icon) and see what the result would be without it - so, toggle on/off a code refactor, product placements in a film, or changes in writing style to a book.

Be able to swiftly navigate the object of creation, make changes, try things, and thanks to AI go far with each change, and operate the entire workspace like a creative suite with one-off tools that enhance, or auto-fill text, or isolate - should be able to right click anything isolated and "Generate new..." of whatever it is.

Anyway

Re: Show HN: What should the GUI for AI agents look like?

#84
post #35
post #10

Best interface in my opinion is a git tracked folder, files as state, agents coming in and doing work. It's not just a chat interface because in chat mode everything flows like water under the bridge. It is files as agents, where you get the agent to write a task in a file, and then it updates the file like a blackboard as it performs work. Same task file can be seen by worker agent, multiple judge agents (for plan a…

Sure but this is a very software engineer coded answer What about people who don't think like you

All specialists have the tools they like and use daily. It's better to have a familiar UX than the bare chat window. For example, accountants have very strong UX habits

Re: Show HN: What should the GUI for AI agents look like?

#85

Very interesting idea, I actually wonder if you are not going far enough? See this video from the lovely YouTube channel Interface Studies : https://www.youtube.com/watch?v=ARjisP9enZ4 I'm generally very aligned to the thinking in that video. AI offers an opportunity to take a massive step-back and re-evaluate what interfaces are in general. IMHO Douglas Englebart had a very compelling and beautiful vision for what c…

Thanks for the video rec! Engelbart was heavily influenced by the 1945 essay "As We May Think". We agree with the general principles from the essay i.e. cognitive augmentation over automation and the focus on extending our cognitive ability.

We just want to make sure we're grounded in real-world practical utility because it's easy for ambitious interface ideas to remain demos. We have a lot more exciting, weird, and strange ideas to come!

Re: Show HN: What should the GUI for AI agents look like?

#87

Creative take, I like the visual representation on infinite canvas a lot. Feels like freedom. What doesn't feel like freedom nor future is the manual selection of tools. AI should naturally pick the right tools out of all possible tools, choosing what abilities or data it can have each time is hassle. What is the reasoning for manual and explicit tools? Still, very cool UI :-)

The AI will auto-select tools if your request has enough context for it to pick up on the relevant tools. That said, sometimes you want to think in terms of "tool first" rather than prompting on its own.

Re: Show HN: What should the GUI for AI agents look like?

#88
I built one of these for trading agents. Two things I had wrong at the start.

1. I showed what each agent was thinking. Nobody read it. People wanted what it did and what it was about to do.

2. The stop button belongs on every screen. The moment you can't find it is the moment you need it.

Here is my sight deadcatfound.org . I would be happy to assist you. This is a true fun passion/ hobby. The best way to kill time before GTA comes out lol.

Re: Show HN: What should the GUI for AI agents look like?

#89
Nice idea. I like that you’re exposing it through MCP instead of another custom API.

Curious—have you thought about how you’ll handle discoverability? It feels like there are a growing number of useful AI agents and MCP servers, but they’re scattered across GitHub, X, and personal websites.

Re: Show HN: What should the GUI for AI agents look like?

#90

Very interesting idea, I actually wonder if you are not going far enough? See this video from the lovely YouTube channel Interface Studies : https://www.youtube.com/watch?v=ARjisP9enZ4 I'm generally very aligned to the thinking in that video. AI offers an opportunity to take a massive step-back and re-evaluate what interfaces are in general. IMHO Douglas Englebart had a very compelling and beautiful vision for what c…

Thanks for the video rec! Engelbart was heavily influenced by the 1945 essay "As We May Think". We agree with the general principles from the essay i.e. cognitive augmentation over automation and the focus on extending our cognitive ability. We just want to make sure we're grounded in real-world practical utility because it's easy for ambitious interface ideas to remain demos. We have a lot more exciting, weird, and…

Best of luck! I hope in the not too distant future we see a cool demo from you where there's something mind-blowing happening :)
Post reply on HN