Live data from Hacker News

Apps SDK

developers.openai.com

181–190 of 397 posts

Re: Apps SDK

#181
I can't understand the documentation. How are the interactive elements embedded in the chat? Are they just iFrames?

The docs mention returning resources, and the example is returning a rust file as a resource, which is nonsensical.

This seems similar to MCP UI in result but it's not clear how it works internally.

Re: Apps SDK

#183
post #125

Earlier quoted context omitted.

I have yet to see a chat agent deployed that is more popular than tailored browsing methods. The most charitable way to explain this is that the tailored browsing methods already in place are the results of years of careful design and battle testing and that the chat agent is providing most of the value that a tailored browsing method would but without any of the investment required to bring a traditional UX to fruit…

> the tailored browsing methods already in place are the results of years of careful design and battle testing Have you ever worked in a corporation? Do you really think that Windows 8 UI was the fruit of years of careful design? What about Workday? > but it is bizarre that so many businesses seem to be discarding battle tested UXes for chatbots Not really. If the chatbot is smart enough then chatbot is the more natu…

In an alarm, there is only one parameter to set. In more complex tasks, chat is a bad ui because it does not scale well and it does not offer good ways to arrange information. Eg if I want to buy something and I have a bunch of constraints, I would rather use a search-based UI where i can fast tweak these constraints and decide. Chathpt being smart or not here is irrelevant, it would just be bad ui for the task.

Re: Apps SDK

#184
post #12

There was a recent post here about how deeply ingrained the chat interface is in OpenAIs organization. This really doubles down on that, but does anyone really like to interact with so much language instead of visual elements? Also feels horrible that you are supposed to remember a bunch of app names like "zillow" and punch them in the chat. And like an opportunity for them to slowly introduce ads for this apps or "p…

I feel like we're rehashing the debate around whether or not a GUI or terminal is more powerful.

For a large number of tasks that cleanly generalize into a stream of tokens, command line or chat is probably superior. We'll get some affordances like tab auto completion to help remember the name of certain bots or mCP endpoints that can be brought in as needed...

But for anything that involves discovery, graphical interaction feels more intuitive and we'll probably get bespoke interfaces relevant to that particular task at hand with some sort of partially hidden layers to abstract away the token stream?

Re: Apps SDK

#185

This conception makes sense iff you believe in ChatGPT as the universal user interface of the future. If anything the agentic wave is showing that the chat interfaces are better off hidden behind stricter user interface paradigms.

>If anything the agentic wave is showing that the chat interfaces are better off hidden behind stricter user interface paradigms. I'm not sure that claim is justified. The primary agentic use case today is code generation, and the target demographic is used to IDEs/code editors. While that's probably a good chunk of total token usage, it's not representative of the average user's needs or desires. I strongly doubt th…

Having already seen this explored late '24, what ends up happening is that the end user generates apps that have lots of jank, quirks, and logical errors that they lack the ability to troubleshoot or resolve. Like the fast forward button corrupting their settings config, the cloud sync feature causing 100% CPU load, icons gradually drifting away from their original positions on each window resize event, or the GUI tutorial activating every time they switch views in the app. Even worse, because their app is the only one of its kind, there is no other human to turn to for advice.

Re: Apps SDK

#186

I can't understand the documentation. How are the interactive elements embedded in the chat? Are they just iFrames? The docs mention returning resources, and the example is returning a rust file as a resource, which is nonsensical. This seems similar to MCP UI in result but it's not clear how it works internally.

If the connector is enabled by the prompt or via a UI interaction, it calls your MCP server. They have created some meta fields your tool can respond with, one of which is something about producing a widget along with a field for html.

In the current implementation, it makes an iframe (or webview on native) that loads a sandboxed environment which then gets another iframe with your html injected. Your html can include meta field whitelisted remote resources.

Re: Apps SDK

#187
post #180

Earlier quoted context omitted.

They don't lack affordances, you can do stuff. They lack signifiers, ie it's not easy to discover the stuff you can do.

Affordance is not what it can do, it is what it signals that it can do. It needs to be perceptible, by the definition I use (Norman's). I see others go by different definitions that even admit hidden affordances. I do not.

From The Design of Everyday Things:

> Affordances represent the possibilities in the world for how an agent (a person, animal, or machine) can interact with something. Some affordances are perceivable, others are invisible. Signifiers are signals. Some signifiers are signs, labels, and drawings placed in the world, such as the signs labeled “push,” “pull,” or “exit” on doors, or arrows and diagrams indicating what is to be acted upon or in which direction to gesture, or other instructions. Some signifiers are simply the perceived affordances, such as the handle of a door or the physical structure of a switch. Note that some perceived affordances may not be real: they may look like doors or places to push, or an impediment to entry, when in fact they are not.

With Norman's definition, if a conversational interface can perform an action, it affords that action. The fact that you don't know that it affords that action means there's a lack of a signifier.

As you say, this is a matter of definition, I'm just commenting on Norman's specific definition from the book.

Re: Apps SDK

#188
post #12

There was a recent post here about how deeply ingrained the chat interface is in OpenAIs organization. This really doubles down on that, but does anyone really like to interact with so much language instead of visual elements? Also feels horrible that you are supposed to remember a bunch of app names like "zillow" and punch them in the chat. And like an opportunity for them to slowly introduce ads for this apps or "p…

counterpoint: a lot of people around me just type "zillow" in google to access it, so maybe it's not absurd to refer to it by name in a chat interface

Re: Apps SDK

#189
Just let the AI control my mouse and keyboard, let it use my device like a human. There's a huge swath of software already designed to be used by humans and anyone who uses ChatGPT knows that it's already been trained on every scrap of knowledge on how to use any existing complex software.

Re: Apps SDK

#190

This conception makes sense iff you believe in ChatGPT as the universal user interface of the future. If anything the agentic wave is showing that the chat interfaces are better off hidden behind stricter user interface paradigms.

I suspect there are many, many things for which chat is a great interface. And by positioning ChatGPT as the distributor for all these things, they get to be the new Google. But you're also right that many domains for which a purpose-built interface is the right approach, and if the domain is valuable enough, it'll have someone coming after it to build that.

[deleted]
Post reply on HN