Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I'm betting on HTML

catskull.net

Post reply on HN
Print view View original

151–160 of 458 posts

Re: I'm betting on HTML

#151

Post by kevinstubbs »

wruza wrote: post #133
kevinstubbs wrote: post #116

Earlier quoted context omitted.

> can they really look at a DOM tree and tell what it is/does Yes, if you encode the DOM as a list of options for ChatGPT to choose from. In fact I developed a proof of concept of this for a client. https://jarvys.ai/ although they seem to have pivoted from automating just the browser to automating all software.

if you encode the DOM as a list of options for ChatGPT to choose from Not sure if I understand this, does it mean you have to pre-cook DOM in a specific way? If yes, then isn’t the answer to my question “no”, like “no, it can’t take any site and use it as is”?

You have to give GPT an objective, like "find an apartment in Florida" and then say something like "given the following options, which one would you interact with to get closer to your objective."

So if you assume that you start on google.com, then your options are like 1.) Input with name "search", placeholder "search anything", value "" 2.) Button with label "I'm feeling lucky" 3.) Button with label "search"

Obviously, doing just one of these doesn't achieve the objective - it just needs to pick which one it thinks has the most "value" for completing the objective. If you repeat that enough times, then it can actually do what your overall goal of the session was.

I'm just giving a simplistic answer, and if you implemented only what I've written, then it's going to get stuck in a loop more often than not. But that's the gist of how you could encode the DOM into something that GPT can interpret and make decisions/take actions based on.

Re: I'm betting on HTML

#152

Post by jll29 »

HTML was designed as an SGML application for networked hypertext.

As an alternative to what the original post posits, we could leave it for that purpose, and design another (now XML) application for user interfaces: windows, buttons, scroll bars (if desired), text controls (no I'm not talking about textarea for CGI), the kind of controls that Windows or X11+MOTIF provide, expressed as tags in a UIML (User Interface Markup Language). This would have the advantage that we could start from a clean slate, and the open source interpreter for this technology could be integrated into all Web browsers, so behavior would be identical.

UIML would be designed as an XML application for networked software applications' user interfaces.

Of course, you could execute them locally, too. There could be graphical UI designer of the types that already exist, e.g. Visual Studio would just write out a UIML as a new export format.

Crazy idea? Actually, it's just applying the "Do one thing, and do it well." mantra to XML XHTML + UIML instead of packing everything possible into one now-bloated markup language it was never designed to do. So if this comment had a title, it would be "I'm betting on Internet standards" (plural).

Re: I'm betting on HTML

#153

Post by wruza »

kevinstubbs wrote: post #151
wruza wrote: post #133

Earlier quoted context omitted.

if you encode the DOM as a list of options for ChatGPT to choose from Not sure if I understand this, does it mean you have to pre-cook DOM in a specific way? If yes, then isn’t the answer to my question “no”, like “no, it can’t take any site and use it as is”?

You have to give GPT an objective, like "find an apartment in Florida" and then say something like "given the following options, which one would you interact with to get closer to your objective." So if you assume that you start on google.com, then your options are like 1.) Input with name "search", placeholder "search anything", value "" 2.) Button with label "I'm feeling lucky" 3.) Button with label "search" Obviou…

Got it, thanks!

Re: I'm betting on HTML

#154

Post by abwizz »

delusional wrote: post #112
wruza wrote: post #95

Earlier quoted context omitted.

ChatGPT (and its future offspring) can navigate regular websites the same way humans do It seems to be at least a not-yet-true claim, but let’s ignore that for now. It’s interesting if LLMs actually could do this. As I understand it, LLMs are trained on texts and source code among other things. But lacking… let’s name it a reasoning apparatus, can they really look at a DOM tree and tell what it is/does? It’s not a te…

> “LLMs will look at any .exe and be able to integrate with it immediately”. The LLM will look at any .exe and determine if it halts.

i suspect this is because they can divide by zero

Re: I'm betting on HTML

#155

Post by abwizz »

p-e-w wrote: post #26

HTML is the solution to walled-garden lock-in? What? Those walled gardens already use HTML, including some of the semantic elements mentioned (plus ARIA semantic attributes, which are much more sophisticated). > ChatGPT-like interfaces are likely the future of human data access. And the whole point of artificial intelligence systems is that they don't require specialized "machine-readable" annotations in order to pro…

> They know when a sequence of paragraphs constitutes a "list", without it having to be explicitly marked as such, etc.

that's the thing; they don't know - they guess

Re: I'm betting on HTML

#156

Post by EspressoGPT »

klardotsh wrote: post #9

I had heard of almost none of these HTML elements, and that's such a shame, because they could seriously help put the "we need JavaScript for every gosh darn thing" ecosystem to an end (or at least return JS to what it was originally meant to be: a way to add some flair, some interactivity, some whatever, but not necessarily a replacement for all of your markup and a full-DOM manager). I'm starting to think my dream…

> I had heard of almost none of these HTML elements, and that's such a shame

I guess, that's on you – if you're a web developer, you definitely 100% need to know these elements. They're not new.

Re: I'm betting on HTML

#157

Post by yyyk »

Sjeiti wrote: post #80
Gigachad wrote: post #56

Earlier quoted context omitted.

In general the issue with these built in components is that you can't theme them. And they stick out like a sore thumb when you get a windows 7 style component in the middle of a modern looking app. They also have basically no extensibility so when you inevitably need to do something half complex, you have to scrap it and start again with JS. So you may as well have just started with JS which just works, gives you fu…

JS does not 'just work'. This is why a lot of these custom components have bad touch interaction and no accessibility. Take the datepicker; the native mobile version works great, why annoy users with a custom component?

>Take the [HTML] datepicker

* Lets you enter nonexistent dates like 31/2

* Can and often does accidentally place the user in American-style MM/DD format where it should be European-style DD/MM (I have a replicable case now on that page example).

* No ability to force date style by design. So there's no way to fix the above from the server, or to use ISO-style dates. Only way to reliably prevent MM/DD by default is to fix every client configuration - not very likely even in small companies.

* No way to have the datetime dialog open by default.

* Poorish but getting better keyboard support (the pagedown-up keys finally work in most browsers, but once you've opened the dialog you can't enter a new date with the keyboard).

* Timezones must be handled separately, which is just poor design.

(Entire list checked on desktop)

Re: I'm betting on HTML

#158

Post by mhoad »

troupo wrote: post #139
satvikpendem wrote: post #75

Earlier quoted context omitted.

Do you realize that Flutter has an accessibility system that is often more advanced than most desktop UI frameworks? [0] I notice this often from non-Flutter devs, Flutter has some pretty good a11y. [0] https://docs.flutter.dev/accessibility-and-localization/acce...

Good to know! For the longest time ever their export to web was just canvas with no attempt at making it accessible.

I don’t know if even that was actually true? As far as I know the moment they introduced canvas as a rendering target they also produced a DOM structure along side it to capture the accessibility tree as they were still waiting on a more native solution in the form of AOM (accessibility object model) to be finalised.

Re: I'm betting on HTML

#159

Post by rezonant »

p-e-w wrote: post #26

HTML is the solution to walled-garden lock-in? What? Those walled gardens already use HTML, including some of the semantic elements mentioned (plus ARIA semantic attributes, which are much more sophisticated). > ChatGPT-like interfaces are likely the future of human data access. And the whole point of artificial intelligence systems is that they don't require specialized "machine-readable" annotations in order to pro…

I got the sense that the article was advocating for using good semantic machine readable content over HTTP instead of (or at least in addition to) JS-only div soup so that automated agents like the new wave of LLMs can easily pull out the important details without spinning up a headless browser and rendering the page first.

I have the same interest but for the purpose of crawling and upstart search engines. If indexing every page required running the page in a headless browser first, the barrier to entry for new search engines is a lot higher.

Re: I'm betting on HTML

#160

Post by Zen1th »

jll29 wrote: post #152

HTML was designed as an SGML application for networked hypertext. As an alternative to what the original post posits, we could leave it for that purpose, and design another (now XML) application for user interfaces: windows, buttons, scroll bars (if desired), text controls (no I'm not talking about textarea for CGI), the kind of controls that Windows or X11+MOTIF provide, expressed as tags in a UIML (User Interface M…

Wouldn't that be XUL[1] ? It's been there since 1997 and never took off outside of Mozilla, so it was deprecated and removed in 2017. It wasn't meant for use in the wen directly as replacing an entire ecosystem with a completely different way of doing UIs would be close to impossible, all for small benefits.

[1]: https://wiki.mozilla.org/XUL:Home_Page

Post reply on HN