Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Enough AI copilots, we need AI HUDs

geoffreylitt.com

Post reply on HN
Print view View original

201–210 of 290 posts

Re: Enough AI copilots, we need AI HUDs

#201

Post by CharlieDigital »

juped wrote: post #175
gklitt wrote: post #6

Earlier quoted context omitted.

Author here. Yes, I think the original GitHub Copilot autocomplete UI is (ironically) a good example of a HUD! Tab autocomplete just becomes part of your mental flow. Recent coding interfaces are all trending towards chat agents though. It’s interesting to consider what a “tab autocomplete” UI for coding might look like at a higher level of abstraction, letting you mold code in a direct-feeling way without being bogg…

If that's what you think a HUD is, then a HUD is definitely way, way worse. Rather than a copilot sitting next to you, that's someone grabbing your hands and doing things with them while you're at the controls. But if I invoke the death of the author and pretend HUD meant HUD, then it's a good point: tools are things you can form a cybernetic system with, classic examples being things like hand tools or cars, and you…

I take "HUD" here to just mean "in your line of vision" or "in the context of your actual task" or minimizing any context switch to another interaction (chat window).

Rather I think most implementations of HUD AI interactions so far have been quite poor because the interaction model itself is perhaps immature and no one has quite hit the sweet spot yet (that I know of). Tab autocompletion is a simple gesture, but trades off too much control for more complex scenarios and is too easy to accidentally activate. Inline chat is still a context switch and also not quite right.

Re: Enough AI copilots, we need AI HUDs

#203

Post by [deleted] »

kevinphy wrote: post #145

This whole Copilot vs HUD debate instantly brought to mind a classic Japanese anime from 1991 called Future GPX Cyber Formula ( https://en.wikipedia.org/wiki/Future_GPX_Cyber_Formula ). Yeah, it’s a racing anime set in the then-distant future of 2015, where cars come with full-on intelligent AIs. The main character’s car, Asurada, is basically a "Copilot" in every sense. It was designed by his dad to be more than jus…

[deleted]

Re: Enough AI copilots, we need AI HUDs

#204

Post by kibwen »

piker wrote: post #90

Absolutely agree, and spellchecker is a great analogy. I've recently been snoozing co-pilot for hours at a time in VS Code because it’s adding a ton of latency to my keystrokes. Instead, it turns out that `rust_analyzer` is actually all that I need. Go-to definition and hover-over give me exactly what the article describes: extra senses. Rust is straightforward, but the tricky part may be figuring out what additional…

On the topic of Rust IDE plugins that give you more senses, take a look at Flowistry: https://github.com/willcrichton/flowistry . It's not AI, it's using information flow analysis.

Re: Enough AI copilots, we need AI HUDs

#205

Post by javcasas »

JimDabell wrote: post #66
skydhash wrote: post #54

Earlier quoted context omitted.

I think your perspective is heavily influenced by the imperative paradigm where you actually write the state transition. Compare that to functional programming where you only describe the relation between the initial and final state. Or logic programming where you describe the properties of the final state and where it would find the elements with those properties in the initial state. Those does not involves writing…

I didn’t mention state transitions. When I said “mechanically iterate over every possible state”, I was referring to writing tests that cover every type of input and output. Acceptance criteria might be something like “the user can enter their email address”. Tests might cover what happens when the user enters an email address, what happens when the user tries to enter the empty string, what happens when the user tri…

> "the user can enter their email address”

That only defines one of the things the user can enter. Should they be allowed to enter their postal address? Maybe. Should they be allowed to enter their friend's email address? Maybe.

Your acceptance criteria is too shy of details.

Re: Enough AI copilots, we need AI HUDs

#206

Post by furyofantares »

WithinReason wrote: post #139
Too wrote: post #138

Earlier quoted context omitted.

All editors do this already.

No, I mean when an LLM encounters a previously unseen name it doesn't expect it so it would be red, even though it's perfectly valid.

I'm imagining everything the LLM could produce (with a given top_k setting) would be shades of green to yellow. Just outside of that orange, and far outside red.

LLMs generate new functions all the time, I'd guess these would be light green, maybe the first token in the name would be yellow and it would get brighter green as the name unfolds.

The logits are probably all small when in the global scope where it's not clear what will be defined next. I'm not imagining mapping logits directly to heat, the ordering of tokens seems much more appropriate.

Re: Enough AI copilots, we need AI HUDs

#208

Post by soco »

betterhealth12 wrote: post #207

How about contextual AI insights as you're interacting with the world, wherever you are? Like a quality signal. Oh wait, this is something FB/Goog could never do because what would it say about the quality of information hosted...

AR for the regular guy is something which I remember installing on my mobile over ten years ago - with low expectations, and it never got any better than that, quite the contrary. Powered by AI or not, seems nobody has found (or looked for) a market for this.

Re: Enough AI copilots, we need AI HUDs

#209

Post by twbarber »

federiconafria wrote: post #135
sothatsit wrote: post #33

AI building complex visualisations for you on-the-fly seems like a great use-case. For example, if you are debugging memory leaks in a specific code path, you could get AI to write a visualisation of all the memory allocations and frees under that code path to help you identify the problem. This opens up an interesting new direction where building visualisations to debug specific problems is probably becoming viable.…

Independently of visualizations, I think LLMs allow in general the creation of ad-hoc tools. I've experienced the case where asking for a quick python script was faster and more powerful than learning how to use a cli to interact with an API.

Anecdotally, the low time to prototype has had me throwing usable tool projects up on my local network. I wrote a meal planner and review tool for our weekly vegetable share, and local board game / book library that guests can connect to and reference.

It scratches the itch to build and ship with the benefit of a growing library of low scope, high performance, highly customized web tools I can write over a few hours in an evening instead of devoting weekends to it. It feels like switching from hand to power tools

Re: Enough AI copilots, we need AI HUDs

#210

Post by davedx »

kevinphy wrote: post #145

This whole Copilot vs HUD debate instantly brought to mind a classic Japanese anime from 1991 called Future GPX Cyber Formula ( https://en.wikipedia.org/wiki/Future_GPX_Cyber_Formula ). Yeah, it’s a racing anime set in the then-distant future of 2015, where cars come with full-on intelligent AIs. The main character’s car, Asurada, is basically a "Copilot" in every sense. It was designed by his dad to be more than jus…

They were discussing human augmentation back at the dawn of computing. Read “The Dream Machine”
Post reply on HN