Enough AI copilots, we need AI HUDs • Hacker News Board

Re: Enough AI copilots, we need AI HUDs

#101

Post by Oras » Mon, Jul 28, 2025, 6:54 AM UTC

Great post, informative and precise.

I think the challenge is primarily the context and intent.

The spellchecker knows my context easily, and there is a setting to choose from (American English, British English, etc.), as well as the paragraphs I'm writing. The intent is easy to recognise. While in a codebase, the context is longer and vaguer, the assistant would hardly know why I'm changing a function and how that impacts the rest of the codebase.

However, as the article mentions, it may not be a universal solution, but it's a perspective to consider when designing AI systems.

Re: Enough AI copilots, we need AI HUDs

#102

Post by thinkingemote » Mon, Jul 28, 2025, 6:55 AM UTC

We also need what goes along with HUDs : switches, nubs, switches, dials. Actual controls.

Although we are talking HUDs, I'm not really talking about UI widgets having the good old skew-morphism or better buttons. In the cockpit the pilot doesn't have his controls on a touch screen, he has an array of buttons and dials and switches all around him. It's these controls that are used in response to what the pilot sees on the HUD and it's these controls that change the aircraft according to the pilots will, which in turn change what the HUD shows.

Re: Enough AI copilots, we need AI HUDs

#103

Post by TimCTRL » Mon, Jul 28, 2025, 7:02 AM UTC

makes sense, i guess tools like cluely are HUDs

Re: Enough AI copilots, we need AI HUDs

#104

Post by _samjarman » Mon, Jul 28, 2025, 7:04 AM UTC

I'd love to mount a top-down webcam above me when I work on electronics and have a real time feed into ChatGPT or similar for Q&A, for a real time coach that can see. One day... :)

Re: Enough AI copilots, we need AI HUDs

#105

Post by precompute » Mon, Jul 28, 2025, 7:10 AM UTC

HUDs are primarily of use to people that are used to parsing dense visual information. Wanting HUDs on a platform that promises to do your work for you is quite pointless.

Re: Enough AI copilots, we need AI HUDs

#106

Post by precompute » Mon, Jul 28, 2025, 7:11 AM UTC

aaron695 wrote: post #14

This is a mess. The 1992 talk wasn't at all about AI and since then our phones have given us "ubiquitous computing" en masse. The original talk required no 'artificial intelligence' for relevance which makes it strange to apply to todays artificial intelligence. The original talk made good points for instance "voice recognition" has been solved forever at a reasonable level, yet people kept claiming if it was 'better…

FYI, you're shadowbanned.

Re: Enough AI copilots, we need AI HUDs

#107

Post by afro88 » Mon, Jul 28, 2025, 7:11 AM UTC

A HUD is an even more "confident" display of data than text though. What do you do with a HUD that hallucinates? Is there a button on each element that shows you sources?

Re: Enough AI copilots, we need AI HUDs

#108

Post by Dilettante_ » Mon, Jul 28, 2025, 7:16 AM UTC

afro88 wrote: post #107

A HUD is an even more "confident" display of data than text though. What do you do with a HUD that hallucinates? Is there a button on each element that shows you sources?

I could imagine a system in which the model only chooses which data points to show at what time, but the actual passing is still handled by good old deterministic programming.

Re: Enough AI copilots, we need AI HUDs

#109

Post by darkwater » Mon, Jul 28, 2025, 7:21 AM UTC

energy123 wrote: post #53

hi_hi wrote: post #8

Doesn't it all come down to "what is the ideal interface for humans to deal with digital information"? We're getting more and more information thrown at us each day, and the AIs are adding to that, not reducing it. The ability to summarise dense and specialist information (I'm thinking error logs, but could be anything really) just means more ways for people to access and view that information who previously wouldn't…

The designers of 6th gen fighter jets are confronting the same challenge. The cockpit, which is an interface between the pilot and the airframe, will be optionally manned. If the cockpit is manned, the pilot will take on a reduced set of roles focused on higher-level decision making. By the 7th generation it's hard to see how humans will still be value-add, unless it's for international law reasons to keep a human in…

Is this just what you think it might happen or are you directly involved in these decisions and first-hand exposing a challenge?

Re: Enough AI copilots, we need AI HUDs

#110

Post by darkwater » Mon, Jul 28, 2025, 7:25 AM UTC

bigyabai wrote: post #62

ravila4 wrote: post #60

Earlier quoted context omitted.

It’s a similar idea, but imagine you could fire off a task, and go for a run, or do the dishes. Then be notified when it completes, and have the option to review the changes, or see a summary of tests that are failing, without having to be at your workstation.

I kinda do this today, with Alpaca[0]'s sandboxed terminal runner and GSConnect[1] syncing the response notifications to my phone over LAN. [0] https://jeffser.com/alpaca/ [1] https://github.com/GSConnect/gnome-shell-extension-gsconnect

And, out of curiosity, what are the outputs of this agentic work?