Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Enough AI copilots, we need AI HUDs

geoffreylitt.com

Post reply on HN
Print view View original

281–290 of 290 posts

Re: Enough AI copilots, we need AI HUDs

#281

Post by digdugdirk »

tionis wrote: post #253
furyofantares wrote: post #68

I'm very curious if a toggle would be useful that would display a heatmap of a source file showing how surprising each token is to the model. Red tokens are more likely to be errors, bad names, or wrong comments.

That's actually something I implemented for a university project a few weeks ago. My professor also did some research into how this can be used for more advanced UIs. I'm sure it's a very common idea.

Do you have a link to the code? I'm curious how you implemented it. I'd also be really intrigued to see that research - does your professor have any published papers or something for those UIs?

Re: Enough AI copilots, we need AI HUDs

#282

Post by nojs »

furyofantares wrote: post #68

I'm very curious if a toggle would be useful that would display a heatmap of a source file showing how surprising each token is to the model. Red tokens are more likely to be errors, bad names, or wrong comments.

There is some argument to be made here about entropy, compression, and how if there are no surprises, the program communicates no new information.

Interestingly, frequency of "surprising" sentences is one of the ways quality of AI novels is judged: https://arxiv.org/abs/2411.02316

Re: Enough AI copilots, we need AI HUDs

#283

Post by yencabulator »

stavros wrote: post #255
trainsarebetter wrote: post #241

Earlier quoted context omitted.

This is I’d say partially because we don’t have a HUD that can handle the bandwidth and pace of data required for rally. Overlaying a visualization of the turns ahead would be much better than a copilot for sure

If we have HUDs that can handle aerial combat, why don't we have HUDs that can handle racing?

Budget? It's a better trade-off to have a second human and to put the money into tuning the car.

Re: Enough AI copilots, we need AI HUDs

#285

Post by ColonelPhantom »

throwaway12345t wrote: post #242

This is now (more) dated. Copilots as an interface are dated. The current initiative is full agents with human in the loop at the very start, occasionally the middle and the end. Huds are just good UI, something copilots can natively exist as part of in the form of contextual insights and alerts. We’re moving on to agency where it’s everything else vs an entirely different entity taking the action of flying the plane…

I don't think you understood what a "copilot" and a "HUD" is. Confusingly, GitHub Copilot (the original one that suggests completions) is basically a HUD. On the other hand, agents that you give a task are clearly copilots that work via a natural language interface.

The article also mentions that agents are copilots:

> Here’s another personal example from AI coding. Let’s say you want to fix a bug. The obvious “copilot” way is to open an agent chat and ask it to do the fix.

Re: Enough AI copilots, we need AI HUDs

#286

Post by timeinput »

itslennysfault wrote: post #226
timeinput wrote: post #223

There is a giant confusing tech stack here, but opening up this simple plain text page resulted in like 100% CPU usage from from gnome-shell when visiting from firefox on a pretty non-customized Ubuntu 24 install. In the firefox task manager nothing really looked odd, but opening that tab and displaying it is insanely CPU intensive. Pausing the autoplaying video makes it seem like a sane web page in terms of CPU usag…

Very odd. This must be some kind of Ubuntu issue. Out of curiosity I tried it in Chrome, Firefox and Safari on MacOS 15.5 and saw very little CPU usage and no difference with/without the video playing. I don't have a Ubuntu Desktop handy to confirm. Looking at the video it's nothing special. Just a fairly small mp4 using a native html5 video element to play it. Really no reason this should be causing issues.

Yeah, totally agreed. It was quite surprising to me. Who knows it may even be related to the random GPU I happen to have or ....

Re: Enough AI copilots, we need AI HUDs

#287

Post by 8n4vidtmkvmk »

geon wrote: post #151
furyofantares wrote: post #68

I'm very curious if a toggle would be useful that would display a heatmap of a source file showing how surprising each token is to the model. Red tokens are more likely to be errors, bad names, or wrong comments.

Sounds great. I'd like to see more contextually meaningful refactoring tools. Like "Remove this dependency" or "Externalize this code with a callback". And refactoring shouldn't be done by generatively rewriting the code, but as a series of guaranteed equivalent transformations of the AST, each of which should be committed separately. The AI should be used to analyse the value of the transformation and filter out asi…

If it can't guarantee exact equivalence, it should at least rerun the relevant tests in the background and make sure they still pass before making suggestions.

Re: Enough AI copilots, we need AI HUDs

#288

Post by geon »

8n4vidtmkvmk wrote: post #287
geon wrote: post #151

Earlier quoted context omitted.

Sounds great. I'd like to see more contextually meaningful refactoring tools. Like "Remove this dependency" or "Externalize this code with a callback". And refactoring shouldn't be done by generatively rewriting the code, but as a series of guaranteed equivalent transformations of the AST, each of which should be committed separately. The AI should be used to analyse the value of the transformation and filter out asi…

If it can't guarantee exact equivalence, it should at least rerun the relevant tests in the background and make sure they still pass before making suggestions.

It would be awesome if an ai could speculatively refactor sections of code to an exact equivalent where inconsistency stands out more. That way you could find bugs and make actual features more understandable.

Re: Enough AI copilots, we need AI HUDs

#289

Post by betterhealth12 »

soco wrote: post #208
betterhealth12 wrote: post #207

How about contextual AI insights as you're interacting with the world, wherever you are? Like a quality signal. Oh wait, this is something FB/Goog could never do because what would it say about the quality of information hosted...

AR for the regular guy is something which I remember installing on my mobile over ten years ago - with low expectations, and it never got any better than that, quite the contrary. Powered by AI or not, seems nobody has found (or looked for) a market for this.

Another decade, another attempt? We are aiming for this with our work at Credible (https://mycredible.ai and our chrome extension) - product is still early, I'd be curious to get your first thoughts or reaction.

Re: Enough AI copilots, we need AI HUDs

#290

Post by jachee »

brookst wrote: post #271
jachee wrote: post #247

Earlier quoted context omitted.

Other way around. The masses aren’t interested in force-multiplying tools. They only want to buy force-eliminating tools. They don’t want to work smarter or harder. They don’t want to work at all.

A fairly misanthropic view that hasn’t born out in my experience.

Counterpoint, these are actually the two most-frequently used functions of non-programmers:

https://bsky.app/profile/eddelbuettel.com/post/3lvjs26kruc2g

Post reply on HN