Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Enough AI copilots, we need AI HUDs

geoffreylitt.com

Post reply on HN
Print view View original

131–140 of 290 posts

Re: Enough AI copilots, we need AI HUDs

#131

Post by [deleted] »

Kichererbsen wrote: post #85
nextaccountic wrote: post #81

Earlier quoted context omitted.

Even if something is surprising just because it's a novel algorithm, it warrants better documentation - but commenting the code explaining how it works will make the code itself less surprising! In short, it's probably possible (and it's maybe a good engineering practice) to structure the source such as no specific part is really surprising It reminds me how LLMs finally made people to care about having good document…

I often find myself leaving review comments on pull requests where I was surprised. I'll state as much: This surprised me - I was expecting XYZ at this point. Or I wasn't expecting X to be in charge of Y.

[deleted]

Re: Enough AI copilots, we need AI HUDs

#132

Post by smolder »

smolder wrote: post #125
furyofantares wrote: post #68

I'm very curious if a toggle would be useful that would display a heatmap of a source file showing how surprising each token is to the model. Red tokens are more likely to be errors, bad names, or wrong comments.

Nope, not surprising. Parent changed their text but they are just as wrong.

[flagged]

Re: Enough AI copilots, we need AI HUDs

#133

Post by federiconafria »

cadamsdotcom wrote: post #4

Love the idea & spitballing ways to generalize to coding.. Thought experiment: as you write code, an LLM generates tests for it & the IDE runs those tests as you type, showing which ones are passing & failing, updating in real time. Imagine 10-100 tests that take The tests could appear in a separated panel next to your code, and pass/fail status in the gutter of that panel. As simple as red and green dots for tests t…

That's already part of most IDE's and they know which tests to re-run, because of coverage, so it's really fast.

It also updates the coverage on the fly, you don't even have to look at the test output to know that you've broken something since the tests are not reaching your lines.

https://gavindraper.com/2020/05/27/VS-Code-Continious-Testin...

Re: Enough AI copilots, we need AI HUDs

#134

Post by anaisbetts »

I mean, fundamentally the reason that we don't build most AI HUD ideas right now is that they are either stupidly insanely expensive at scale, or you have to run the AI on-device on an NPU so underpowered that it's going to give you really really bad results.

Lots of great ideas in this space but it's tough to make something that delivers value and also is economically viable

Re: Enough AI copilots, we need AI HUDs

#135

Post by federiconafria »

sothatsit wrote: post #33

AI building complex visualisations for you on-the-fly seems like a great use-case. For example, if you are debugging memory leaks in a specific code path, you could get AI to write a visualisation of all the memory allocations and frees under that code path to help you identify the problem. This opens up an interesting new direction where building visualisations to debug specific problems is probably becoming viable.…

Independently of visualizations, I think LLMs allow in general the creation of ad-hoc tools.

I've experienced the case where asking for a quick python script was faster and more powerful than learning how to use a cli to interact with an API.

Re: Enough AI copilots, we need AI HUDs

#138

Post by Too »

WithinReason wrote: post #74
furyofantares wrote: post #68

I'm very curious if a toggle would be useful that would display a heatmap of a source file showing how surprising each token is to the model. Red tokens are more likely to be errors, bad names, or wrong comments.

previously undefined variable and function names would be red as well

All editors do this already.

Re: Enough AI copilots, we need AI HUDs

#139

Post by WithinReason »

Too wrote: post #138
WithinReason wrote: post #74

Earlier quoted context omitted.

previously undefined variable and function names would be red as well

All editors do this already.

No, I mean when an LLM encounters a previously unseen name it doesn't expect it so it would be red, even though it's perfectly valid.

Re: Enough AI copilots, we need AI HUDs

#140

Post by vidarh »

William_BB wrote: post #80
motorest wrote: post #76

Earlier quoted context omitted.

> There's no way this would work for any serious C++ codebase. Compile times alone make this impossible There's nothing in C++ that prevents this. If build times are your bogeyman, you'd be pleased to know that all mainstream build systems support incremental builds.

The original example was (paraphrasing) "rerunning 10-100 tests that take 1ms after each keystroke". Even with incremental builds, that surely does not sound plausible? I only mentioned C++ because that's my main working language, but this wouldn't sound reasonable for Rust either, no?

Some languages make this harder than others, and languages that require expensive compilation step will certainly make it hard, while e.g. interpreted languages that allows dynamic reloading of code can potentially make it easy - allowing preloading of the tests and reloading of the modified code.

If you also don't expect necessarily running the entire test suite, but just a subset of tests that are, say, labelled to test a specific function only without expensive setup, it'd potentially be viable.

You can also ignore running it on every keypress with some extra work:

- Keypresses that don't change the token sequence (e.g. because you're editing a comment) does not require re-running any tests. - Keypresses that results in a syntactically invalid file does not require re-running any tests, just marking the error.

I think it'd be an interesting experiment to have editing rather than file save trigger a test-suite watcher. My own editor syncronises the file state to a server process that other processes can observe, so if I wanted to I could wire a watcher up to re-tokenize an edited line and trigger the test suite (the caveat being I'd need to deal with the file state not being on the file system) when the state changes instead of just on save. It already retokenizes the line for syntax highlighting anyway.

Post reply on HN