Enough AI copilots, we need AI HUDs • Hacker News Board

Re: Enough AI copilots, we need AI HUDs

#231

Post by marcosdumay » Mon, Jul 28, 2025, 7:10 PM UTC

motorest wrote: post #77

JimDabell wrote: post #29

Earlier quoted context omitted.

> When you give up the work of deciding what the expected inputs and outputs of the code/program is you are no longer in the drivers seat. You don’t need to write tests for that, you need to write acceptance criteria.

> You don’t need to write tests for that, you need to write acceptance criteria. Sir, those are called tests.

I see you have little experience with Scrum...

Acceptance criteria is a human-readable text that the person specifying the software has to write to fill-up a field in Scrum tools and not at all guide the work of the developers.

It's usually derived from the description by an algorithm (that the person writing it has to run on their mind), and any deviation from that algorithm should make the person edit the description instead to make the deviation go away.

Re: Enough AI copilots, we need AI HUDs

#232

Post by jpm_sd » Mon, Jul 28, 2025, 7:38 PM UTC

skydhash wrote: post #52

jpm_sd wrote: post #15

This is how ship's AI is depicted in The Expanse (TV series) and I think it's really compelling. Quiet and unobtrusive, but Alex can ask the Rocinante to plot a new course or display the tactical situation and it's fast, effective and effortlessly superhuman with no back-talk or unnecessary personality. Compare another sci-fi depiction taken to the opposite extreme: Sirius Cybernetics products in the Hitchhikers Guid…

I may remember wrongly, but I don't believe the expanse was depicting AI. It was more powerful computation and unobtrusive interfaces. There was nothing like Jarvis. Rocinante was a war vessel and all its feature was tailored to that. I believe even the mechanical suits of the Martians were very manual (no cortana a la Master Chief).

That's exactly what this article is about, though. Assistive systems that are incredibly capable without needing to have a fake personality.

Re: Enough AI copilots, we need AI HUDs

#233

Post by motorest » Mon, Jul 28, 2025, 7:49 PM UTC

marcosdumay wrote: post #231

motorest wrote: post #77

Earlier quoted context omitted.

> You don’t need to write tests for that, you need to write acceptance criteria. Sir, those are called tests.

I see you have little experience with Scrum... Acceptance criteria is a human-readable text that the person specifying the software has to write to fill-up a field in Scrum tools and not at all guide the work of the developers. It's usually derived from the description by an algorithm (that the person writing it has to run on their mind), and any deviation from that algorithm should make the person edit the descripti…

> Acceptance criteria is a human-readable text that the person specifying the software has to write (...)

You're not familiar with automated testing or BDD, are you?

> (...) to fill-up a field in Scrum tools (..)

It seems you are confusing test management software used to tracks manual tests with actual acceptance tests.

This sort of confusion would be ok 20 years ago, but it has since went the way of the dodo.

Re: Enough AI copilots, we need AI HUDs

#234

Post by Garlef » Mon, Jul 28, 2025, 8:55 PM UTC

_jab wrote: post #218

There’s a lot of ideation for coding HUDs in the comments, but ironically I think the core feature of most coding copilots is already best described as a HUD: tab completion. And interestingly, that is indeed the feature I find most compelling from Cursor. I particularly love when I’m doing a small refactor, like changing a naming convention for a few variables, and after I make the first edit manually Cursor will ju…

I think there's a lot of room for even better UI.

I'd love to have something that operates more at the codebase level. Autocomplete is very local.

(Maybe "tab completion" when setting up a new package in a monorepo? Or make architectural patterns consistent across a whole project? Highlight areas in the codebase where the tests are weak? Or collect on the fly a full view of a path from FE to BE to DB?)

Re: Enough AI copilots, we need AI HUDs

#235

Post by reneherse » Mon, Jul 28, 2025, 9:14 PM UTC

teoremma wrote: post #111

furyofantares wrote: post #68

I'm very curious if a toggle would be useful that would display a heatmap of a source file showing how surprising each token is to the model. Red tokens are more likely to be errors, bad names, or wrong comments.

We explored this exact idea in our recent paper https://arxiv.org/abs/2505.22906 Turns out this kind of UI is not only useful to spot bugs, but also allows users to discover implementation choices and design decisions that are obscured by traditional assistant interfaces. Very exciting research direction!

Very exciting indeed. I will definitely do a deep dive into this paper, as my current work is exploring layers of affordances such as these in workflows beyond coding.

Re: Enough AI copilots, we need AI HUDs

#236

Post by ThunderSizzle » Mon, Jul 28, 2025, 10:12 PM UTC

motorest wrote: post #170

JimDabell wrote: post #113

Earlier quoted context omitted.

The line you wrote does not describe a feature. Typically you have many of those cases and they collectively describe one feature. I’m talking about describing the feature. Do you seriously think there is no higher level than given/when/thens?

> The line you wrote does not describe a feature. I'm describing a scenario as implemented in a gherkin feature file. A feature is tracked by one or more scenarios. https://cucumber.io/docs/gherkin/reference/ > Do you seriously think there is no higher level than given/when/thens? You tell me which higher level you have in mind.

I'm curious what it could possibly be too. I guess he's trying to say the comments you might make at the top of a feature file to describe a feature would be his goal, but I'm not aware of a structured way to do that.

The problem is that tests are for the unhappy path just as much as the happy path, and unhappy paths tend to get particular and detailed, which means even in gherkin it can get cumbersome.

If AI is to handle production code, the unhappy paths need to at least be certain, even if repetitive.

Re: Enough AI copilots, we need AI HUDs

#237

Post by pests » Mon, Jul 28, 2025, 10:52 PM UTC

cleverwebble wrote: post #222

irthomasthomas wrote: post #179

Earlier quoted context omitted.

import openai, math, os, textwrap, json, sys query = 'Paris is the capital of' # short demo input os.environ['OPENAI_API_KEY'] # check key early client = openai.OpenAI() resp = client.chat.completions.create( model='gpt-3.5-turbo', messages=[{'role': 'user', 'content': query}], max_tokens=12, logprobs=True, top_logprobs=1 ) logprobs = [t.logprob for t in resp.choices[0].logprobs.content] perplexity = math.exp(-sum(lo…

If you want to generate a heatmap of existing text, you will have to take a different approach here. The naive solution I could come up with would be really expensive with openai, but if you have an open source model, you can write up custom inference that goes one-token-at-a-time through the text, and on each token you look up the difference in logprobs between the token that the LLM predicted vs what was actually t…

> so logprobs of tokens after the bug happened might only be slightly higher than normal.

Sounds like it’s easier to pinpoint the bug.

Re: Enough AI copilots, we need AI HUDs

#238

Post by borgel » Mon, Jul 28, 2025, 11:03 PM UTC

federiconafria wrote: post #135

sothatsit wrote: post #33

AI building complex visualisations for you on-the-fly seems like a great use-case. For example, if you are debugging memory leaks in a specific code path, you could get AI to write a visualisation of all the memory allocations and frees under that code path to help you identify the problem. This opens up an interesting new direction where building visualisations to debug specific problems is probably becoming viable.…

Independently of visualizations, I think LLMs allow in general the creation of ad-hoc tools. I've experienced the case where asking for a quick python script was faster and more powerful than learning how to use a cli to interact with an API.

Same. It's very handy to be able to ask for one-off tools for things that I _could_ try to figure out, but probably wouldn't be worth the time. My favorite example so far was a Python script to help debug communication between a microcontroller and a motor driver. I was able to dump the entire datasheet PDF for the driver into Gemini and ask it for a Python CLI to decode communication traffic, and 30 seconds later it was done. Fantastic bang/buck ratio!

Re: Enough AI copilots, we need AI HUDs

#239

Post by mikestorrent » Mon, Jul 28, 2025, 11:03 PM UTC

imglorp wrote: post #219

beezlebroxxxxxx wrote: post #191

Earlier quoted context omitted.

There already are debates about "drive by wire" in racing today, I can't imagine how bad it'll be when it's "drive by copilot."

Rally co-driver/navigator seems reasonable, given its augmented senses like gps and radar. One part "90 degree right in 200m" and one part "OMG, sheep, dodge left".

It will be fun when we get to the era where this is bulletproof-reliable to this extent, but for now, recorded notes with accelerometer-backed GPS locations are probably a better idea than hoping the AI will be able to do this for you dynamically without you driving off a cliff.

Re: Enough AI copilots, we need AI HUDs

#240

Post by Einenlum » Mon, Jul 28, 2025, 11:06 PM UTC

Makes me think that I would love to have an AI tool that not only allows autocomplete but also provides suggestions while I write code. "Have you thought about using useCallback here?" "This feature seems already provided by library X and Y" "It seems like this code already exists in function Foo. Maybe a small refactoring would be helpful"