Nobody mentioned Manna [1] yet? That suggests a mostly audio headset giving orders. There is a real-world version using AR glasses.[2]
Enough AI copilots, we need AI HUDs
91–100 of 290 posts
Re: Enough AI copilots, we need AI HUDs
#92Earlier quoted context omitted.
> There's no way this would work for any serious C++ codebase. Compile times alone make this impossible There's nothing in C++ that prevents this. If build times are your bogeyman, you'd be pleased to know that all mainstream build systems support incremental builds.
The original example was (paraphrasing) "rerunning 10-100 tests that take 1ms after each keystroke". Even with incremental builds, that surely does not sound plausible? I only mentioned C++ because that's my main working language, but this wouldn't sound reasonable for Rust either, no?
> The original example was (paraphrasing) "rerunning 10-100 tests that take 1ms after each keystroke".
Yeah, OP's point is completely unrealistic and doesn't reflect real-world experience. This sort of test watchers is mundane in any project involving JavaScript, and not even those tests re-run at each keystroke. Watch mode triggers tests when they detect changes, and waits for test executions to finish to re-run tests.
This feature consists of running a small command line app that is designed to run a command whenever specific files within a project tree are touched. There is zero requirement to only watch for JavaScript files or only trigger npm build when a file changes.
To be very clear, this means that right now anyone at all, including you and me, can install a watcher, configure it to run make test/cutest/etc when any file in your project is touched, and call it a day. This is a 5 minute job.
By the way, nowadays even Microsoft's dotnet tool supports watch mode, which means there's out-of-the-box support to "rerunning 10-100 tests that take 1ms after each keystroke".
Re: Enough AI copilots, we need AI HUDs
#93You know that feature in JetBrains (and possibly other) IDEs that highlights non-errors, like code that could be optimized for speed or readability (inverting ifs, using LINQ instead of a foreach, and so on)? As far as I can tell, these are just heuristics, and it feels like the perfect place for an “AI HUD.”
I don’t use Copilot or other coding AIs directly in the IDE because, most of the time, they just get in the way. I mainly use ChatGPT as a more powerful search engine, and this feels like exactly the kind of IDE integration that would fit well with my workflow.
Re: Enough AI copilots, we need AI HUDs
#94I want a link to the GitHub for this[0] which he linked to. Makes Prolog quite interesting. [0]: https://www.geoffreylitt.com/2024/12/22/making-programming-m...
Yeah, it looks interesting. I would give it a spin. Maybe then I could finally make a MUD in Prolog. :D
Re: Enough AI copilots, we need AI HUDs
#95> design the cockpit so that the human pilot is naturally aware of their surroundings.
This is a design interface problem. Self-driving cars can easily ingest this HUD. This is the reason what makes Apple's AI different from other microservice-like AI. The spell checker, rewrite, proofread are naturally integrated into the UI to the extent it doesn't feel like AI powered operations.
Re: Enough AI copilots, we need AI HUDs
#96This is what we are going for in Closer - an AI native experience that extends the mind of salespeople. https://closer.so
Re: Enough AI copilots, we need AI HUDs
#97Earlier quoted context omitted.
> Thought experiment: as you write code, an LLM generates tests for it & the IDE runs those tests as you type, showing which ones are passing & failing, updating in real time. Imagine 10-100 tests that take I think this is a bad approach. Tests enforce invariants, and they are exactly the type of code we don't want LLMs to touch willy-nilly. You want your tests to only change if you explicitly want them to, and even…
I think tests should be rewritten as much as needed. But to counter the invariant part, maybe let the user zoom back and forth through past revisions and pull in whatever they want to the current version, in case something important is deleted? And then allow “pinning” of some stuff so it can’t be changed? Would that solve for your concerns?
> I think tests should be rewritten as much as needed.
Yes, I agree. The nuance is that they need to be rewritten independently and without touching the code. You can't change both and expect to get a working system.
I'm speaking based on personal experience, by the way. Today's LLMs don't enforce correctness out of the box and agent mode has only one goal: getting things to work. I had agent mode flip invariants in tests when trying to fix unit tests it broke, and I'm talking about egregious changes such as flipping requirements in line with "normal users should not have access to the admin panel" to "normal users should have access to the admin panel". The worst part is that if agent mode is left unsupervised, it will even adjust the CSS to make sure normal users have a seamless experience going through the admin panel.
Re: Enough AI copilots, we need AI HUDs
#98Earlier quoted context omitted.
All of this at the end reduces to a simple fact at the end of the discussion. You need some of way of precisely telling AI what to do. As it turns out there is only that much you can do with text. Come to think of it, you can write a whole book about a scenery, and yet 100 people will imagine it quite differently. And still that actual photograph would be totally different compared to the imagination of all those 100…
> You need some of way of precisely telling AI what to do. I think this is the detail you are not getting quite right. The truth of the matter is that you don't need precision to get acceptable results, at least in 100% of the cases. As everything in software engineering, there is indeed "good enough". Also worth noting, LLMs allow anyone to improve upon "good enough". > As it turns out if you wish to describe someth…
Yes, you are right. But in the sense that a human decides if AI generated code is right.
But if you want a near 100% automation, you need precise way to specify what you want, else there is no reliable way interpreting what you mean. And by that definition lots of regression/breakage has to be endured everytime a release is made.
Re: Enough AI copilots, we need AI HUDs
#99[dead]
Re: Enough AI copilots, we need AI HUDs
#100Earlier quoted context omitted.
> When you give up the work of deciding what the expected inputs and outputs of the code/program is you are no longer in the drivers seat. You're describing the happy path of BDD-style testing frameworks.
I know about BDD frameworks. I’m talking higher level than that.
> I know about BDD frameworks. I’m talking higher level than that.
What level do you think there is above "Given I'm logged in as a Regular User When I go to the front page Then I see the Profile button"?