Ask HN: Is anyone experimenting with different ways of using LLMs for coding?

My company tried to build something like this pre-TUI as a tool-AI-IO dag dispatcher. The biggest mistake I made was thinking that people would have no problem figuring out how they could translate their work or define multi-step automations, and focusing on the orchestration and sandboxing thinking that was the core, when it was really figuring out how to get the onboarding UX/complexity to not feel daunting or more trouble than it was worth.

Eventually for my own work, I discovered that the context management and runtime was more like a stream or active service mesh than a dispatching / one-off processing problem, most others' were too. Then all my prompts would degrade across model versions or providers, and I realized that actually setting the context for the tasks and keeping track of it all was a ton of work and something I had to do everytime as an actual user, but never when I was testing or demoing it on existing data.

Curious how you're testing your work and if you've managed to avoid the problems I ran into. I need to permute across the same set of workloads/configs you mention (and maybe more) for my next set of work so I'd be very interested in sharing or collaborating on the test infrastructure! At Google I did a lot of permutation testing using https://github.com/cloudprober/cloudprober and was going to start using it sometime in the next couple weeks. It exists basically one layer above the workload content/targets so it's probably compatible with everything except the test client/driver you're using.

[flagged]

I, like many nerds of the same stripe, have a dragon's hoard of every PC component I've owned in the last 20 years. I've attached as much of it to my homelab as is practical, but there's still a pile of GPUs from the last decade plus.

So I decided to load up everything with more then 3GB of VRAM into various machines on the network. Anything that could conceivably run an LLM of any utility. I've been experimenting with driving a swarm of heterogenous LLMs into coding tasks. I have models as small as llama3.2:3b up to Qwen3.6:27b dense. Over 10 unique models in the swarm.

So far, the results are... interesting. Coding isn't great, but what has worked shockingly well is polling the swarm for opinions. Getting ten unique perspectives synthesized into a single summary has been astonishingly useful. When I gave the swarm the ability to debate with itself, the results got even more interesting.

The end goal here is an autonomous routing network that learns which models excel at which tasks, which machines can fit which models, and intelligently routes requests and models to where they're most effective.

I can't afford an RTX 6000, but I can run smaller models on the pile of GPUs I do have. So far it hasn't worked out the way I'd hoped, but it did turn out to be very useful in other ways. Hopefully soon I can get coding worked out and the swarm can drive itself into self-improvement

Doing similar stuff. I curse a lot. The processes do transfer mostly between different models which a bit validates the approach.

[dead]