Viewing profile — warwickmcintosh

ROCm has improved but the reality is you're still fighting the driver stack more than the models. If you're actually doing local inference on AMD you're spending your time on CUDA …

[dead]

Regex and NER both have fun edge cases for redaction. Medical record numbers that look like dates, addresses embedded in prose, account numbers with varying formats. Detection meth…

laziness makes you understand the problem before writing anything. an LLM will happily generate 500 lines for something that needed 20 because it never has to maintain any of it.

The sanitised optimism problem mentioned upthread is the real gap here. Event stream logging tells you what tools were called and in what order, but it doesn't tell you whether the…

LLM as judge drifts in weird ways if you don't have ground truth to calibrate against. Good that you've got that built in. Would love to see eval stability tracking over time thoug…

[dead]

Biggest change for us has been alert volume. AI-generated code ships faster, which means more deploys per day, which means more things to monitor...

The "every new model is THE one" cycle is getting a bit old but the Capybara tier thing is actually worth paying attention to.

The approval layer problem is interesting because the naive approach (approve every action) doesn't scale, but the fully autonomous approach terrifies anyone running these in produ…

I've noticed the time-of-day variance too. My working theory is it's related to load, not model changes. Same prompt at 6am Sydney time (when US is asleep) consistently gets better…

Retry logic modeling is a great inclusion most cost estimates miss this completely. In my experience retries account for 15-25% of total cost in production agent systems, especiall…

The file interaction heatmap is a smart addition. When debugging long agent runs, knowing which files got touched 30+ times vs once tells you immediately where the agent got stuck …

[dead]

The 71% reduction is interesting but I'd want to see where those tokens are actually going in a typical agent session. In my experience running multi-step coding agents, the git ou…