Viewing profile — qwesr123

How do you plan to be harness agnostic if you design requires using the agent SDKs (Claude Agent SDK, etc.)?

FYI the MarginLab Claude Code degradation tracker is showing a statistically significant ~4% drop in SWE-Bench-Pro accuracy over the past month

Thanks! Daily confidence intervals are quite large and not super useful at the moment. Weekly aggregation is more sensitive. Hoping to increase sample sizes but it is quite expensi…

Where are you getting SWE-Bench Verified scores for 5.2-Codex? AFAIK those have not been published. And I don't think your Terminal-Bench 2.0 scores are accurate. Per the latest be…

Not sure why they don't compare with others, but they are actually leading on the benchmarks they published. See here (bottom) for a chart comparing to other models: https://margin…

Hi HN! My family has a history of IBS. For those not aware, a diet low in a class of carbs called FODMAPs is among the most effective treatments for IBS. Following a low-FODMAP die…