Viewing profile — jakozaur

[flagged]

This isn’t limited to large system prompts. Coding-agent harnesses are also becoming more aggressive about using tools, even for trivial requests. In our tests, prompts such as “He…

Fable is also extended by another week till July 19: https://support.claude.com/en/articles/15424964-claude-fable...

It's rather hard to do at the proxy level with agentic coding, such as Claude Code or similar. These are long-chained sessions of tool use that heavily rely on prompt caching. Chan…

Poland's first partly free election was on 4 June 1989, preceded by the roundtable negotiations. The protests in Czechoslovakia came later, called the Velvet Revolution, from 17 to…

The story is longer: Poland was the first country to make a remarkable peaceful transition from a bankrupt, failed Soviet satellite state. The shock therapy, plus NATO and EU aspir…

A heat pump could win as the best HVAC technology, though a better drilling for ground-sourced ones. Just a shallow drilling (up to 100m) that works in retrofit mode, such as drill…

Build systems are tested by CompileBench (Quesma's benchmark). Disclaimer: I'm the founder.

I can provide trajectories. Though probably we are not going to publish them this time. This would need some extra safeguards. Email me. The address is in profile.

As I do eval and training data sets for living, in niche skills, you can find plenty of surprises. The code is open-source; you can run it yourself using Harbor Framework: git clon…

So many models refuse to do that due to alignment and safety concerns. So cross-model comparison doesn't make sense. We do, however, require proof (such as providing a location in …

Oh, nice find... We end up using PyGhidra, but the models waste some cycles because of bad ergonomics. Perhaps your cli would be easier. Still, Ghidra's most painful limitation was…

See direct benchmark link: https://quesma.com/benchmarks/binaryaudit/ Open-source GitHub: https://github.com/QuesmaOrg/BinaryAudit

Funny thing, AI is not that terrible at using Ghidra. We released a benchmark on that and hopefully models will improve: https://quesma.com/blog/introducing-binaryaudit/

Funny coincidence, I'm working on a benchmark showcasing AI capabilities in binary analysis. Actually, AI has huge potential for superhuman capabilities in reverse engineering. Thi…

Great idea. Currently, people have to rely on client-side spans in OpenTelemetry. However, it would be awesome if we could get spans for slow SQL queries, along with explanations.

In this benchmark, micro-services are really small, ~300 lines, and sometimes just two of them. More realistic tasks (large codebases, more microservices) would have a lower succes…

See x thread for rationale: https://x.com/mitchellh/status/2014433315261124760?s=46&t=FU... “ Ultimately, I want to see full session transcripts, but we don't have enough tool supp…

I wish Neal would do behind the scenes, how he built this art. I wonder whether LLM assistants like Claude Code make such an interactive show more feasible. He previously did a gam…

Not sure if I get this: WASM lets you use any language in the browser, though it still works way better with languages without GC, such as Rust or a transpiling C engine. Java is u…

LLVM IR is quite fun to play with from many programming languages. The Java example is rather educational, but there are several practical example,s such as in Go Lang: https://git…