Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Code can debug low-level cryptography

words.filippo.io

Post reply on HN
Print view View original

61–70 of 217 posts

Re: Claude Code can debug low-level cryptography

#61

Post by rtpg »

simonw wrote: post #11

Using coding agents to track down the root cause of bugs like this works really well: > Three out of three one-shot debugging hits with no help is extremely impressive. Importantly, there is no need to trust the LLM or review its output when its job is just saving me an hour or two by telling me where the bug is, for me to reason about it and fix it. The approach described here could also be a good way for LLM-skepti…

I understand the pitch here ("it finds bugs! it's basically all upside because worst case there's no output anyways"), but I'm finding some of these agents to be ... uhhh... kind of agressive at trying to find the solution and end up missing the forest for the trees. And there's some "oh you should fix this" stuff which, while sometimes isn't _wrong_, is completely besides the point.

The end result being these robots doing bikeshedding. When paired with junior engineers looking at this output and deciding to act on it, it just generates busywork. Not helping that everyone and their dog wants to automatically run their agent against PRs now

I'm trying to use these to some extent when I find myself in a canonical situation that should work and am not getting the value everyone else seems to get in many cases. Very much "trying to explain a thing to a junior engineer taking more time than doing it myself" thing, except at least the junior is a person.

Re: Claude Code can debug low-level cryptography

#62

Post by cmrdporcupine »

behnamoh wrote: post #56
cmrdporcupine wrote: post #8

Earlier quoted context omitted.

Try what I've done: use the Claude Code tool but point your ANTHROPIC_URL at a DeepSeek API membership. It's like 1/10th the cost, and about 2/3rds the intelligence. Sometimes I can't really tell.

I use this to proxy ANTHROPIC_BASE_URL to other models: https://github.com/ujisati/claude-code-provider-proxy unfortunately it doesn't support local models but they're too slow for coding anyway.

I've used that too, but in DeepSeek's case they provide an Anthropic API compatible endpoint so you don't have to.

Re: Claude Code can debug low-level cryptography

#63

Post by tptacek »

delusional wrote: post #48
simonw wrote: post #37

Earlier quoted context omitted.

You're posting this comment on a thread attached to an article where Filippo Valsorda - a noted cryptography expert - used these tools to track down gnarly bugs in Go cryptography code.

These are not "gnarly bugs".

They're not?

Re: Claude Code can debug low-level cryptography

#64

Post by fn-mote »

XenophileJKO wrote: post #53
imiric wrote: post #35

Earlier quoted context omitted.

> By throwing things over the wall to the AI first, you learn what it can do at the same time as you learn how to structure your requests. Unfortunately, it doesn't quite work out that way. Yes, you will get better at using these tools the more you use them, which is the case with any tool. But you will not learn what they can do as easily, or at all. The main problem with them is the same one they've had since the b…

I guess there are several unsaid assumptions here. The article is by a domain expert working on their domain. Throw work you understand at it, see what it does. Do it before you even work on it. I kind of assumed based on the audience that most people here would be domain experts. As for the building intuition, perhaps I am over-estimating what most people are capable of. Working with and building systems using LLMs…

> I kind of assumed based on the audience that most people here would be domain experts.

No take on the rest of your comment, but it’s the nature of software engineering that we work on a breadth of problems. Nobody can be a domain expert in everything.

For example: I use a configurable editor every day, but I’m not a domain expert in the configuration. An LLM wasted an hour of my day pointing me in “almost the right direction” when after 10 minutes I really needed to RTFM.

I am a domain expert in some programming languages, but now I need to implement a certain algorithm… I’m not an expert in that algorithm. There’s lots of traps for the unwary.

I just wanted to challenge the assumption that we are all domain experts in the things we do daily. We are, but … with limitations.

Re: Claude Code can debug low-level cryptography

#65

Post by cmrdporcupine »

imiric wrote: post #45
qsort wrote: post #2

This resonates with me a lot: > As ever, I wish we had better tooling for using LLMs which didn’t look like chat or autocomplete I think part of the reason why I was initially more skeptical than I ought to have been is because chat is such a garbage modality. LLMs started to "click" for me with Claude Code/Codex. A "continuously running" mode that would ping me would be interesting to try.

On the one hand, I agree with this. The chat UI is very slow and inefficient. But on the other, given what I know about these tools and how error-prone they are, I simply refuse to give them access to my system, to run commands, or do any action for me. Partly due to security concerns, partly due to privacy, but mostly distrust that they will do the right thing. When they screw up in a chat, I can clean up the contex…

Aider hurt my head it did not seem... good. Sorry to say.

If you want open source and want to target something over an API "crush" https://github.com/charmbracelet/crush is excellent

But you should try Claude Code or Codex just to understand them. Can always run them in a container or VM if you fear their idiocy (and it's not a bad idea to fear it)

Like I said sibling, it's not the right modality. Others agree. I'm a good typer and good at writing, so it doesn't bug me too much, but it does too much without asking or working through it. Sometimes this is brilliant. Other times it's like.. c'mon guy, what did you do over there? What Balrog have I disturbed?

It's good to be familiar with these things in any case because they're flooding the industry and you'll be reviewing their code for better or for worse.

Re: Claude Code can debug low-level cryptography

#67

Post by nl »

BrokenCogs wrote: post #5
Frannky wrote: post #4

CLI terminals are incredibly powerful. They are also free if you use Gemini CLI or Qwen Code. Plus, you can access any OpenAI-compatible API(2k TPS via Cerebras at 2$/M or local models). And you can use them in IDEs like Zed with ACP mode. All the simple stuff (creating a repo, pushing, frontend edits, testing, Docker images, deployment, etc.) is automated. For the difficult parts, you can just use free Grok to one-s…

How good is Gemini CLI compared to Claude code and openAi codex?

Not great.

It's ok for documentation or small tasks, but consistently fails at tasks that both Claude or Codex succeed at.

Re: Claude Code can debug low-level cryptography

#69

Post by nl »

SamInTheShell wrote: post #15
wdfx wrote: post #12

Earlier quoted context omitted.

Gemini and it's tooling is absolute shit. The LLM itself is barely usable and needs so much supervision you might as well do the work yourself. Then couple that with an awful cli and vscode interface and you'll find that it's just a complete waste of time. Compared to the anthropic offering is night and day. Claude gets on with the job and makes me way more productive.

> Gemini and it's tooling is absolute shit. Which model were you using? In my experience Gemini 2.5 Pro is just as good as Claude Sonnet 4 and 4.5. It's literally what I use as a fallback to wrap something up if I hit the 5 hour limit on Claude and want to just push past some incomplete work. I'm just going to throw this out there. I get good results from a truly trash model like gpt-oss-20b (quantized at 4bits). The…

I think you must be using it quite differently to me.

I can one-shot new webapps in Claude and Codex and can't in Gemini Pro.

Re: Claude Code can debug low-level cryptography

#70

Post by 8note »

vidarh wrote: post #43
lordnacho wrote: post #14

I'm not surprised it worked. Before I used Claude, I would be surprised. I think it works because Claude takes some standard coding issues and systematizes them. The list is long, but Claude doesn't run out of patience like a human being does. Or at least it has some credulity left after trying a few initial failed hypotheses. This being a cryptography problem helps a little bit, in that there are very specific keywo…

> The list is long, but Claude doesn't run out of patience like a human being does I've flat out had Claude tell me it's task was getting tedious, and it will often grasp at straws to use as excuses for stopping a repetitive task and moving in to something else. Keeping it on task when something keeps moving forward, is easy, but when it gets repetitive it takes a lot of effort to make it stick to it.

ive been getting that experience with both claude-code and gemini, but not from cline and qcli. i wonder why
Post reply on HN