Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Code can debug low-level cryptography

words.filippo.io

Post reply on HN
Print view View original

41–50 of 217 posts

Re: Claude Code can debug low-level cryptography

#41

Post by simonw »

pton_xd wrote: post #33

> Full disclosure: Anthropic gave me a few months of Claude Max for free. They reached out one day and told me they were giving it away to some open source maintainers. Related, lately I've been getting tons of Anthropic Instagram ads; they must be near a quarter of all the sponsored content I see for the last month or so. Various people vibe coding random apps and whatnot using different incarnations of Claude. Or j…

I think it might be that they've hit product-market fit.

Developers find Claude Code extremely useful (once they figure out how to use it). Many developers subscribe to their $200/month plan. Assuming that's profitable (and I expect it is, since even for that much money it cuts off at a certain point to avoid over-use) Anthropic would be wise to spend a lot of money on marketing to try and grow their paying subscriber base for it.

Re: Claude Code can debug low-level cryptography

#42

Post by SamInTheShell »

sega_sai wrote: post #23
SamInTheShell wrote: post #15

Earlier quoted context omitted.

> Gemini and it's tooling is absolute shit. Which model were you using? In my experience Gemini 2.5 Pro is just as good as Claude Sonnet 4 and 4.5. It's literally what I use as a fallback to wrap something up if I hit the 5 hour limit on Claude and want to just push past some incomplete work. I'm just going to throw this out there. I get good results from a truly trash model like gpt-oss-20b (quantized at 4bits). The…

I share the same opinion on Gemini cli. Other than for simplest tasks it is just not usable, it gets stuck in loops, ignores instructions, fails to edit files. Plus it just has a plenty of bugs in the cli that you occasionally hit. I wish I could use it rather than pay an extra subscription for Claude Code, but it is just in a different league (at least as recently as couple of weeks ago)

Which model are you using though? When I run out of Gemini 2.5 Pro and it falls back to the Flash model, the Flash model is absolute trash for sure. I have to prompt it like I do local models. Gemini 2.5 Pro has shown me good results though. Nothing like "ignores instructions" has really occurred for me with the Pro model.

Re: Claude Code can debug low-level cryptography

#43

Post by vidarh »

lordnacho wrote: post #14

I'm not surprised it worked. Before I used Claude, I would be surprised. I think it works because Claude takes some standard coding issues and systematizes them. The list is long, but Claude doesn't run out of patience like a human being does. Or at least it has some credulity left after trying a few initial failed hypotheses. This being a cryptography problem helps a little bit, in that there are very specific keywo…

> The list is long, but Claude doesn't run out of patience like a human being does

I've flat out had Claude tell me it's task was getting tedious, and it will often grasp at straws to use as excuses for stopping a repetitive task and moving in to something else.

Keeping it on task when something keeps moving forward, is easy, but when it gets repetitive it takes a lot of effort to make it stick to it.

Re: Claude Code can debug low-level cryptography

#44

Post by theptip »

jack_tripper wrote: post #22
simonw wrote: post #11

Using coding agents to track down the root cause of bugs like this works really well: > Three out of three one-shot debugging hits with no help is extremely impressive. Importantly, there is no need to trust the LLM or review its output when its job is just saving me an hour or two by telling me where the bug is, for me to reason about it and fix it. The approach described here could also be a good way for LLM-skepti…

>Have the coding agents do the work of digging around hunting down those frustratingly difficult bugs - don't have it write code on your behalf. Why? Bug hunting is more challenging and cognitive intensive than writing code.

Bug hunting tends to be interpolation, which LLMs are really good at. Writing code is often some extrapolation (or interpolating at a much more abstract level).

Re: Claude Code can debug low-level cryptography

#45

Post by imiric »

qsort wrote: post #2

This resonates with me a lot: > As ever, I wish we had better tooling for using LLMs which didn’t look like chat or autocomplete I think part of the reason why I was initially more skeptical than I ought to have been is because chat is such a garbage modality. LLMs started to "click" for me with Claude Code/Codex. A "continuously running" mode that would ping me would be interesting to try.

On the one hand, I agree with this. The chat UI is very slow and inefficient.

But on the other, given what I know about these tools and how error-prone they are, I simply refuse to give them access to my system, to run commands, or do any action for me. Partly due to security concerns, partly due to privacy, but mostly distrust that they will do the right thing. When they screw up in a chat, I can clean up the context and try again. Reverting a removed file or messed up Git repo is much more difficult. This is how you get a dropped database during code freeze...

The idea of giving any of these corporations such privileges is unthinkable for me. It seems that most people either don't care about this, or are willing to accept it as the price of admission.

I experimented with Aider and a self-hosted model a few months ago, and wasn't impressed. I imagine the experience with SOTA hosted models is much better, but I'll probably use a sandbox next time I look into this.

Re: Claude Code can debug low-level cryptography

#46

Post by [deleted] »

imiric wrote: post #35
XenophileJKO wrote: post #19

Personally my biggest piece of advice is: AI First. If you really want to understand what the limitations are of the current frontier models (and also really learn how to use them), ask the AI first. By throwing things over the wall to the AI first, you learn what it can do at the same time as you learn how to structure your requests. The newer models are quite capable and in my experience can largely be treated like…

> By throwing things over the wall to the AI first, you learn what it can do at the same time as you learn how to structure your requests. Unfortunately, it doesn't quite work out that way. Yes, you will get better at using these tools the more you use them, which is the case with any tool. But you will not learn what they can do as easily, or at all. The main problem with them is the same one they've had since the b…

[deleted]

Re: Claude Code can debug low-level cryptography

#47

Post by thadt »

cluckindan wrote: post #26

So the ”fix” includes a completely new function? In a cryptography implementation? I feel like the article is giving out very bad advice which is going to end up shooting someone in the foot.

Can you expand on what you find to be 'bad advice'?

The author uses an LLM to find bugs and then throw away the fix and instead write the code he would have written anyway. This seems like a rather conservative application of LLMs. Using the 'shooting someone in the foot' analogy - this article is an illustration of professional and responsible firearm handling.

Re: Claude Code can debug low-level cryptography

#48

Post by delusional »

simonw wrote: post #37
qa34514324 wrote: post #30

Earlier quoted context omitted.

I have tested the AI SAST tools that were hyped after a curl article on several C code bases and they found nothing. Which low level code base have you tried this latest tool on? Official Anthropic commercials do not count.

You're posting this comment on a thread attached to an article where Filippo Valsorda - a noted cryptography expert - used these tools to track down gnarly bugs in Go cryptography code.

These are not "gnarly bugs".

Re: Claude Code can debug low-level cryptography

#49

Post by dboreham »

mschulkind wrote: post #31
simonw wrote: post #11

Using coding agents to track down the root cause of bugs like this works really well: > Three out of three one-shot debugging hits with no help is extremely impressive. Importantly, there is no need to trust the LLM or review its output when its job is just saving me an hour or two by telling me where the bug is, for me to reason about it and fix it. The approach described here could also be a good way for LLM-skepti…

One of my favorite ways to use LLM agents for coding is to have them write extensive documentation on whatever I'm about to dig in coding on. Pretty low stakes if the LLM makes a few mistakes. It's perhaps even a better place to start for skeptics.

Same. Initially surprised how good it was. Now routinely do this on every new codebase. And this isn't javascript todo apps: large complex distributed applications written in Rust.

Re: Claude Code can debug low-level cryptography

#50

Post by sega_sai »

SamInTheShell wrote: post #42
sega_sai wrote: post #23

Earlier quoted context omitted.

I share the same opinion on Gemini cli. Other than for simplest tasks it is just not usable, it gets stuck in loops, ignores instructions, fails to edit files. Plus it just has a plenty of bugs in the cli that you occasionally hit. I wish I could use it rather than pay an extra subscription for Claude Code, but it is just in a different league (at least as recently as couple of weeks ago)

Which model are you using though? When I run out of Gemini 2.5 Pro and it falls back to the Flash model, the Flash model is absolute trash for sure. I have to prompt it like I do local models. Gemini 2.5 Pro has shown me good results though. Nothing like "ignores instructions" has really occurred for me with the Pro model.

I get that even with the 2.5 pro
Post reply on HN