Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Code can debug low-level cryptography

words.filippo.io

Post reply on HN
Print view View original

51–60 of 217 posts

Re: Claude Code can debug low-level cryptography

#51

Post by Frannky »

wdfx wrote: post #12
BrokenCogs wrote: post #5

Earlier quoted context omitted.

How good is Gemini CLI compared to Claude code and openAi codex?

Gemini and it's tooling is absolute shit. The LLM itself is barely usable and needs so much supervision you might as well do the work yourself. Then couple that with an awful cli and vscode interface and you'll find that it's just a complete waste of time. Compared to the anthropic offering is night and day. Claude gets on with the job and makes me way more productive.

It's probably a mix of what you're working on and how you're using the tool. If you can't get it done for free or cheaply, it makes sense to pay. I first design the architecture in my mind, then use Grok 4 fast (free) for single-shot generation of main files. This forces me to think first, and read the generated code to double-check. Then, the CLI is mostly for editing, clerical work, testing, etc. That said, I do try to avoid coding altogether if the CLI + MCP servers + MD files can solve the problem.

Re: Claude Code can debug low-level cryptography

#52

Post by delaminator »

jamesponddotco wrote: post #24
delaminator wrote: post #10

> For example, how nice would it be if every time tests fail, an LLM agent was kicked off with the task of figuring out why, and only notified us if it did before we fixed it? You can use Git hooks to do that. If you have tests and one fails, spawn an instance of claude a prompt -p 'tests/test4.sh failed, look in src/ and try and work out why' $ claude -p 'hello, just tell me a joke about databases' A SQL query walks…

This could probably be implemented as a simple Bash script, if the user wants to run everything manually. I might just do that to burn some time.

sure, there a multiple ways of spawning an instance

the only thing I imagine might be problem is claude demanding a login token as it happens quite regularly

Re: Claude Code can debug low-level cryptography

#53

Post by XenophileJKO »

imiric wrote: post #35
XenophileJKO wrote: post #19

Personally my biggest piece of advice is: AI First. If you really want to understand what the limitations are of the current frontier models (and also really learn how to use them), ask the AI first. By throwing things over the wall to the AI first, you learn what it can do at the same time as you learn how to structure your requests. The newer models are quite capable and in my experience can largely be treated like…

> By throwing things over the wall to the AI first, you learn what it can do at the same time as you learn how to structure your requests. Unfortunately, it doesn't quite work out that way. Yes, you will get better at using these tools the more you use them, which is the case with any tool. But you will not learn what they can do as easily, or at all. The main problem with them is the same one they've had since the b…

I guess there are several unsaid assumptions here. The article is by a domain expert working on their domain. Throw work you understand at it, see what it does. Do it before you even work on it. I kind of assumed based on the audience that most people here would be domain experts.

As for the building intuition, perhaps I am over-estimating what most people are capable of.

Working with and building systems using LLMs over the last few years has helped me build a pretty good intuition about what is breaking down when the model fails at a task. While having an ML background is useful in some very narrow cases (like: 'why does an LLM suck at ranking...'), I "think" a person can get a pretty good intuition purely based on observational outcomes.

I've been wrong before though. When we first started building LLM products, I thought, "Anyone can prompt, there is no barrier for this skill." That was not the case at all. Most people don't do well trying to quantify ambiguity, specificity, and logical contridiction when writing a process or set of instructions. I was REALLY surprised how I became a "go-to" person to "fix" prompt systems all based on linguistics and systematic process decomposition. Some of this was understaing how the auto-regressive attention system benefits from breaking the work down into steps, but really most of it was just "don't contradict yourself and be clear".

Working with them extensively also has helped me hone in on how the models get "better" with each release. Though most of my expertise is with OpenAI and Antrhopic model families.

I still think most engineers "should" be able to build intuition generally on what works well with LLMs and how to interact with them, but you are probably right. It will be just like most ML engineers where they see something work in a paper and then just paste it onto their model with no intuition about what systemically that structurally changes in the model dynamics.

Re: Claude Code can debug low-level cryptography

#54

Post by jen729w »

cmrdporcupine wrote: post #8
Frannky wrote: post #6

Earlier quoted context omitted.

I started with Claude Code, realized it was too much money for every message, then switched to Gemini CLI, then Qwen. Probably Claude Code is better, but I don't need it since I can solve my problems without it.

Try what I've done: use the Claude Code tool but point your ANTHROPIC_URL at a DeepSeek API membership. It's like 1/10th the cost, and about 2/3rds the intelligence. Sometimes I can't really tell.

Or a 3rd party service like https://synthetic.new, of which I am an unaffiliated user.

Re: Claude Code can debug low-level cryptography

#55

Post by [deleted] »

jack_tripper wrote: post #22
simonw wrote: post #11

Using coding agents to track down the root cause of bugs like this works really well: > Three out of three one-shot debugging hits with no help is extremely impressive. Importantly, there is no need to trust the LLM or review its output when its job is just saving me an hour or two by telling me where the bug is, for me to reason about it and fix it. The approach described here could also be a good way for LLM-skepti…

>Have the coding agents do the work of digging around hunting down those frustratingly difficult bugs - don't have it write code on your behalf. Why? Bug hunting is more challenging and cognitive intensive than writing code.

[deleted]

Re: Claude Code can debug low-level cryptography

#56

Post by behnamoh »

cmrdporcupine wrote: post #8
Frannky wrote: post #6

Earlier quoted context omitted.

I started with Claude Code, realized it was too much money for every message, then switched to Gemini CLI, then Qwen. Probably Claude Code is better, but I don't need it since I can solve my problems without it.

Try what I've done: use the Claude Code tool but point your ANTHROPIC_URL at a DeepSeek API membership. It's like 1/10th the cost, and about 2/3rds the intelligence. Sometimes I can't really tell.

I use this to proxy ANTHROPIC_BASE_URL to other models: https://github.com/ujisati/claude-code-provider-proxy

unfortunately it doesn't support local models but they're too slow for coding anyway.

Re: Claude Code can debug low-level cryptography

#57

Post by cmrdporcupine »

jen729w wrote: post #54
cmrdporcupine wrote: post #8

Earlier quoted context omitted.

Try what I've done: use the Claude Code tool but point your ANTHROPIC_URL at a DeepSeek API membership. It's like 1/10th the cost, and about 2/3rds the intelligence. Sometimes I can't really tell.

Or a 3rd party service like https://synthetic.new , of which I am an unaffiliated user.

So, Deepseek 3.1 from their own platform:

Input $0.28 / 1M tokens cache miss Output $0.42 / 1M tokens

Via synthetic (which otherwise looks cool):

Input $0.56/mtok Output $1.68/mtok

So 2-3 better value through https://platform.deepseek.com

(Granted Synthetic gives you way more models to choose from, including ones that don't parrot CPC/PLA propaganda and censor)

Re: Claude Code can debug low-level cryptography

#58

Post by lxgr »

simonw wrote: post #11

Using coding agents to track down the root cause of bugs like this works really well: > Three out of three one-shot debugging hits with no help is extremely impressive. Importantly, there is no need to trust the LLM or review its output when its job is just saving me an hour or two by telling me where the bug is, for me to reason about it and fix it. The approach described here could also be a good way for LLM-skepti…

I’ve been pretty impressed with LLMs at (to me) greenfield hobby projects, but not so much at work in a huge codebase.

After reading one of your blog posts recommending it, I decided to specifically give them a try as bug hunters/codebase explainers instead, and I’ve been blown away. Several hard-to-spot production bugs down in two weeks or so that would have all taken me at least a few focused hours to spot all in all.

Re: Claude Code can debug low-level cryptography

#59

Post by lxgr »

jack_tripper wrote: post #22
simonw wrote: post #11

Using coding agents to track down the root cause of bugs like this works really well: > Three out of three one-shot debugging hits with no help is extremely impressive. Importantly, there is no need to trust the LLM or review its output when its job is just saving me an hour or two by telling me where the bug is, for me to reason about it and fix it. The approach described here could also be a good way for LLM-skepti…

>Have the coding agents do the work of digging around hunting down those frustratingly difficult bugs - don't have it write code on your behalf. Why? Bug hunting is more challenging and cognitive intensive than writing code.

Why as in “why should it work” or “why should we let them do it”?

For the latter, the good news is that you’re free to use LLMs for debugging or completely ignore them.

Re: Claude Code can debug low-level cryptography

#60

Post by majormajor »

simonw wrote: post #11

Using coding agents to track down the root cause of bugs like this works really well: > Three out of three one-shot debugging hits with no help is extremely impressive. Importantly, there is no need to trust the LLM or review its output when its job is just saving me an hour or two by telling me where the bug is, for me to reason about it and fix it. The approach described here could also be a good way for LLM-skepti…

They're quite good at algorithm bugs, a lot less good at concurrency bugs, IME. Which is very valuable still, just that's where I've seen the limits so far.

Their also better at making tests for algorithmic things than for concurrency situations, but can get pretty close. Just usually don't have great out-of-the-box ideas for "how to ensure these two different things run in the desired order."

Everything that I dislike about generating non-greenfield code with LLMs isn't relevant to the "make tests" or "debug something" usage. (Weird/bad choices about when to duplicate code vs refactor things, lack of awareness around desired "shape" of codebase for long-term maintainability, limited depth of search for impact/related existing stuff sometimes, running off the rails and doing almost-but-not-quite stuff that ends up entirely the wrong thing.)

Post reply on HN