Claude Code can debug low-level cryptography • Hacker News Board

Re: Claude Code can debug low-level cryptography

#161

Post by simonw » Sun, Nov 02, 2025, 10:11 AM UTC

Razengan wrote: post #158

simonw wrote: post #151

Earlier quoted context omitted.

> Claude still sometimes insists that iOS 26 isn't out yet. How would you imagine an AI system working that didn't make mistakes like that? iOS 26 came out on September 15th. LLMs aren't omniscient or constantly updated with new knowledge. Which means we have to figure out how to make use of them despite them not having up-to-the-second knowledge of the world.

> How would you imagine an AI system working that didn't make mistakes like that? I mean, if the user says "Use the latest APIs as of version N" and the AI thinks version N isn't out yet, then it should CHECK on the web first, it's right there, before second guessing the user. I didn't ask it whether 26 was out or not. I told it. Oh but I guess AIs aren't allowed to have free use of Google's web search or scrap other…

Was this Claude Code or Claude.ai or some other tool that used Claude under the hood?

Different harnesses have different search capabilities.

If I'm doing something that benefits from search I tend to switch to ChatGPT because I know it has a really good search feature available to it. I don't trust Claude's as much.

Re: Claude Code can debug low-level cryptography

#162

Post by theshrike79 » Sun, Nov 02, 2025, 10:19 AM UTC

joshvm wrote: post #104

rtpg wrote: post #61

Earlier quoted context omitted.

I understand the pitch here ("it finds bugs! it's basically all upside because worst case there's no output anyways"), but I'm finding some of these agents to be ... uhhh... kind of agressive at trying to find the solution and end up missing the forest for the trees. And there's some "oh you should fix this" stuff which, while sometimes isn't _wrong_, is completely besides the point. The end result being these robots…

When models start to forage around in the weeds, it's a good idea to restart the session and add more information to the prompt for what it should ignore or assume. For example in ML projects, Claude gets very worried that datasets aren't available or are perhaps responsible. Usually if you tell it outright where you suspect the bug to be (or straight up tell it, even if you're unsure) it will focus on that. Or, make…

Context7 MCP is the one I keep enabled for all sessions. Then there are MCPs that give LSP access to the models as well as tools like Crush[0] that have LSPs built in.

[0] https://github.com/charmbracelet/crush

Re: Claude Code can debug low-level cryptography

#163

Post by theshrike79 » Sun, Nov 02, 2025, 10:20 AM UTC

jack_tripper wrote: post #22

simonw wrote: post #11

Using coding agents to track down the root cause of bugs like this works really well: > Three out of three one-shot debugging hits with no help is extremely impressive. Importantly, there is no need to trust the LLM or review its output when its job is just saving me an hour or two by telling me where the bug is, for me to reason about it and fix it. The approach described here could also be a good way for LLM-skepti…

>Have the coding agents do the work of digging around hunting down those frustratingly difficult bugs - don't have it write code on your behalf. Why? Bug hunting is more challenging and cognitive intensive than writing code.

Because it's easy to automate.

"this should return X, it returns Y, find out why"

With enough tooling LLMs can pretty easily figure out the reason eventually.

Re: Claude Code can debug low-level cryptography

#164

Post by theshrike79 » Sun, Nov 02, 2025, 10:26 AM UTC

Razengan wrote: post #139

XenophileJKO wrote: post #19

Personally my biggest piece of advice is: AI First. If you really want to understand what the limitations are of the current frontier models (and also really learn how to use them), ask the AI first. By throwing things over the wall to the AI first, you learn what it can do at the same time as you learn how to structure your requests. The newer models are quite capable and in my experience can largely be treated like…

I did ask the AI first, about some things that I already knew how to do. It gave me horribly inefficient or long-winded ways of doing it. In the time it took for "prompt tuning" I could have just written the damn code myself. It decreased the confidence for anything else it suggested about things I didn't already know about. Claude still sometimes insists that iOS 26 isn't out yet. sigh.. I suppose I just have to tre…

You can write the "prompt tuning" down in AGENTS.md and then you only need to do it once. This is why you need to keep working with different ones to get the feeling what they're good at and how you can steer them closer to your style and preferences without having to reiterate from scratch every time.

I personally have a git submodule built specifically for shared instructions like that, it contains the assumptions and defaults for my specific style of project for 3 different programming languages. When I update it on one project, all my projects benefit.

This way I don't need to tell whatever LLM I'm working with to use modernc.org/sqlite for database connections, for example.

Re: Claude Code can debug low-level cryptography

#165

Post by theshrike79 » Sun, Nov 02, 2025, 10:28 AM UTC

chatmasta wrote: post #72

simonw wrote: post #41

Earlier quoted context omitted.

I think it might be that they've hit product-market fit. Developers find Claude Code extremely useful (once they figure out how to use it). Many developers subscribe to their $200/month plan. Assuming that's profitable (and I expect it is, since even for that much money it cuts off at a certain point to avoid over-use) Anthropic would be wise to spend a lot of money on marketing to try and grow their paying subscribe…

What makes it better than VSCode Co-pilot with Claude 4.5? I barely program these days since I switched to PM but I recently started using that and it seems pretty effective… why should I use a fork instead?

Agents, skills etc. Stuff that's specific to the Claude CLI tooling and not the model.

Sonnet 4.5 as a raw model is good, but what makes it great is the tool that calls it.

Think of it like this: Sonnet 4.5 is the engine, but the whole car around it matters a LOT.

Copilot is kinda crap as a LLM tool, the tool calling permissions are clunky, it doesn't support sub agents or skills or anything fancy really. The best thing about it is that it can see the "problems" tab on VSCode provided by different addons and linters and you can tell an agent "fix the active problems" and it'll get to work.

Re: Claude Code can debug low-level cryptography

#166

Post by marginalia_nu » Sun, Nov 02, 2025, 10:37 AM UTC

lisbbb wrote: post #130

deadbabe wrote: post #66

With AI, we will finally be able to do the impossible: roll our own crypto .

You're not going to do better than the NSA.

To be fair you're also not going to be backdoored by the NSA.

Re: Claude Code can debug low-level cryptography

#167

Post by bravetraveler » Sun, Nov 02, 2025, 10:39 AM UTC

dns_snek wrote: post #147

advael wrote: post #122

Earlier quoted context omitted.

With all due respect, "wow this is cool, I wonder what it'll turn into" is basically the mandatory baseline stance to take. I'm lucky that's where I'm still basically at, because anyone in a technical position who shows even mild reticence beyond that is likely to be unable to hold a job in the face of their bosses' frothing enthusiastic optimism about these technologies

Is it that bad out there? Yeah, I don't think I could last in a job that tries to force these tools into my workflow.

Drive-by comment: it's not so bad, here. I work with a few peers who've proven to be evangelists with much stronger social skills. When the proposition comes up, I ask how my ass should be cleaned, too. Thankfully: the bosses haven't heard/don't care.

Varying degrees of 'force' at play; I'm lucky that nobody significant is minding my [absence of] LLM usage. Just some peers excited to do more for the same or, arguably, less reward. Remember: we're now all in an arms race. Some of us had a head start.

How crass I respond to the suggestion depends on their delivery/relevance in my process/product, of course. May be placated like a child with a new toy... or the gross question to, hopefully, express the suggestion isn't wanted, needed, or welcome.

Faced with a real mandate, I'd feed it garbage while looking for new work. Willing to make the bet I can beat enough machines while people are still involved at all.

Re: Claude Code can debug low-level cryptography

#168

Post by timr » Sun, Nov 02, 2025, 10:48 AM UTC

oefrha wrote: post #159

timr wrote: post #133

Earlier quoted context omitted.

Meh. The context stuff is changing by the day, so whatever you're saying now will be out of date by next week. Regardless, you're basically saying that GHC is trying to optimize for cost, which is true for any provider. Even if there's some slight immediate performance advantage for Cursor over GHC, the ability to trivially switch models more than makes up for it, IMO.

The question was whether Claude Code's better than GHC. "They may release a new version that bridges the gap any moment now" is a completely useless answer to that. And your argument is "people either have never tried it, or tried it a long time ago when it was something else", and I told you I'm comparing it right now, and have done the same a year ago, and many points in between, and GHC is inferior at every single…

I've used both, and you're exaggerating. Whatever difference in performance there is between providers changes constantly, and like I said, it's more than offset for me by the practical advantage of being able to switch models regularly.

Re: Claude Code can debug low-level cryptography

#169

Post by kryogen1c » Sun, Nov 02, 2025, 10:58 AM UTC

Wowfunhappy wrote: post #107

ori_b wrote: post #83

Earlier quoted context omitted.

Doing it yourself is how you build and maintain the muscles to do it yourself. If you only do it yourself when the LLM fails, how will you maintain those muscles?

I agree, and I can actively feel myself slipping (and perhaps more critically, not learning new skills I would otherwise have been forced to learn). It's a big problem, but somewhat orthogonal to "what is the quickest way to solve the task currently in front of me."

> but somewhat orthogonal to "what is the quickest way to solve the task currently in front of me."

That depends on if you ignore the future. You are never just solving the problem in front of you; you should always act in a way that propagates positivity forward in time.

Re: Claude Code can debug low-level cryptography

#170

Post by nikanj » Sun, Nov 02, 2025, 11:00 AM UTC

I'm surprised it didn't fix it by removing the code. In my experience, if you give Claude a failing test, it fixes it by hard-coding the code to return the value expected by the test or something similar.

Last week I asked it to look at why a certain device enumeration caused a sigsegv, and it quickly solved the issue by completely removing the enumeration. No functionality, no bugs!