Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Code can debug low-level cryptography

words.filippo.io

Post reply on HN
Print view View original

151–160 of 217 posts

Re: Claude Code can debug low-level cryptography

#151

Post by simonw »

Razengan wrote: post #139
XenophileJKO wrote: post #19

Personally my biggest piece of advice is: AI First. If you really want to understand what the limitations are of the current frontier models (and also really learn how to use them), ask the AI first. By throwing things over the wall to the AI first, you learn what it can do at the same time as you learn how to structure your requests. The newer models are quite capable and in my experience can largely be treated like…

I did ask the AI first, about some things that I already knew how to do. It gave me horribly inefficient or long-winded ways of doing it. In the time it took for "prompt tuning" I could have just written the damn code myself. It decreased the confidence for anything else it suggested about things I didn't already know about. Claude still sometimes insists that iOS 26 isn't out yet. sigh.. I suppose I just have to tre…

> Claude still sometimes insists that iOS 26 isn't out yet.

How would you imagine an AI system working that didn't make mistakes like that?

iOS 26 came out on September 15th.

LLMs aren't omniscient or constantly updated with new knowledge. Which means we have to figure out how to make use of them despite them not having up-to-the-second knowledge of the world.

Re: Claude Code can debug low-level cryptography

#153

Post by imiric »

XenophileJKO wrote: post #99
imiric wrote: post #88

Earlier quoted context omitted.

Exactly. A typical programmer works within unfamiliar domains all the time. It's not just about being familiar with the programming language or tooling. Every project potentially has new challenges you haven't faced before, new APIs to evaluate and design, new tradeoffs to consider, etc. The less familiar you are with the domain or API, the less instincts and influence you have to steer the LLM in the right direction…

I mean I "understand" your point. However, this isn't any different than being a technical lead in a system of any significant complexity.. you will constantly be reviewing work that you are not always an expert on, it is a very similar practice. I'm constantly reviewing things that I am not a domain expert on. I have to identify what is risky, what I don't know, etc. Throwing to the AI first is no different than thr…

Well, you're right that reviewing someone else's work isn't new, but interacting with these tools is vastly different from communicating with a coworker.

A competent human engineer won't delude you with claims not based in reality, and be confident about it. They can be wrong about practical ways of accomplishing something, but they won't suggest using APIs that don't exist, or go off on wild tangents because a certain word was mentioned. They won't give a different answer whenever you ask them the same question. Most importantly, conversations with humans can be productive in ways that both parties gain a deeper understanding of the topic and respect for each other. Humans can actually think and reason about topics and ideas, they can actually verify their and your claims, and they won't automatically respond with "You're right!" at any counterargument or suggestion.

Furthermore, the marketing around "AI" is strongly based on promoting their superhuman abilities. If we're led to believe that these are superintelligent machines, we're more inclined to trust their output. We have people using them as medical professionals, thinking that they're talking to a god, and being influenced by them. Trusting them to produce software is somewhere on that scale. All of this is highly misleading and potentially dangerous.

Any attempt at anthropomorphizing "AI" is a mistake. You can get much more out of them by using them as what they are: excellent pattern matching probabilistic tools.

Re: Claude Code can debug low-level cryptography

#154

Post by simonw »

dns_snek wrote: post #149
simonw wrote: post #11

Using coding agents to track down the root cause of bugs like this works really well: > Three out of three one-shot debugging hits with no help is extremely impressive. Importantly, there is no need to trust the LLM or review its output when its job is just saving me an hour or two by telling me where the bug is, for me to reason about it and fix it. The approach described here could also be a good way for LLM-skepti…

This is no different than when LLMs write code. In both scenarios they often turn into bullshit factories that are capable, willing, and happy to write pages and pages of intricate, convincing-sounding explanations for bugs that don't exist, wasting everyone's time and testing my patience.

That's not my experience at all. When I ask them to track down the root cause of a bug about 80% of the time they reply with a few sentences correctly identifying the source of the bug.

1/5 times the get it wrong and I might waste a minute or two confirming they they missed. I can live with those odds.

Re: Claude Code can debug low-level cryptography

#155

Post by danielbln »

8note wrote: post #70
vidarh wrote: post #43

Earlier quoted context omitted.

> The list is long, but Claude doesn't run out of patience like a human being does I've flat out had Claude tell me it's task was getting tedious, and it will often grasp at straws to use as excuses for stopping a repetitive task and moving in to something else. Keeping it on task when something keeps moving forward, is easy, but when it gets repetitive it takes a lot of effort to make it stick to it.

ive been getting that experience with both claude-code and gemini, but not from cline and qcli. i wonder why

Different system prompts, also Claude Code is aware of its own context and will sometimes try to simplify or take a short cut as the context nears exhaustion.

Some global rules will generally keep it on track though, telling it to ask me before it simplifies or give up, and I ask it frequently to ask me clarifying questions, which generally also helps keeping it chugging in the right direction and uncover gaps in its understanding.

Re: Claude Code can debug low-level cryptography

#156

Post by danielbln »

spacechild1 wrote: post #123
danielbln wrote: post #76

Earlier quoted context omitted.

OP said "for me to reason about it", not for the LLM to reason about it. I agree though, LLMs can be incredible debugging tools, but they are also incredibly gullable and love to jump to conclusions. The moment you turn your own fleshy brain off is when they go to lala land.

> OP said "for me to reason about it", not for the LLM to reason about it. But that's what I meant! Just recently I asked an LLM about a weird backtrace and it pointed me the supposed source of the issue. It sounded reasonable and I spent 1-2 hours researching the issue, only to find out it was a total red herring. Without the LLM I wouldn't have gone down that road in the first place. (But again, there have been man…

Yeah that's fair, I've been there before myself. It doesn't help when it throws "This is the smoking gun!" at you. I've started using subagents more, specifically a subagent that shells out codex. This way I can have Claude throw a problem over to GPT5 and both can come to a consensus. Doesn't completely prevent wild goose chases, but it helps a lot.

I also agree that many more times the LLM is like a blood hound leading me to the right thing (which makes it all the more annoying the few times when it chases a red herring).

Re: Claude Code can debug low-level cryptography

#157

Post by dns_snek »

simonw wrote: post #154
dns_snek wrote: post #149

Earlier quoted context omitted.

This is no different than when LLMs write code. In both scenarios they often turn into bullshit factories that are capable, willing, and happy to write pages and pages of intricate, convincing-sounding explanations for bugs that don't exist, wasting everyone's time and testing my patience.

That's not my experience at all. When I ask them to track down the root cause of a bug about 80% of the time they reply with a few sentences correctly identifying the source of the bug. 1/5 times the get it wrong and I might waste a minute or two confirming they they missed. I can live with those odds.

I'm assuming you delegate for most of your bugs? I only ask when I'm stumped and at that point it's very prone to generating false positives.

Re: Claude Code can debug low-level cryptography

#158

Post by Razengan »

simonw wrote: post #151
Razengan wrote: post #139

Earlier quoted context omitted.

I did ask the AI first, about some things that I already knew how to do. It gave me horribly inefficient or long-winded ways of doing it. In the time it took for "prompt tuning" I could have just written the damn code myself. It decreased the confidence for anything else it suggested about things I didn't already know about. Claude still sometimes insists that iOS 26 isn't out yet. sigh.. I suppose I just have to tre…

> Claude still sometimes insists that iOS 26 isn't out yet. How would you imagine an AI system working that didn't make mistakes like that? iOS 26 came out on September 15th. LLMs aren't omniscient or constantly updated with new knowledge. Which means we have to figure out how to make use of them despite them not having up-to-the-second knowledge of the world.

> How would you imagine an AI system working that didn't make mistakes like that?

I mean, if the user says "Use the latest APIs as of version N" and the AI thinks version N isn't out yet, then it should CHECK on the web first, it's right there, before second guessing the user. I didn't ask it whether 26 was out or not. I told it.

Oh but I guess AIs aren't allowed to have free use of Google's web search or scrap other websites eh

> iOS 26 came out on September 15th.

It was in beta all year and the APIs were publicly available on Apple's docs website. If I told it to use version 26 APIs then it should just use those instead of gaslighting me.

> LLMs aren't omniscient or constantly updated with new knowledge.

So we shouldn't use them if we want to make apps with the latest tech? Despite what the AI companies want us to believe.

You know, on a more general note, I think all AIs should have a toggle between "Do as I say" (Monkey Paw) and "Do what I mean"

Re: Claude Code can debug low-level cryptography

#159

Post by oefrha »

timr wrote: post #133
oefrha wrote: post #112

Earlier quoted context omitted.

No, there’s pretty noticeable difference between different tools even when they use the same model and interaction pattern. For instance I’ve used both GitHub Copilot and Cursor interactive agents (which are basically the same UX) aplenty in the past couple months for comparison, and GH Copilot is almost always dumber then Cursor, sometimes getting stuck on the stupidest issues. I assume context construction is likel…

Meh. The context stuff is changing by the day, so whatever you're saying now will be out of date by next week. Regardless, you're basically saying that GHC is trying to optimize for cost, which is true for any provider. Even if there's some slight immediate performance advantage for Cursor over GHC, the ability to trivially switch models more than makes up for it, IMO.

The question was whether Claude Code's better than GHC. "They may release a new version that bridges the gap any moment now" is a completely useless answer to that. And your argument is "people either have never tried it, or tried it a long time ago when it was something else", and I told you I'm comparing it right now, and have done the same a year ago, and many points in between, and GHC is inferior at every single point, and it's not slight. Cursor etc. wouldn’t have been this big if GHC was only slightly behind when it has such a huge early mover advantage and enormous backing.

Re: Claude Code can debug low-level cryptography

#160

Post by johnisgood »

ori_b wrote: post #143
Klathmon wrote: post #131

Earlier quoted context omitted.

Should you never use a calculator because you want to keep your math skills high? There are a growing set of problems which feel like using a calculator for basic math to me. But also school is a whole other thing which I'm much more worried about with LLMs. Because there's no doubt in my mind I would have abused AI every chance I got if it were around when I was a kid, and I wouldn't have learned a damn thing.

I don't use calculators for most math because punching it in is slower than doing it in my head -- especially for fermi calculations. I will reach for a calculator when it makes sense, but because I don't use a calculator for everything, the number of places where I'm faster than a calculator grows over time. It's not particularly intentional, it just shook out that way. And I hated mental math exercises as a kid.

I do not trust myself, so even if I know how to do mental math, I still use my computer or a calculator just to be sure I got it correct. OCD? Lack of self-trust? No clue.
Post reply on HN