Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Code can debug low-level cryptography

words.filippo.io

Post reply on HN
Print view View original

121–130 of 217 posts

Re: Claude Code can debug low-level cryptography

#121

Post by krackers »

mschulkind wrote: post #31
simonw wrote: post #11

Using coding agents to track down the root cause of bugs like this works really well: > Three out of three one-shot debugging hits with no help is extremely impressive. Importantly, there is no need to trust the LLM or review its output when its job is just saving me an hour or two by telling me where the bug is, for me to reason about it and fix it. The approach described here could also be a good way for LLM-skepti…

One of my favorite ways to use LLM agents for coding is to have them write extensive documentation on whatever I'm about to dig in coding on. Pretty low stakes if the LLM makes a few mistakes. It's perhaps even a better place to start for skeptics.

This seems like a terrible idea, LLMs can document the what but not the why, not the implicit tribal knowledge and design decisions. Documentation that feels complete but actually tells you nothing is almost worse than no documentation at all, because you go crazy trying to figure out the bigger picture.

Re: Claude Code can debug low-level cryptography

#122

Post by advael »

SV_BubbleTime wrote: post #120
rtpg wrote: post #61

Earlier quoted context omitted.

I understand the pitch here ("it finds bugs! it's basically all upside because worst case there's no output anyways"), but I'm finding some of these agents to be ... uhhh... kind of agressive at trying to find the solution and end up missing the forest for the trees. And there's some "oh you should fix this" stuff which, while sometimes isn't _wrong_, is completely besides the point. The end result being these robots…

Ok, fair critique. EXCEPT… What did you have for AI three years ago? Jack fucking shit is what. Why is “wow that’s cool, I wonder what it’ll turn into” a forbidden phrase, but “there are clearly no experts on this topic but let me take a crack at it!!” important for everyone to comment on? One word: Standby. Maybe that’s two words.

With all due respect, "wow this is cool, I wonder what it'll turn into" is basically the mandatory baseline stance to take. I'm lucky that's where I'm still basically at, because anyone in a technical position who shows even mild reticence beyond that is likely to be unable to hold a job in the face of their bosses' frothing enthusiastic optimism about these technologies

Re: Claude Code can debug low-level cryptography

#123

Post by spacechild1 »

danielbln wrote: post #76
spacechild1 wrote: post #74

> Importantly, there is no need to trust the LLM or review its output when its job is just saving me an hour or two by telling me where the bug is, for me to reason about it and fix it. Except they regularly come up with "explanations" that are completely bogus and may actually waste an hour or two. Don't get me wrong, LLMs can be incredibly helpful for identifying bugs, but you still have to keep a critical mindset.

OP said "for me to reason about it", not for the LLM to reason about it. I agree though, LLMs can be incredible debugging tools, but they are also incredibly gullable and love to jump to conclusions. The moment you turn your own fleshy brain off is when they go to lala land.

> OP said "for me to reason about it", not for the LLM to reason about it.

But that's what I meant! Just recently I asked an LLM about a weird backtrace and it pointed me the supposed source of the issue. It sounded reasonable and I spent 1-2 hours researching the issue, only to find out it was a total red herring. Without the LLM I wouldn't have gone down that road in the first place.

(But again, there have been many situations where the LLM did point me to the actual bug.)

Re: Claude Code can debug low-level cryptography

#124

Post by j2kun »

rtpg wrote: post #61
simonw wrote: post #11

Using coding agents to track down the root cause of bugs like this works really well: > Three out of three one-shot debugging hits with no help is extremely impressive. Importantly, there is no need to trust the LLM or review its output when its job is just saving me an hour or two by telling me where the bug is, for me to reason about it and fix it. The approach described here could also be a good way for LLM-skepti…

I understand the pitch here ("it finds bugs! it's basically all upside because worst case there's no output anyways"), but I'm finding some of these agents to be ... uhhh... kind of agressive at trying to find the solution and end up missing the forest for the trees. And there's some "oh you should fix this" stuff which, while sometimes isn't _wrong_, is completely besides the point. The end result being these robots…

> except at least the junior is a person.

+1 Juniors can learn over time.

Re: Claude Code can debug low-level cryptography

#125

Post by j2kun »

SV_BubbleTime wrote: post #120
rtpg wrote: post #61

Earlier quoted context omitted.

I understand the pitch here ("it finds bugs! it's basically all upside because worst case there's no output anyways"), but I'm finding some of these agents to be ... uhhh... kind of agressive at trying to find the solution and end up missing the forest for the trees. And there's some "oh you should fix this" stuff which, while sometimes isn't _wrong_, is completely besides the point. The end result being these robots…

Ok, fair critique. EXCEPT… What did you have for AI three years ago? Jack fucking shit is what. Why is “wow that’s cool, I wonder what it’ll turn into” a forbidden phrase, but “there are clearly no experts on this topic but let me take a crack at it!!” important for everyone to comment on? One word: Standby. Maybe that’s two words.

Careful there, ChatGPT was initially released November 30, 2022, which was just about 3 years ago, and there were coding assistants before that.

If you find yourself saying the same thing every year and adding 1 to the total...

Re: Claude Code can debug low-level cryptography

#126

Post by simonw »

krackers wrote: post #121
mschulkind wrote: post #31

Earlier quoted context omitted.

One of my favorite ways to use LLM agents for coding is to have them write extensive documentation on whatever I'm about to dig in coding on. Pretty low stakes if the LLM makes a few mistakes. It's perhaps even a better place to start for skeptics.

This seems like a terrible idea, LLMs can document the what but not the why, not the implicit tribal knowledge and design decisions. Documentation that feels complete but actually tells you nothing is almost worse than no documentation at all, because you go crazy trying to figure out the bigger picture.

Have you tried it? It's absurdly useful.

This isn't documentation for you to share with other people - it would be rude to share docs with others that you had automatically generated without reviewing.

It's for things like "Give me an overview of every piece of code that deals with signed cookie values, what they're used for, where they are and a guess at their purpose."

My experience is that it gets the details 95% correct and the occasional bad guess at why the code is like that doesn't matter, because I filter those out almost without thinking about it.

Re: Claude Code can debug low-level cryptography

#127

Post by [deleted] »

simonw wrote: post #118
NoraCodes wrote: post #113

Earlier quoted context omitted.

> start exploring how these tools can help them without feeling like they're [...] ripping off the work of everyone who's code was used to train the model But you literally still are. If you weren't, it should be trivially easy to create these models without using huge swathes of non-public-domain code. Right?

It feels less like you're ripping off work if the model is helping you understand your own code as opposed to writing new code from scratch - even though the models were built in exactly the same way. If someone scraped every photo on the internet (along with their captions) and used the data to create a model that was used purely for accessibility purposes - to build tools which described images to people with visua…

[deleted]

Re: Claude Code can debug low-level cryptography

#128

Post by jjmarr »

ribosometronome wrote: post #111
Klathmon wrote: post #95

Earlier quoted context omitted.

If the LLM is able to handle it why do you need to maintain those specific skills?

Should we not teach kids math because calculators can handle it? Practically, though, how would someone become good at just the skills LLMs don't do well? Much of this discussion is about how that's difficult to predict, but even if you were a reliable judge of what sort of coding tasks LLMs would fail at, I'm not sure it's possible to only be good at that without being competent at it all.

> Should we not teach kids math because calculators can handle it?

We don't teach kids how to use an abacus or a slide rule. But we teach positional representations and logarithms.

The goal is theoretical concepts so you can learn the required skills if necessary. The same will occur with code.

You don't need to memorize the syntax to write a for loop or for each loop, but you should understand when you might use either and be able to look up how to write one in a given language.

Re: Claude Code can debug low-level cryptography

#129

Post by lisbbb »

thadt wrote: post #47
cluckindan wrote: post #26

So the ”fix” includes a completely new function? In a cryptography implementation? I feel like the article is giving out very bad advice which is going to end up shooting someone in the foot.

Can you expand on what you find to be 'bad advice'? The author uses an LLM to find bugs and then throw away the fix and instead write the code he would have written anyway. This seems like a rather conservative application of LLMs. Using the 'shooting someone in the foot' analogy - this article is an illustration of professional and responsible firearm handling.

Honestly, it read more like attention seeking. He "live coded" his work, by which I believe he means he streamed everything he was doing while working. It just seems so much more like a performance and building a brand than anything else. I guess that's why I'm just a nobody.
Post reply on HN