Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Measuring the sloppiness of code

earendil.com

Post reply on HN
Print view View original

191–200 of 247 posts

Re: Measuring the sloppiness of code

#191

Post by kolinko »

rayiner wrote: post #174

Is coding "solved?" I've been using Fable and Grok to vibecode a library to improve text extraction for legal PDFs. (Which have things like line numbers in transcripts that messes up text extraction.) This is a simple, dumb problem that should be like killing a mouse with a nuke. But I got to "pretty good" with a few iterations of not looking at the code or design. The program works, pretty well. But the design and h…

What harness? And why not just Fable+Opus?

Personally I use only one family for code editing, other families for code navigation.

Models tend to reason about the code using their own intuition and putting other families onto the same codebases may end up with them getting confused. Esp if that other model is as unhinged as Grok.

Also, harness matters a ton. Use a wrong harness with a good model and you’ll get terrible results.

Re: Measuring the sloppiness of code

#192

Post by kolinko »

rayiner wrote: post #188
keeda wrote: post #182

Earlier quoted context omitted.

Without knowing details of your approach, I would venture that your challenge is not with the coding per se but extracting structured data from PDFs. It’s a surprisingly hard problem because PDFs are optimized for preserving the visual structure and layout of the content for precise rendering and printing… NOT for preserving the logical structure of the data! Which is why the best results these days for extracting st…

The use case is basically the same ones where a programmer would use grep on source files, and it’s too slow to run the model on it directly for the same reasons. Existing tools do text extraction (pdftotext) but not that great when there’s internal numbering and stuff like that. It has to be a heuristic approach, and it won’t be perfect. But it’s an interesting data point that the model isn’t able to figure out the…

I gave a blanket ban on pdftotext to my agents. The output can get so mangled that a smart human wouldn’t untangle it. Did you try understanding the output from pdftotext yourself?

My approach is just ocr-ing with Terra or Gemini flash + checking citations with source both ways. But if I wanted to avoid llm calls, I’d just tell Fable to build a pdf reader directly from pdf binary format. Should be way more robust.

Re: Measuring the sloppiness of code

#193

Post by plourens »

bobkb wrote: post #67

Coding just a stage in the software development. Design and specifications which can help in coding is not solved at all and may never - the end result is software reliability is not a solved problem.

Sounds right to me. As AI has picked up, I have found myself drawn to resources that (I hope) help improve my taste and judgment not only of code but also of overall software design. The lessons in Code Complete, for example, have been helpful to me in thinking about managing the complexity of a codebase. It was useful before, too, but perhaps even more so now.

Re: Measuring the sloppiness of code

#194

Post by Kiro »

shimman wrote: post #184
Kiro wrote: post #118

Earlier quoted context omitted.

I've only worked at companies with great leadership. This is in the Nordics with very strong worker protection. And yet most of my colleagues including myself have been pretty terrible and write dirty code. It's not a management issue and it's not anti-worker to acknowledge this fact.

Your great leadership doesn't seem to care, so either your management knows better than you or maybe you should push back on the notion that you worked with "great leadership." Only poor leaders ignore their workers, which is what you seemed to have actually experience.

Sounds like you've only worked with brilliant people. Good for you but don't use your experience to spread these obviously delusional ideas.

Re: Measuring the sloppiness of code

#195

Post by NichoPaolucci »

toddwprice wrote: post #133

Coding is solved, perhaps, with unlimited token spend on a frontier model. It remains to be seen if it that is prohibitively expensive forever. At my company, we token maxed while the getting was good. But when we had to switch to Anthropic's enterprise plan, and start paying per token, the shit really hit the fan. Now we're retreating back to sane cost levels and finding that - guess what? - people power might just…

I'm interested in hearing more about your experience here.

Were you all just full bore shipping a ton with the per token plans? Was it more effective? How many developers @ your company?

It's really interesting to hear that some companies are back-pedaling. We "slowed" down a bit but are still very much using AI and intend to continue using it for... almost everything.

Re: Measuring the sloppiness of code

#196

Post by rayiner »

kolinko wrote: post #192
rayiner wrote: post #188

Earlier quoted context omitted.

The use case is basically the same ones where a programmer would use grep on source files, and it’s too slow to run the model on it directly for the same reasons. Existing tools do text extraction (pdftotext) but not that great when there’s internal numbering and stuff like that. It has to be a heuristic approach, and it won’t be perfect. But it’s an interesting data point that the model isn’t able to figure out the…

I gave a blanket ban on pdftotext to my agents. The output can get so mangled that a smart human wouldn’t untangle it. Did you try understanding the output from pdftotext yourself? My approach is just ocr-ing with Terra or Gemini flash + checking citations with source both ways. But if I wanted to avoid llm calls, I’d just tell Fable to build a pdf reader directly from pdf binary format. Should be way more robust.

A PDF is a command stream designed for rendering. Interpreting the command stream to get the positions of each glyph is deterministic and existing libraries (I use both pdf_oxide and lopdf) do that fine. Once you have glyph positions, you need to use various heuristics to reconstruct words, paragraphs, columns, headers and footers, etc. For example, in a patent document, there's two columns with a gutter in the middle of line numbers. If you interpret the document as having a single line, you'll get numbers mixed up with the text, which can throw off efforts to find particular phrases. PDF builders also insert all sorts of weird crap into the OCR layers that has to get normalized out.

It's just a pretty pedestrian data-munging problem where there's no closed form perfect solution and you have to use various heuristics to get the right result.

Re: Measuring the sloppiness of code

#197

Post by drsopp »

Goodharts law is invoked here with no discussion. Are we sure that optimizing for minimum LOC that pass all necessary tests produces sloppy code? And: What is sloppy code anyway? If we could define it, could we throw the definition into context and tell the LLM to avoid it?

Re: Measuring the sloppiness of code

#198

Post by hax0ron3 »

shimman wrote: post #185
hax0ron3 wrote: post #126

Earlier quoted context omitted.

The pressures from management and the company environment are not always a bad thing. It really depends on whether the pressures are coming from a logical business perspective or whether they are just coming from stupidity or ignorance. In a business environment, taking a long time to ship great code can mean that the company goes out of business, and then the software developers have a lot of great code and no incom…

I'm sorry but pressuring workers over this stuff is frankly disgusting and I reject you for your anti-worker + anti-human sentiment. What sort of human looks at a group of people and goes "it would be really nice if we made people miserable." Absolutely disgusting.

Wait. I didn't say anything about making people miserable. And I'm not talking about some weird kind of pressure like yelling at people.

I just mean the normal, almost inevitable kind of pressure that comes from the business situation. Management has to somehow figure out strategies to handle the pressure and it has to communicate these strategies and the reasoning for them to the engineers in a constructive way.

Generally, this is the pressure to be competitive and make money. It does the developers no good if they spend so much time writing great code that the company goes out of business.

Re: Measuring the sloppiness of code

#199

Post by NichoPaolucci »

FiberBundle wrote: post #84

Does anybody actually know whether there's a limit to the complexity LLMs are capable of dealing with in a codebase? It's very obvious that they don't write code that is suitable for people to understand it (and it's gonna get worse and worse the more RL is used to train these models), but if there isn't a point at which LLMs also struggle due to the complexity they introduce, then I'm not sure it really matters anym…

In theory - if an LLM could handle infinite complexity, I still think that the business issues + decisions end up getting in the way somewhere.

AI: "You asked to add feature X. Here are 25 questions that impact feature Z, B, and C in your gigantic codebase"

Developer: I can answer 8 of these questions... Guess I need to go figure out the rest of them.

Writing the code + building the functionality has always been the easy part.

Re: Measuring the sloppiness of code

#200

Post by greenowl »

toddwprice wrote: post #133

Coding is solved, perhaps, with unlimited token spend on a frontier model. It remains to be seen if it that is prohibitively expensive forever. At my company, we token maxed while the getting was good. But when we had to switch to Anthropic's enterprise plan, and start paying per token, the shit really hit the fan. Now we're retreating back to sane cost levels and finding that - guess what? - people power might just…

How did the developers react when they had to go back to writing their own code instead of playing Nintendo Switch between prompting sessions?
Post reply on HN