Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Measuring the sloppiness of code

earendil.com

Post reply on HN
Print view View original

221–230 of 246 posts

Re: Measuring the sloppiness of code

#221

Post by _pdp_ »

wang_li wrote: post #48
_pdp_ wrote: post #16

It will be solved when there is no more code left to write. Code is an abstract concept that is not bound to the physical world and I imagine that future will have some much more of it that it is difficult to comprehend. Everything will be code and more code will be written than ever before. Code will never going to be solved. The question is how much humans will be involved and I think the evidence is that perhaps j…

Code and software and applications are an intermediate stage. The final stage is an AI/LLM that just does the thing that is needed without any code being written, there are no applications or programs, just an AI that does everything.

AI needs tools as much as humans do.

Re: Measuring the sloppiness of code

#223

Post by vb-8448 »

kolinko wrote: post #190
conqrr wrote: post #83

Coding is not just the program running in memory, its also the process of distributing the mental model of understanding among the team. If humans increasingly are kept out of coding, then who holds the mental model? If AI holds the mental model, by definition human prompts will be over lossy channel. This is true without AI too. Software quality is directly dependent on good devs that translate from business/PM spea…

Who says anyone needs to hold a mental model? Imho a better way of thinking about it is in terms of views - you can have an unimaginably complex codebase that nobody reads in full, but either agents or humans receive views/perspectives on that codebase. Before LLMs we couldn’t have built tools that would deliver such views, now we can.

The only difference with the past that you need less time to dig through the codebase or documentation, the agent can do it for you and provide only meaningful info, but without a "Mental model"(or knowing what's going under the hood) your own prompts will be the main reason of the sloppiness and high token usage.

Re: Measuring the sloppiness of code

#224

Post by kolinko »

NichoPaolucci wrote: post #203
kolinko wrote: post #190

Earlier quoted context omitted.

Who says anyone needs to hold a mental model? Imho a better way of thinking about it is in terms of views - you can have an unimaginably complex codebase that nobody reads in full, but either agents or humans receive views/perspectives on that codebase. Before LLMs we couldn’t have built tools that would deliver such views, now we can.

I mean... Nobody NEEDs to hold a mental model, but we've been increasingly trying to build that mental model into an agentic codebase... maybe we're doing it wrong but it is NOT going well. The agents jot down absolutely everything (and more, which is the issue), thousands and thousands files of perceived business logic, decisions, workflows, architecture, data modeling, etc... It gets stale extremely quickly - we ch…

Oh, I just dropped all the comments. Memory and session histories are for storing these things, comments just add confusion.

Subsections - how do they run into the same problem? What falsification? The views I mean are dynamic ad hoc. Every single small task gets its own temporary view.

But perhaps I’m misunderstanding something that you’re saying?

Re: Measuring the sloppiness of code

#225

Post by hollerith »

rayiner wrote: post #196
kolinko wrote: post #192

Earlier quoted context omitted.

I gave a blanket ban on pdftotext to my agents. The output can get so mangled that a smart human wouldn’t untangle it. Did you try understanding the output from pdftotext yourself? My approach is just ocr-ing with Terra or Gemini flash + checking citations with source both ways. But if I wanted to avoid llm calls, I’d just tell Fable to build a pdf reader directly from pdf binary format. Should be way more robust.

A PDF is a command stream designed for rendering. Interpreting the command stream to get the positions of each glyph is deterministic and existing libraries (I use both pdf_oxide and lopdf) do that fine. Once you have glyph positions, you need to use various heuristics to reconstruct words, paragraphs, columns, headers and footers, etc. For example, in a patent document, there's two columns with a gutter in the middl…

>It's just a pretty pedestrian data-munging problem

In other words, how complicated could it be?

Well, if Adobe has been introducing complications and making the format brittle and inflexible over the last 35 years to make it hard for its competitors to write software to process PDF files, quite complicated.

Re: Measuring the sloppiness of code

#226

Post by rayiner »

keeda wrote: post #216
rayiner wrote: post #196

Earlier quoted context omitted.

A PDF is a command stream designed for rendering. Interpreting the command stream to get the positions of each glyph is deterministic and existing libraries (I use both pdf_oxide and lopdf) do that fine. Once you have glyph positions, you need to use various heuristics to reconstruct words, paragraphs, columns, headers and footers, etc. For example, in a patent document, there's two columns with a gutter in the middl…

Yes, but the edge cases are infinite and so heuristics don't scale well. As an example, at some point you would likely find yourself with "dueling" heuristics, forcing you to tune them, which is brittle, or find yet another heuristic as a tie-breaker, which ratchets up the complexity. (I just spent a lot of time on an adjacent but much simpler problem before finally giving up on churning heuristics!) As an example, m…

> This is why LLMs / VLMs are so much better at this task, because they can look at the document holistically like we can.

Totally agreed. But in this use case, PDFs are the working format, not just an archival format. An offline batch process to ingest the PDFs isn’t feasible. Unless there are some super fast LLMs I’m not aware of that can handle tens of PDF pages per second. It seems like Grok and Claude don’t try to read the PDF directly, they use pdftotext or some Python wrapper over pdfium. But maybe I’m missing something!

Re: Measuring the sloppiness of code

#227

Post by suprjami »

toddwprice wrote: post #133

Coding is solved, perhaps, with unlimited token spend on a frontier model. It remains to be seen if it that is prohibitively expensive forever. At my company, we token maxed while the getting was good. But when we had to switch to Anthropic's enterprise plan, and start paying per token, the shit really hit the fan. Now we're retreating back to sane cost levels and finding that - guess what? - people power might just…

> Coding is solved, perhaps, with unlimited token spend on a frontier model.

Coding is also a solved problem with unlimited salary budget on the very best developers.

Re: Measuring the sloppiness of code

#228

Post by kolinko »

rayiner wrote: post #196
kolinko wrote: post #192

Earlier quoted context omitted.

I gave a blanket ban on pdftotext to my agents. The output can get so mangled that a smart human wouldn’t untangle it. Did you try understanding the output from pdftotext yourself? My approach is just ocr-ing with Terra or Gemini flash + checking citations with source both ways. But if I wanted to avoid llm calls, I’d just tell Fable to build a pdf reader directly from pdf binary format. Should be way more robust.

A PDF is a command stream designed for rendering. Interpreting the command stream to get the positions of each glyph is deterministic and existing libraries (I use both pdf_oxide and lopdf) do that fine. Once you have glyph positions, you need to use various heuristics to reconstruct words, paragraphs, columns, headers and footers, etc. For example, in a patent document, there's two columns with a gutter in the middl…

Yeah that’s why heuristics should work on the lowest possible layer, not on pdftotext. If you use pdftotext you’re stripping positional data and other stuff.

Do you use a public set of documents? I bet I could almost oneshot this with my harness :p

Re: Measuring the sloppiness of code

#229

Post by rayiner »

kolinko wrote: post #228
rayiner wrote: post #196

Earlier quoted context omitted.

A PDF is a command stream designed for rendering. Interpreting the command stream to get the positions of each glyph is deterministic and existing libraries (I use both pdf_oxide and lopdf) do that fine. Once you have glyph positions, you need to use various heuristics to reconstruct words, paragraphs, columns, headers and footers, etc. For example, in a patent document, there's two columns with a gutter in the middl…

Yeah that’s why heuristics should work on the lowest possible layer, not on pdftotext. If you use pdftotext you’re stripping positional data and other stuff. Do you use a public set of documents? I bet I could almost oneshot this with my harness :p

Yes, the tool I’m describing works on what the pdf_oxide crate returns, which includes glyph positions.

Here’s a public appendix from a recent Federal Circuit case. It has a representative assortment of documents (opinions, briefs, patents, transcripts) but contains only cited pages to the appendix pagination is non-consecutive: https://www.courtlistener.com/docket/68048163/15/ecofactor-i...

The underlying docket is usually 100 to 1,000 times larger than this but it’s similar types of files and the appendix shows the various types of headers and footers that can exist.

Re: Measuring the sloppiness of code

#230

Post by ndriscoll »

shimman wrote: post #180
ndriscoll wrote: post #102

Earlier quoted context omitted.

We're talking about professionals here. People who (at least in the US) often make several multiples of the median worker. Competence is assumed, and every company I've been at has had programs to pay for additional school if the employee wants it. IIRC at least one had an explicit book allowance, and I don't doubt that I could expense books right now if I asked. Do surgeons and lawyers complain so regularly that man…

There is no professional developer in the US. There are no licensing requirements to write code. There are no repercussions against developers that write code that immiserate or kill Americans. The person slinging wordpress plugins at an agency is equivalent to the person writing malware at Meta in the eyes of the government. No one is training us because there are no regulations in our industry. Sorry but I still re…

We are given time though. Maybe not at some small sweatshop? But at least at big companies all the jobs are salaried, and either no one cares how exactly you spend your time, or they explicitly say it's fine to spend some time learning, and will pay for materials or courses.

They're generally not going to teach you how to program or what the latest technologies are or what the best ways to do things are because that's literally what they're paying you for. But IME they definitely give you plenty of time and autonomy to be learning things at work.

Post reply on HN