81–90 of 110 posts
Earlier quoted context omitted.
Wait, are you talking about llms or people?
It's sad that LLMs are learning to devalue human life from people like you.
People who try to get AI to do a lot of un-manned work seem to have a bit of a bias toward viewing LLM work as "good" or "good enough" in a way that they wouldn't have accepted in the pre-AI era.
I'm looking at these reports and then at my sessions with Claude. In latest one, it got stuck in a loop fixing one thing and breaking another for 10 iterations, until I've noticed and gave it the insight. Or where it started continuously writing "echo", I don't know for what purpose. And I ask how, how can you not be the driver; how can you let these agents roam free and actually accomplish things.
our c code probably looks at us and says: how, how can you let these agents roam free.
llms are not code and are incompetent and inefficient relative to code, but on the other hand you cannot roam without a little echo echo echo in your blood.
80 years, yet we still can't mark up hypertext without copying and then modifications to the source.
Earlier quoted context omitted.
Is it like self driving cars, where 100% non-human code would be arguably safer? But then it always comes down to having some human who is accountable for verifying the behavior. I think ultimately we just need a better set of languages and frameworks to minimally encode the types of applications we want to build today, so that human comprehension and verification of the source code becomes easier.
Category error. Self-driving cars are still meticulously coded by humans.
I have been using Claude to build a clone of Bash in Rust ( https://github.com/jdstanhope/huck ). It has been goin going on now for 81 days, 2600 commits and I have covered a good chunk of the features such that I can source by .bashrc with a number of extensions. It has been doing pretty good but it will sometimes spend hours to do few line fix and other times pushback on making big changes despite have over 3000 te…
Very cool, but I'm skeptical. Over the last year, every Cursor/Claude/Codex project I've worked on devolves into shitty, grafted-on, duct-tape and bandaid'ed architecture, with "fixes" just working around broken code, and tons of duplication that lead to whack-a-mole. The system works at first, but then bugs start piling up and eventually take over. Everyone I talk to shares the same experience. Not to mention that i…
Doesn't that happen to a human engineering team if you don't course correct it many times a month (which in code output is similar to an LLM in a day)?
- They build a browser engine from scratch that implements every whatwg spec for the last 15 yrs, fixes every bug chromium has and runs 10x faster than chromium
- they build a super highly optimized version of windows that runs much much faster than windows 11
We (Team of 3) built a retrieval engine from scratch in .NET for investigative search, with AI doing most of the backend implementation while humans handled the design and architecture and front end. It was developed using a task based, incremental approach. Includes more than 3,000 tests and extensive documentation, and is optimized for asynchronous ingestion using NATS and a disk backed WAL. It includes a custom WA…
How long did it take you to review things for correctness? Like an incremental query engine is a few thousand lines of reasonably difficult code on its own, a reliable WAL is another few thousand, etc, and each of those might take me days or weeks of thinking to have any serious confidence in them. I guess you can get provenance and ACL tracking "for free" from a query engine, but it still seems like a very ambitious…
The ingestion side includes durable asynchronous processing, recovery and replay, idempotency, strict ordering, structure aware segmentation, parent child relationships, and batched local embedding. The retrieval side includes hybrid (rrf), text and vector search, document versioning, collection and category scoping, document level ACL enforcement, parent expansion, provenance, and citation tracking.
Your estimate is fair. But, reliable WAL and retrieval layer are each substantial projects. We did not build the underlying database query engine from scratch, but the correctness work around ingestion, recovery, hierarchy, security, ranking, and provenance was still significant.
The project now has more than 3,000 automated tests, including coverage for recovery, concurrency, ordering, ACLs, versioning, and retrieval behaviour. Tests are not proof of correctness, but incremental implementation, explicit invariants, failure testing, and continuous human review are how we built confidence in it. We also had script to automate live endpoint tests with real data samples, this help largely to mitigate issues after a refactoring or enhancement. total production code is 80k LOC and rest is test coverages. One thing to say after seeing the results are, many of the RAG framework does it wrong. We got exceptional results and we also have agentic retrieval for answering complex queries. We designed it in our own way because government systems cannot afford false results, especially when the product is intended for crime investigation or compliance management in banks.
Edit: I forgot to mention that we also support PostgreSQL.
After building this system, we concluded that vector dimensionality is only a small part of retrieval accuracy. We use 384 dim embeddings and still achieve excellent results.