Eight years of wanting, three months of building with AI

Essentially ~all of the tests were found to be useful but in a more "smoke test" capacity i.e. they provided good "basic" coverage but it was clear that it was also not sufficient.

Which is why in the rewrite: 1) I built a TCL driver that run the upstream SQLite tests and verified we accepted or rejected the SQL in the same way as SQLite.

2) I wrote a test runner which checked for "idempotence" i.e. run the formatter over all the SQL from all the other types of tests then verify that the AST was identical in the input and output.

3) I also wrote a script which ran the formatter over the PerfettoSQL standard library [1], a real world SQLite-based codebase that I knew and deeply understood so I could go through each file and manually check the output.

> Examples of that would also be nice (I don't doubt the personal feel that waste was justified)

Some things learned concretely:

1) C was not going to work for the higher level parts of the project, even the formatter was not pleasant to read or write in C, the validator was much worse

2) Doing the SQLite source extraction in the same language meant that I could ship a really cool feature where the syntaqlite CLI could "generate dialect extensions" without people needing to download a separate script, run their own extraction on the SQLite source code, or worse yet, need to fork syntaqlite. This actually makes it technically possible for people in the web playground to dynamically build extensions to SQLite (though I haven't ended up plumbing that feature through yet)

3) Having a DSL [2] for extensions of SQLite (that e.g. PerfettoSQL could use) was the correct way to go rather than using YAML/JSON/XML etc becaue of how much clarity it provided and how AI took a lot of the annoyance of maintaining a DSL away.

4) I need to invest much more in testing from the start and also more testing where the correctness can be "proved" in some way (e.g. idempotence testing or SQLite upstream testing as described above)

[1] https://github.com/google/perfetto/tree/main/src/trace_proce... [2] https://docs.syntaqlite.com/v0.2.15/guides/custom-dialects/

- first used copy and paste in and out of Grok

- started using CLI tools e.g. Claude and OpenCode

- move up to using 3 and sometimes 4 agents at the same time

- considered going to the agents managing agents

- have settled on having LLMs build tools that are both deterministic, usable by humans and the agent, and also faster (b/c there is less "back and forth")

Honestly, it feels a LOT like when Kubernetes came out. e.g. you stopped running containers on a box using Docker Compose plus scripts/configs etc. Instead gave a large part of the operation to an "agent" (in this case k8s) that managed all of the details you didn't need to care about anymore.

I've also realized that while the LLMs can crank out code at a very high rate, someone still needs to make sure everything is running, debug issues etc. You could set up agents to monitor what the agents do but then you still end up with someone needing to keep an eye on everything. If anything, you need MORE people b/c now you can just keep spinning up new components etc.

Also, was in a discussion with one of the best developers I've ever worked with. It came down to the following point:

"Programming is rapidly becoming a hobby. Software engineering is becoming more important than ever."

There seems to be a concencus among people I follow/read, that somewhere around that time was an inflexion point in coding LLMs, and this matches my personal experience.

(My comment was in the context of LLMs being used to generate non-throwaway code, not general GenAI use - apologies if that was unclear.)

"AI is an incredible force multiplier for implementation, but it’s a dangerous substitute for design."

If we are all honest, it seems to be the case - most of the time:

- Refactoring (Sometimes starting again.. this is rarely starting from scratch as there would have been some insights and personal design decisions garnered from the previous experience) - Specificity (It is heavily influenced by energy which also is different depending time of day or on the individual level)

At the end of the day, it takes taste + experience of the user to make anything of notable complexity(architecture) with AI.(For now and the nearest future at least).

I find reading articles as this gives me a renewed sense of agency as a technologist and my growing list of passions.

A solid thank you to Lalit Maganti for sharing and the better HN community. I found a lot to steal reuse from the material/banter.