Show HN: Otlet – Local LLM inference "inside" Postgres

I’ve been working on Otlet, a Postgres extension for running local LLM inference “inside” Postgres (inference runs in a background worker, while Postgres stores the jobs, inputs, outputs, and receipts)

I started it while working on an entity resolution problem (determining which rows in a table refer to the same entity). The data was already in Postgres, but using a model meant pulling rows into another process and then writing the results back. I wanted to see whether I could keep more of that workflow in the database right next to where the data lives

Right now Otlet lets you start model work from SQL, require structured output, keep a record of each run, review proposed writes before they touch source data, etc. The example in the repo uses a small model for the first pass and a stronger model for uncertain cases

This is just something I've been hacking on, not something I would recommend attempting to use in production yet. There’s a Docker demo if anyone wants to try it. I’m mostly posting it because I wanted to share the approach while it’s still rough to see it get ripped to shreds

Show HN: Otlet – Local LLM inference "inside" Postgres
github.com