Viewing profile — charlie-haley

Hey, good question. Marmot is designed to be as generic as possible. An "Asset", whether it's a database, glossary term, topic, API or anything else, has the exact same schema, API…

[dead]

I wrote this after spending weeks figuring out how far Postgres could go before reaching for dedicated search indexers or a graph database. Pretty far, as it turns out. pg_trgm can…

Hey, that's a good question! At the moment, it treats the latest run as the desired state. So any new changes to a schema will simply overwrite the old version. I'd like to version…

Postgres has a lot of features such as trigram-based search which is pretty essential if I don't want to use a dedicated search indexer. It's also much better at handling concurren…

It can handle discovery within a plugin if the asset types are related. You can also manually add lineage via the UI or use Terraform to create lineage links via IaC. It's pretty c…

It supports either, I didn't want to restrict people to just one method of getting their catalog populated. The CLI and Plugin system works on needing read credentials to a given S…

Hey, there's some documentation around creating plugins here. It's relatively simple and involves adding a new Go package to the repo. Currently they have to be compiled into the B…

That's great to know, I wasn't aware anybody even attempted to used it yet! I'm currently in the process of overhauling the Plugin system, it's been quite hard to test some enterpr…

It depends on your ecosystem. If everything lives under one vendor their native catalog will probably work really well for you. But most of the time (especially for older orgs) the…

Hey HN, I wanted to show off my project Marmot! I decided to build Marmot after discovering a lot of data catalogs can be complex and require many external dependencies such as Kaf…

Open-source data catalog focused on simplicity with a single binary deployment and only a PostgreSQL dependency. Features powerful search with boolean/text/metadata queries, visual…