Live data from Hacker News

Show HN: I built a local Elixir/Python pipeline to curate 14,000 RAW photos

qwelian.com

1–3 of 3 posts

Show HN: I built a local Elixir/Python pipeline to curate 14,000 RAW photos

#1
I had 14,000 photos sitting on a drive and wanted an excuse to play with local vision models and Elixir/Phoenix. I originally tried to get LLaVA to tell me if a photo was 'good' or matched my style, but quickly learned that LLMs have terrible taste. I ended up demoting the LLM to just extract metadata, and built a custom CLIP/Ridge Regression pipeline to actually learn my preferences based on how I rate things.

The stack is Phoenix/Oban on the orchestrator side, and Python/FastAPI/Instructor for the AI workers. Happy to answer any questions about the architecture, fighting with local RAW file ingestion, or the pains of Pydantic validation with open weights.

Show HN: I built a local Elixir/Python pipeline to curate 14,000 RAW photos
qwelian.com

Re: Show HN: I built a local Elixir/Python pipeline to curate 14,000 RAW photos

#3
post #2

what was the biggest I/O bottleneck? I would assume RAW files are really heavy?

Locally this isn’t an issue but over a network call it would introduce some latency. It happens pretty fast tho. I’d say it’s a bottleneck because it happens too fast. I don’t have enough RAM to perform the next step which is metadata extraction leading to a lot of files being constantly processed.

It would be nice to be able to convert RAW files natively in elixir in tho