Live data from Hacker News

Run structured extraction on documents/images locally with Ollama and Pydantic

github.com

31–32 of 32 posts

Re: Run structured extraction on documents/images locally with Ollama and Pydantic

#31
post #3

Interesting. We're using a SAAS solution for document extraction right now. I don't know if it's in our interest to build out more but I do like the idea of keeping extraction local.

Cool, what types of documents do you currently handle? We could share some of our learnings/schemas here too.

Mostly tax forms, state-specific formations documents (Articles of X), and state-specific payroll registration documents.

Re: Run structured extraction on documents/images locally with Ollama and Pydantic

#32

I'd really like to play with Qwen2.5-VL at some point, perhaps for reading data-sheets for microchips. Nicely for some applications, it's also very good at reporting position of what it finds, which many ML tools are pretty mediocre at. https://qwenlm.github.io/blog/qwen2.5-vl/ Not really this application, but QvQ for visual reasoning is also impressive. https://qwenlm.github.io/blog/qvq-72b-preview/ Meta has used Qw…

Is Qwen2.5-VL on Ollama? Could give it a try with a few of the schemas we have. We’ve locally tested with Llama 3.2 11B Vision on Ollama: https://github.com/vlm-run/vlmrun-hub/blob/main/tests/benchm... FWIW I think Ollama structured outputs API is quite buggy compared to the HF transformers variant.

Just ran them for Qwen2.5-VL: https://github.com/vlm-run/vlmrun-hub/blob/main/tests/benchm...
Post reply on HN