Interesting. We're using a SAAS solution for document extraction right now. I don't know if it's in our interest to build out more but I do like the idea of keeping extraction local.
Cool, what types of documents do you currently handle? We could share some of our learnings/schemas here too.
Run structured extraction on documents/images locally with Ollama and Pydantic
31–32 of 32 posts
Mostly tax forms, state-specific formations documents (Articles of X), and state-specific payroll registration documents.
Re: Run structured extraction on documents/images locally with Ollama and Pydantic
#32I'd really like to play with Qwen2.5-VL at some point, perhaps for reading data-sheets for microchips. Nicely for some applications, it's also very good at reporting position of what it finds, which many ML tools are pretty mediocre at. https://qwenlm.github.io/blog/qwen2.5-vl/ Not really this application, but QvQ for visual reasoning is also impressive. https://qwenlm.github.io/blog/qvq-72b-preview/ Meta has used Qw…
Is Qwen2.5-VL on Ollama? Could give it a try with a few of the schemas we have. We’ve locally tested with Llama 3.2 11B Vision on Ollama: https://github.com/vlm-run/vlmrun-hub/blob/main/tests/benchm... FWIW I think Ollama structured outputs API is quite buggy compared to the HF transformers variant.
Just ran them for Qwen2.5-VL: https://github.com/vlm-run/vlmrun-hub/blob/main/tests/benchm...