Viewing profile — EarlyOom

Shocking how poor frontier models perform on simple visual tasks. Best-in-domain tool calling will Become the norm

VLM Run | Member of Technical Staff, ML Systems | Full-time | Hybrid Bay Area, CA | https://vlm.run | 150k-220k / yr + Equity VLM Run is a first-of-its-kind API dedicated to runnin…

This is the main focus of VLM Run and typed extraction more generally. If you provide proper type constraints (e.g. with Pydantic) you can dramatically reduce the surface area for …

You can try out some of our schemas with Ollama if you want: https://github.com/vlm-run/vlmrun-hub (instructions in Readme)

VLMs are able to take context into account when filling in fields, following either a global or field specific prompt. This is great for e.g. unlabeled axes, checking a legend for …

You can! it works with Ollama https://github.com/vlm-run/vlmrun-hub At the end of the day its just schemas. You can decide for yourself if its work upgrading to a larger, more expe…

We can do bounding boxes too :) we just call it visual grounding https://github.com/vlm-run/vlmrun-cookbook/blob/main/noteboo...

We convert to a JSON schema, but it would be trivial to convert this to yaml. There are some minor differences in e.g. tokens required to output JSON vs yaml which is why we've opt…

OCR seems to be mostly solved for 'normal' text laid out according to Latin alphabet norms (left to right, normal spacing etc.), but would love to see more adversarial examples. We…

Would love to chat! reach out scott@vlm.run

That's one of our main focuses, yes: https://docs.vlm.run/api-reference/v1/fine-tuning/post-finet...

We put together an open-source collection of Pydantic schemas for a variety of document categories (W2 filings, invoices etc.), including instructions for how to get structured JSO…

VLM Run | Member of Technical Staff, ML Systems, Developer Relations | Full-time | Bay Area, CA | https://vlm.run | 150k-220k / yr + Equity VLM Run is a first-of-its-kind API dedic…

Hey HN, We’ve been building out an API for ‘Visual ETL’ that we call vlm.run. We’ve been working with foundation models (GPT4o, Gemini) for a few months and kept running into failu…

Curious how this compares to platforms like https://unstructured.io/

This is a bigger issue than folks realize, visual inputs to GPT4 are really expensive (like several cents per dozen images in some cases), which means that you can't just spam the …

We've been building something similar with https://vlm.run/ : we're starting out with documents, but feel like the real killer app will involve agentic workflows grounded in visual…

We've been trying to solve this with https://vlm.run : the idea is to combine the character level accuracy of an OCR pipeline (like Tesseract) with the flexibility of a VLM. OCR pi…

Hi Arthur! There's a bit of confusion here. It looks like you're referring to _audio_ transcription; that is, passing the audio component into an ASR pipeline (like Whisper, Otter …

TLDR: There are dozens of audio transcription APIs, but nothing for video and visual transcriptions. So we built one. If you want visual chaptering, summarization, OCR / text-extra…