> And many companies today want data preprocessing in ETL pipelines and data ingestion for RAG.
I'm curious, have you (or your customers) deployed this in a RAG use case already, and what have been the results like?
Launch HN: Trellis (YC W24) – AI-powered workflows for unstructured data
61–70 of 124 posts
Re: Launch HN: Trellis (YC W24) – AI-powered workflows for unstructured data
#62What about a pdf with many separate datapoints on it?
For instance, I have 100 pdfs, each with 10-100 individual products listed (in different formats).
I want to create a single table with one row per product appearing in any of the PDFs, with various details like price, product description, etc.,
From what I can tell from the demo, it seems like 1 file = 1 row in Trellis?
Re: Launch HN: Trellis (YC W24) – AI-powered workflows for unstructured data
#63I've had do some of this recently, as a one-off, to extract the same fields from thousands of scanned documents. I used OpenAI's function calling (via Langchain's https://python.langchain.com/v0.1/docs/modules/model_io/chat... API). Some of the challenges I had: 1. poor recall for some fields, even with a wide variety of input document formats 2. needing to experiment with the json schema (particularly field descript…
FWIW I've seen noticeably better results on (1) and (4) extracting JSON from images via Claude, although (2) and (3) still take effort.
Re: Launch HN: Trellis (YC W24) – AI-powered workflows for unstructured data
#64What about a pdf with many separate datapoints on it? For instance, I have 100 pdfs, each with 10-100 individual products listed (in different formats). I want to create a single table with one row per product appearing in any of the PDFs, with various details like price, product description, etc., From what I can tell from the demo, it seems like 1 file = 1 row in Trellis?
Good question and we have seen this extraction workflow a lot in financial services. We just added table mode to the product (select table in transformation parameters) where we extract table structure in the documents that match that schema. So 1 file map to N rows where N is all the row in the table.
Re: Launch HN: Trellis (YC W24) – AI-powered workflows for unstructured data
#65> At the Stanford AI lab where we met... 80% of enterprise data is unstructured, and traditional platforms can’t handle it You guys came out of an academic lab, so you must know that hypothesis fishing expeditions are not viable. > ... a major commercial bank... couldn’t improve credit risk models because critical data was stuck in PDFs and emails. In this example there will be no improvement to the risk model or wha…
> 80% of enterprise data is unstructured I've seen quotes like this many times. It's silly. I worked at a big bank for over a decade. 95% of the data we cared about was already in a SQL database. Maybe ~80% of our data was "unstructured", but it wasn't stuff we cared about for risk management or other critical functions. > people are not willing to pay more money for less risk I'd disagree here. Banks are willing to…
Who, in your example, put the customers financial data in the SQL database? Because in my part of finance that’s either the customer, or an employee.
Our customers are asking for integration with a lot of their systems (say HR / patrolling), but never ever offer to hook up their accounting system. If we want financial data, we either get a PDF with their audited financial statement or in exceptional cases a custom audited statement (you know, the one where a print of a part of the ledger gets a signature from the CPA for a not insignificant bill).
So I am enthusiastic from a data science point of view. Financial data processing of customer data is / was scarce since limited to what was feasible to manually process. That is nearly in the past.
Re: Launch HN: Trellis (YC W24) – AI-powered workflows for unstructured data
#66I don't understand why you need an LLM for this, wouldn't a simple NER + entity normalization do this at a fraction of the cost?
(congrats on the launch!)
Re: Launch HN: Trellis (YC W24) – AI-powered workflows for unstructured data
#67I've had do some of this recently, as a one-off, to extract the same fields from thousands of scanned documents. I used OpenAI's function calling (via Langchain's https://python.langchain.com/v0.1/docs/modules/model_io/chat... API). Some of the challenges I had: 1. poor recall for some fields, even with a wide variety of input document formats 2. needing to experiment with the json schema (particularly field descript…
FWIW I've seen noticeably better results on (1) and (4) extracting JSON from images via Claude, although (2) and (3) still take effort.
Thanks for sharing.
I'm curious about what types of source documents you tried, and whether you ever suffer from hallucinations?
Re: Launch HN: Trellis (YC W24) – AI-powered workflows for unstructured data
#68Earlier quoted context omitted.
> 80% of enterprise data is unstructured I've seen quotes like this many times. It's silly. I worked at a big bank for over a decade. 95% of the data we cared about was already in a SQL database. Maybe ~80% of our data was "unstructured", but it wasn't stuff we cared about for risk management or other critical functions. > people are not willing to pay more money for less risk I'd disagree here. Banks are willing to…
Yep, and nowadays, banks are already deploying this stuff internally via their own IT teams. They have 1-2 decades of having built up ETL/orchestration talent + infra, and have been growing deals with openai/azure/google/aws/databricks for the LLM bits. Internally, big banks are rolling out hundreds of LLM apps each, and generally have freezes on new external AI vendors due to 'AI compliance risk'. NLP commoditized s…
Am I in another world? (See my response above.) Most of the ‘hundreds of LLM apps’ I see are, well, not very fancy and struggling to keep up on accuracy in comparison to the meatspace solutions they promised to massively outperform.
I agree with your assessment that the IT risk barrier is very high in big corp so that entry might be hard for Trellis. Plus a continuous push afterwards to go back to traditional cloud once their offerings catch up.
Re: Launch HN: Trellis (YC W24) – AI-powered workflows for unstructured data
#69Hey folks. Congrats on the launch. Everyone here knows that it's a really big problem that no one has nailed yet. My 2 cents: 1. It took us (newscatcherapi.com) three years to realize that customers with the biggest problems and with the biggest budgets are the most underserved. The reason is that everyone is building an infinitely scalable AI/LLM/whatever to gain insights from news. In reality, this NLP/AI works qui…
We've been building something similar with https://vlm.run/: we're starting out with documents, but feel like the real killer app will involve agentic workflows grounded in visual inputs like websites. The challenge is that even the best foundation models still struggle a lot with hallucination and rate limits, which means that you have to chain together both OCR and LLMs to get a good result. Platforms like Tesseract work fine for simple, dense documents, but don't help with more complex visual media like charts and graphs. LLMs are great, but even the release of JSON schemas by OpenAI hasn't really fixed 'making things up' or 'giving up halfway through'.
Re: Launch HN: Trellis (YC W24) – AI-powered workflows for unstructured data
#70Getting a lot of love from HN so the demo site and data processing might slow down by quite a bit. We're fixing it right now!