Live data from Hacker News

Launch HN: Trellis (YC W24) – AI-powered workflows for unstructured data

news.ycombinator.com

11–20 of 124 posts

Re: Launch HN: Trellis (YC W24) – AI-powered workflows for unstructured data

#11

Nice! How's accuracy of produced data?

Exact accuracy depends on the domain and tasks. Processing emails will naturally have higher accuracy than 150+ pages of private credit documents. Generally, we see 95%+ accuracy out of the box and can go up to 99%+ with fine-tuning, and human in the loop validation.

Re: Launch HN: Trellis (YC W24) – AI-powered workflows for unstructured data

#13
post #9

Congratulations on launching! Trellis looks amazing ... but only if it works well enough, i.e., if the rate of edge cases that trip up the service consistently remains close to 0%. Every organization in the world needs and wants this, like, right now. If you make it work well enough, you'll have customers knocking on your door around the clock. I'm going to take a look. Like others here, I'm rooting for you guys to s…

Maintaining the right level of accuracy across different domain is quite hard and something we spend a lot of time on. The accuracy bar tends to be quite high for financial services so we're adding some validation steps and checking to make sure any errors get caught beforehand.

Thank you. Yes, I'm not at all surprised to hear that.

The biggest challenge I see for you guys is that your best customer prospects, i.e., those organizations which need this most urgently and are willing to pay the most for it are the ones already spending gobs of money to do it with human labor because mistakes are too costly, so they need at least human-level performance.

As you know, current-generation LLMs/LMMs are not yet reliable enough to do it on their own. They need all the help they can get -- sanity data checks, post-processing logic, ensembles of models, organization into teams of agents, etc., etc. -- I'm sure you're looking at all options.

Absent human beings in the loop, you're at the frontier of LLM/LMM research.

If you pull it off, you'll make megabucks.

Re: Launch HN: Trellis (YC W24) – AI-powered workflows for unstructured data

#15
Has Trellis explored partnerships or integrations with major ERP systems or existing ETL pipelines? The ability to seamlessly fit into existing enterprise architectures could be a significant competitive advantage and a compelling value proposition for large enterprises looking to modernize their data infrastructure.

Re: Launch HN: Trellis (YC W24) – AI-powered workflows for unstructured data

#16
post #2

Congrats on launching. What model or AI you use underneath?

We use a combination of fine-tune LLMs models that're specialized at extraction, data validation and parsing and large foundational models for more general reasoning tasks. Model routing architecture has been quite interesting to explore.

Have you tried the Structured Output feature that OpenAI released last week?

Re: Launch HN: Trellis (YC W24) – AI-powered workflows for unstructured data

#19
> At the Stanford AI lab where we met... 80% of enterprise data is unstructured, and traditional platforms can’t handle it

You guys came out of an academic lab, so you must know that hypothesis fishing expeditions are not viable.

> ... a major commercial bank... couldn’t improve credit risk models because critical data was stuck in PDFs and emails.

In this example there will be no improvement to the risk model or whatever, because 19/20 times there will be no improvement. In an academic setting this is seen as normal, but in a business setting with no executive champions, only product managers, this will be seen as a failure, and it will be associated with you and your technology, which is bad.

Unfortunately these people are not willing to pay more money for less risk. What they want is a base consulting cost (i.e., a non-venture business) to identify the lowest risk, promotion worthy endeavor, and then they want to pay as little as possible to achieve that. In a sense, the kind of customers who need unstructured data ETLs are poorly positioned to use such a technology, because they don't value technology generally, they aren't forward looking.

Assembling attractive websites that are really features on top of Dagster? There's a lot of value in that. Question is, are people willing to pay for that? Anyone can make attractive Dagster UIs, anyone can do Python glue. It's very challenging to differentiate yourselves, even when you feel like you have some customers, because eventually, one of those middlemen at BankCo are going to punch your USP into Google, and find the pre-existing services with huge account management teams (i.e., the hand holding consulting business people really pay for) that outpace you.

Re: Launch HN: Trellis (YC W24) – AI-powered workflows for unstructured data

#20

> At the Stanford AI lab where we met... 80% of enterprise data is unstructured, and traditional platforms can’t handle it You guys came out of an academic lab, so you must know that hypothesis fishing expeditions are not viable. > ... a major commercial bank... couldn’t improve credit risk models because critical data was stuck in PDFs and emails. In this example there will be no improvement to the risk model or wha…

> 80% of enterprise data is unstructured

I've seen quotes like this many times. It's silly. I worked at a big bank for over a decade. 95% of the data we cared about was already in a SQL database. Maybe ~80% of our data was "unstructured", but it wasn't stuff we cared about for risk management or other critical functions.

> people are not willing to pay more money for less risk

I'd disagree here. Banks are willing to pay money to reduce risk, it's just unlikely to come from scraping data out of PDFs with an LLM because they've already done this if it's worth it.

Post reply on HN