Live data from Hacker News

Launch HN: Narrator (YC S19) – a data modeling platform built on a single table

news.ycombinator.com

51–60 of 61 posts

Re: Launch HN: Narrator (YC S19) – a data modeling platform built on a single table

#51
post #47
post #32

Earlier quoted context omitted.

Appreciate the skepticism, since yes, we're a totally different approach. I'll try to address your points in order. Yes, we agree that 99% of the work is determining what the data means. Our structure doesn't magically make things better because of its structure. It's that once you have it analysis / aggregation on top of it becomes substantially less work because you don't have to constantly redo models to answer ne…

If I am understanding correctly, if my BI folks want to report on things like order total or order sub total, or order sub total + shipping, or order sub total + tax, or presumably anything about the contents of the order (SKUs, quantities, per item pricing, price adjustments, coupons and promotions, etc...) instead of capturing a "order submitted" event, we'd have to capture every add to cart, every cart pricing rec…

I think the point is that you can choose your own granularity, and that it tries to make things fast no matter that choice.

Re: Launch HN: Narrator (YC S19) – a data modeling platform built on a single table

#54
post #53

whats the underlying storage engine? tech stack?

We don't use an underlying storage engine. Everything runs directly on your warehouse. We build the activity stream by running queries against the warehouse and writing to a table that we create inside of it. When people build datasets with Narrator we compile everything down to sql and query the warehouse directly.

The tech stack is Python for the backend scheduling and query engine hosted on AWS. For the frontend it's React. We have some internal data stores for managing our own state and a bit of caching - Postgres, S3, ElasticSearch. We use GraphQL a fair bit.

Re: Launch HN: Narrator (YC S19) – a data modeling platform built on a single table

#55

So you offer a free tier, then the next tier is $200 a month? Seems a bit steep does it not? What about a $50 plan for like say 10 million rows?

Thanks for the feedback. Yeah, we can definitely consider it. We haven't totally optimized our pricing yet.

If you (or anyone) uses our free tier and wants to upgrade to something between it and the lowest paid tier just send us a message at support@narrator.ai and we'll set up something for you.

Re: Launch HN: Narrator (YC S19) – a data modeling platform built on a single table

#56

Earlier quoted context omitted.

> activities and how they relate This is the problem with EAV/nosql/schemaless/etc and ultimately the problem I think you are going to have to solve. Instead of using ETL to model how the activities relate and reifying that model as database objects, EAV just kicks the can down the road to the query/BI tool. Sprawl - The BI tool will end up containing most of the real business logic sprawled across many reports. Sing…

Sprawl - YES! I would never put a single time-series table in your BI tool. It is not queryable and you will hate the insane results. - We actually built our own query layer called Dataset to make sure that the dataset is materialized. This way if you put it in your BI tool, you can always go back to the dataset which points direct to the activity stream. Single Source of truth. & traceability - 100%. We really aim t…

I took a deeper look at the Dataset portion of your product this morning and it definitely piqued my interest. It wasn't clear to me from your original post and my initial scan of your site that there was a way to create queries/models/views (whatever folks want to call them, they're essentially the same concept) on top of the single activity table and then either materialize them or integrate them with other services via webhooks or native API integrations. That's definitely super useful. Also, the "Relationship" concept does a nice job of trying to approach joins/window functions in plain english. Query builders are always a difficult UX problem and I think you're onto something interesting. Finally, the validation, identity resolution and spend features are also nice and I could see you adding value via more features in this vein in the future.

The main thing this product strikes me as is "The ETL tool that understands your business". Whereas the domain language of most ETL tools is at the level of DW technologies (rows, columns, schemas, facts, dimensions, indexes, join algos, views, dags, orchestration schedules), the domain language of Narrator is at the level of the business (activities, customers, relationships, spend, etc). In a way it's sort of similar to the old convention over configuration religious war. I could see companies using Narrator for the 80% of ETL that is just plain table stakes in order to compete nowadays and offloading most of the definition and minor customization of this ETL to less technical folks. And maybe in parallel the data engineers would use plain old code to do the last 20% of ETL that is truly proprietary and specific to the business.

Not sure if my biased initial reading of your pitch was off but it seemed like you were focusing heavily on addressing the pain points of the star schema. I've found that most people fall into two camps: either they don't care at all about the kimball star schema world and they're just loading tables however they see fit into their warehouse or they are willing to go to their grave defending the star schema and its variants. In either case, I don't think you gain much by positioning yourself as the antidote to the star schema. I think you could capture customers in both camps by focusing instead on the fact that your ETL tool has a deep understanding of how companies that rely heavily on a web presence work. I think this would also better align you with the ability to increase your customers' revenue as opposed to optimizing engineering/infrastructure concerns which is an easier sell.

Anyway, sorry for the rant. I'm going to shoot you a short email in case you want to connect.

Re: Launch HN: Narrator (YC S19) – a data modeling platform built on a single table

#58

Earlier quoted context omitted.

Sprawl - YES! I would never put a single time-series table in your BI tool. It is not queryable and you will hate the insane results. - We actually built our own query layer called Dataset to make sure that the dataset is materialized. This way if you put it in your BI tool, you can always go back to the dataset which points direct to the activity stream. Single Source of truth. & traceability - 100%. We really aim t…

I took a deeper look at the Dataset portion of your product this morning and it definitely piqued my interest. It wasn't clear to me from your original post and my initial scan of your site that there was a way to create queries/models/views (whatever folks want to call them, they're essentially the same concept) on top of the single activity table and then either materialize them or integrate them with other service…

That is really great to hear! I think you make a great point and we will discuss your recommendations internally to improve our messaging.

I am excited to chat in person.

Re: Launch HN: Narrator (YC S19) – a data modeling platform built on a single table

#59

[Co-founder here of a start-up that provided monitoring / metadata analytics for cloud warehouses] My unsolicited $0.02 - I think your approach is spot on. As a company, you will never have one consistent data set and metrics if you keep building an individual model for each user / use case / etc. And I've seen the explosion of tables and models in real-time. They just keep growing. And how do you even know that the…

This is so great! You see exactly what we see and clearly you have shared similar experiences with dashboards not matching because of wrong table. (The good old "spent 3 weeks debugging an analysis using sales_data and then finally found that sales_data_v2 was built to solve it). Yeah we do something very similar to dbt for taking restructuring the data into a single time-series table. We add things like identity res…

> restructuring the data into a single time-series table. We add things like identity resolution, diffing, incremental update

So is this where the customer still has to do some work? Defining states and transforming their sources into a series of events with these states?

Re: Launch HN: Narrator (YC S19) – a data modeling platform built on a single table

#60
post #59

Earlier quoted context omitted.

This is so great! You see exactly what we see and clearly you have shared similar experiences with dashboards not matching because of wrong table. (The good old "spent 3 weeks debugging an analysis using sales_data and then finally found that sales_data_v2 was built to solve it). Yeah we do something very similar to dbt for taking restructuring the data into a single time-series table. We add things like identity res…

> restructuring the data into a single time-series table. We add things like identity resolution, diffing, incremental update So is this where the customer still has to do some work? Defining states and transforming their sources into a series of events with these states?

Yes, the customer would have to define their activities (e.g. 'page view', 'completed order', 'support ticket opened') and write sql snippets to define them.

https://docs.narrator.ai/docs/activity-transformations describes these scripts and links to a few examples

Post reply on HN