Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Launch HN: Narrator (YC S19) – a data modeling platform built on a single table

news.ycombinator.com

Post reply on HN
Print view View original

11–20 of 61 posts

Re: Launch HN: Narrator (YC S19) – a data modeling platform built on a single table

#11

Post by cedricd »

treis wrote: post #7

So the concept behind this is that every action is written to a single activity stream across the enterprise. Then you query that table when you want to do analytics. Does this mean I need to modify all my other applications to write to this table when an event happens?

That's a great question. No, you don't have to change the any other applications at all. We work from the raw data in a warehouse.

So the typical flow is that your production applications structure their data however they want. From there the data is sent into your warehouse as-is (using an EL tool like Fivetran).

From there you write small sql scripts in Narrator for each activity you'd like to create. Those scripts are responsible from transforming into your format to ours. They're generally fairly short. Our docs describe them here: https://docs.narrator.ai/docs/transformations

Re: Launch HN: Narrator (YC S19) – a data modeling platform built on a single table

#13

Post by cedricd »

chasers wrote: post #10

So event sourcing?

In some respects it's similar, in that our table is time-series, but we're fully on the data analysis side. Data flows from production databases (in any format) into a warehouse, and from there we transform it into the activity stream.

Event sourcing is a great way to model production data that would work really well with Narrator, but it's by no means necessary.

Re: Launch HN: Narrator (YC S19) – a data modeling platform built on a single table

#14

Post by ahmedNarrator »

scapecast wrote: post #6

[Co-founder here of a start-up that provided monitoring / metadata analytics for cloud warehouses] My unsolicited $0.02 - I think your approach is spot on. As a company, you will never have one consistent data set and metrics if you keep building an individual model for each user / use case / etc. And I've seen the explosion of tables and models in real-time. They just keep growing. And how do you even know that the…

This is so great! You see exactly what we see and clearly you have shared similar experiences with dashboards not matching because of wrong table. (The good old "spent 3 weeks debugging an analysis using sales_data and then finally found that sales_data_v2 was built to solve it).

Yeah we do something very similar to dbt for taking restructuring the data into a single time-series table. We add things like identity resolution, diffing, incremental update and computing some cache columns.

Your Crystal Ball is SPOT ON!!! We get 3 kinds of data people. The ones who are like: "THIS WILL NEVER WORK", "Too bad I already built all this" or the "THIS IS THE FUTURE, HOW IS EVERYONE NOT USING IT".

I would love to chat and show you what we have (schedule a demo on our site and it will go to me and we can chat!)

Also, Teaser... When you standardize all of data and you create a consistent way to relating that standardized structure then analysis become very consistent. Imagine a world where your email attribution deep dive can be run by loading a template and point it to your "opened email" activity and your "order activity".... coming soon ... a Narrative Library.

Re: Launch HN: Narrator (YC S19) – a data modeling platform built on a single table

#15

Post by theboat »

scapecast wrote: post #6

[Co-founder here of a start-up that provided monitoring / metadata analytics for cloud warehouses] My unsolicited $0.02 - I think your approach is spot on. As a company, you will never have one consistent data set and metrics if you keep building an individual model for each user / use case / etc. And I've seen the explosion of tables and models in real-time. They just keep growing. And how do you even know that the…

Question for the narrator folks...what about using dbt to create the activity stream?

Re: Launch HN: Narrator (YC S19) – a data modeling platform built on a single table

#16

Post by cedricd »

sroussey wrote: post #5

What are the 11 columns?

Also replying since I wrote this up :)

- activity_id : a unique identifier for the row

- activity : the type of activity (eg 'page_view')

- timestamp : time the activity happened

- customer : the unique customer identifier

Metadata columns

  Three columns for any info we'd like to add to an activity. Eg for a purchased product activity it could be product name. 

 - feature_1
 - feature_2
 - feature_3

 - revenue_impact : the amount of money related to this activity. A completed order activity would have this
 - link : a hyperlink related to the activity ('ticket submitted' might have a link to the ticket in Zendesk)
Additional customer identifier - source and source id are used when you're not entirely sure who the customer is. For example, a 'page view' activity wouldn't know the actual customer, but might have a unique identifier. So the source could be 'segment.io' and source_id could be their generated uuid

- source

- source_id

Re: Launch HN: Narrator (YC S19) – a data modeling platform built on a single table

#19

Post by ahmedNarrator »

theboat wrote: post #15
scapecast wrote: post #6

[Co-founder here of a start-up that provided monitoring / metadata analytics for cloud warehouses] My unsolicited $0.02 - I think your approach is spot on. As a company, you will never have one consistent data set and metrics if you keep building an individual model for each user / use case / etc. And I've seen the explosion of tables and models in real-time. They just keep growing. And how do you even know that the…

Question for the narrator folks...what about using dbt to create the activity stream?

We would love that but Narrator works on any warehouse. To support that we built a query abstraction later that compiles to the flavor of SQL used by the customers warehouse.

(We will open source that query abstraction later with a demo where you can translate Redshift Queries to Snowflake Queries).

Maybe in the future we can get that project into dbt so that dbt models can work on any warehouse as well.

Re: Launch HN: Narrator (YC S19) – a data modeling platform built on a single table

#20

Post by cedricd »

theboat wrote: post #17

I love the paradigm and I think the Narrator team has done a great job so far, but I'm unclear about the business model. Are you still operating as a consultancy, or are you providing tooling?

Sorry for not making that clear :). We're a SaaS product.

You can check out our pricing page here https://www.narrator.ai/pricing

The initial consultancy approach helped us build out the product. Once we could show internally that it made us far faster to analyze data we were ready to launch.

Post reply on HN