Live data from Hacker News

Launch HN: Secoda (YC S21) – Searchable Company Data

news.ycombinator.com

1–10 of 30 posts

Launch HN: Secoda (YC S21) – Searchable Company Data

#1
Hey HN, we’re Etai and Andrew, and together with our team, we’re building Secoda (https://www.secoda.co). We do search (that actually works!) for your company data. The name Secoda stands for "Searchable Company Data".

As a company grows, so does its data. Tables, metrics, queries, and dashboards often become isolated and are difficult to find. Even with great practices, organizations still struggle to get value out of their data - up to 73% of all enterprise data goes unused. One of the big contributors to this problem is that organizations create data silos by not documenting and centralizing their data knowledge in a single place where every employee can access information about data.

Andrew and I experienced this problem first hand at the last company we worked at. Andrew led the Product team and I led the Operations team and found that it was extremely difficult to find, understand and use data without looping in someone on the data team to help. The problem was that we only had 1 employee on the data team who supported over 100 employees asking questions about how to find and use company data, which meant that it would take around 2 weeks to get an answer to any data request.

Other data management tools focus on listing all data resources, regardless of their relevance or accuracy - you generally just get a list of what's available, but not in a form that's very meaningful. We adopted some of these tools in our last jobs but found that they created an overwhelming index of too many tables, dashboards and queries that weren’t relevant to most employees. This meant that even after adopting a tool to solve the problem, most employees still couldn’t use them to find, understand and use data.

Our approach to solving this problem is to build Secoda as a tool that helps data teams curate metadata for less technical employees. Instead of listing every resource, data teams can use our tool to curate and document data for specific departments or roles. As a result, employees who are less familiar with data will not be overloaded by information that is irrelevant or too technical. Our goal is basically to be like Google search for in-company data. You enter what you need and you get back the relevant information. We integrate into databases, data warehouses, BI, and transformation tools and offer both an on-prem and cloud-hosted deployment.

Over the last six months, our team has been improving our product closely with our early adopters to build a better product. Today, we’re excited to share the launch of our self-service product with the HN community. You can now sign up to Secoda, connect your database or data warehouse and start using Secoda without a sales call. We offer a free 14-day trial (no credit card required). After the free trial, we charge per editor, per month. If you’d like, you can also take a look at this video of us setting up our Secoda workspace: https://www.loom.com/share/f41b317441554a36930b9cfe4c91a45f.

We're also hiring for a number of roles, which you can find here: https://www.workatastartup.com/companies/secoda.

We’d love to hear about your experiences with data discovery and any ideas/feedback/questions you might have about what we’re building!

Re: Launch HN: Secoda (YC S21) – Searchable Company Data

#3
This would go a long way to addressing one of the key needs that we’ve been planning around: a central library to manage all the different documents and datasets that are accumulated by different teams.

We’ve sketched out an initial solution which looks a lot like Secoda, except focused on csv files, the concept being to check csv data sets into the library, add metadata, and then define how to bridge it into the central data store.

I’ll dig further into the website, it looks like you’ve done a lot of good work avoiding repeatedly addressing the same questions!

Re: Launch HN: Secoda (YC S21) – Searchable Company Data

#4
A very cool approach. When I worked at Uber, there were internal tools to query data sources. In practice, I found few people outside data scientists knew how to query the data, or know what was available. I was an engineer and barely used this tool, until a peer DS showed me how to do this. Even then I was overwhelmed by the number of tables to join, knowing what data source contained what data, or knowing which tables I had access to query or join on.

A few questions:

1. How do you go about permission? This was a major question at Uber (where permission were put in place early enough). Especially with GDPR and other regulations, you cannot have anyone access any data.

2. What about PII? Some data needs to be stored, but cannot be viewed except for very, very few people and with a strong audit tail. This is a more specialized case for #1.

3. How do you see the tool "spread" the most within companies? I would assume that easy sharing is how people learn about this, then try it themselves... but would love to hear what you actually see.

Re: Launch HN: Secoda (YC S21) – Searchable Company Data

#6
Congrats on launching!

So how do you compare to a Data Catalog like datahub? https://datahubproject.io/

From the video you looked very similar to them as a metadata consumer and they provide extensive API integrations so you can add basically any set of metadata you want including slack, jira etc. They're also offering a hosted version.

Their metadata is indexed into a tuneable ES cluster so you can fiddle with relevance etc to your hearts content.

What's your big differentiator?

Re: Launch HN: Secoda (YC S21) – Searchable Company Data

#9
Wait, is it curated, or is it search? To me "search" implies that the tool _discovers_ my stuff and makes it searchable. If I have to tell it about my stuff ("curation"), then that's just a metadata catalog.

The reason the distinction matters is that if it's curation-based, the onus is still on the data team to document all relevant assets, which they could already do, and have already demonstrated they don't want to.

Now, it could still be a good metadata catalog! Most of what's out there is bad. But if that's what you're shooting for, pitching it as "search" will be confusing.

Re: Launch HN: Secoda (YC S21) – Searchable Company Data

#10

How is this different than ThoughtSpot?

Secoda is different from ThoughtSpot in a few ways:

1. It is likely that you'll need to setup a 1:1 meeting with a ThoughtSpot expert to help get your company up and running on the software. Our goal with Secoda is to be a self-service platform that is designed for any company, large or small, to get their data knowledge base setup in 5 minutes.

2. ThoughtSpot's price point is typically in the six figure range, which is much higher than Secoda's price point that starts at $29/editor on the platform.

3. Thoughtspot's core focus is providing answers to data questions through visualizations. Secoda takes a more comprehensive approach to documenting data knowledge. In addition to having visualizations that help answer questions, we also provide a shared data dictionary for defining metrics, as well as a catalog that can store tables, dashboards, jobs, and many other data resources.

Post reply on HN