11–20 of 30 posts
Enterprise search is a hard problem, but this looks pretty slick. What's your tech stack? Did you create the integrations from scratch, or use something like Zapier?
This would go a long way to addressing one of the key needs that we’ve been planning around: a central library to manage all the different documents and datasets that are accumulated by different teams. We’ve sketched out an initial solution which looks a lot like Secoda, except focused on csv files, the concept being to check csv data sets into the library, add metadata, and then define how to bridge it into the cen…
We always have a hard time figuring out how to define our source of truth and which tables/graphs to use for measuring our KPIs. Definitely going to reach out about using Secoda... would save us a lot of number wrangling during our weekly syncs.
We have exactly this problem (and have tried to solve it in various ways) and launched Secoda internally at PartnerStack a few days ago. Excited to see it roll-out in the org and build a solid data knowledge base.
Congrats on launching! So how do you compare to a Data Catalog like datahub? https://datahubproject.io/ From the video you looked very similar to them as a metadata consumer and they provide extensive API integrations so you can add basically any set of metadata you want including slack, jira etc. They're also offering a hosted version. Their metadata is indexed into a tuneable ES cluster so you can fiddle with relev…
1. If you're using the DataHub open source solution it requires a data engineer to get the platform up and running and maintained, which can be a fairly expensive cost depending on the salary of the data engineer. Secoda has 15+ no code integrations that can be setup in 5 minutes and is a fully managed solution. We are releasing a metadata API that will be available before the end of the year, in case an organization is using a product that we do not currently integrate with.
2. Acryl (managed version of DataHub) is mainly focused on the data catalog, which they do a great job for. However, they don't provide the questions, dictionary, and visualization components that we provide in addition to the catalog. These additional components of the product add more context around data knowledge, and are also focused on helping non-technical users understand company data. Whereas the data catalog is focused more on helping technical data users understand company data.
3. Also if you're using Acryl, you'll have to get in touch with their team to get a demo of the product. For Secoda, you can signup at https://app.secoda.co and try out a free trial of the product without having to talk with our team. We do offer demos if people are interested though.
We have exactly this problem (and have tried to solve it in various ways) and launched Secoda internally at PartnerStack a few days ago. Excited to see it roll-out in the org and build a solid data knowledge base.
A very cool approach. When I worked at Uber, there were internal tools to query data sources. In practice, I found few people outside data scientists knew how to query the data, or know what was available. I was an engineer and barely used this tool, until a peer DS showed me how to do this. Even then I was overwhelmed by the number of tables to join, knowing what data source contained what data, or knowing which tab…
We have pretty advanced RBAC in Secoda. You can make anyone a viewer, guest, admin or editor in the workspace. Viewers and Editors are only able to see the information. Secondly, we allow you to create "groups" for different functions in the organizations (ie. marketing, sales etc.). You can choose to share any resource with a specific user or group. This works similar to the RBAC that Notion uses, which only means that the right people are seeing the right information in Secoda. Lastly, we allow data teams to create "collections" of information, which can be shared with specific groups or specific users. Without sounding bias, I think this is where Secoda excels as a product.
2. What about PII? Some data needs to be stored, but cannot be viewed except for very, very few people and with a strong audit tail. This is a more specialized case for #1.
We have an ability to auto tag PII on a table and column level. Any PII data won't be viewable without permission from the admin.
3. How do you see the tool "spread" the most within companies? I would assume that easy sharing is how people learn about this, then try it themselves... but would love to hear what you actually see.
Usually the Slack integration is the best way to spread Secoda. With our Slack integration, any employee can search for information by pressing /secoda in Slack. You can also push information from Slack to Secoda and vice versa. This exposes Secoda to new employees in the place they work.
Wait, is it curated, or is it search? To me "search" implies that the tool _discovers_ my stuff and makes it searchable. If I have to tell it about my stuff ("curation"), then that's just a metadata catalog. The reason the distinction matters is that if it's curation-based, the onus is still on the data team to document all relevant assets, which they could already do, and have already demonstrated they don't want to…
What is your strategy to scale out & maintain integrations? Speaking from experience, it's not something that is easy to scale out unless you have a dedicated team whose job is to build them out, or you have some third-party provider like CData providing OOTB connectors for your product.
(On a side note, this looks fantastic. Are you hiring any product folks per chance? I have significant experience tackling this same problem).