Show HN: Open-source infra for building embedded data pipelines

Hey HN!

We are building *open source infrastructure for deploying customer-facing data pipelines.*

Here’s our repo https://github.com/pipebird/pipebird and website https://pipebird.com/.

Pipebird (YC W22) is designed to enable companies that generate important data to offer secure data pushes to their customers’ warehouses, directly from their products.

Our team was previously building in fintech, where we heard from many of our peers that their customers wanted data pushed directly to their warehouses. Customers wanted to bring data into their source of truth without having to maintain custom built pipelines or introduce security risks by contracting a third-party ETL/ELT provider.

After seeing Stripe https://stripe.com/data-pipeline and customer.io https://customer.io/data-warehouse recently invest in building out their own native data sharing products, we realized that many SaaS companies could better support their customers and even generate additional revenue by offering native data pipelines.

Our goal with Pipebird is to make creating a reliable data pipeline as simple as pressing a button from a vendor's dashboard.

With the current iteration of the product, data can be selected from a number of sources (ex: Postgres, MySQL, CockroachDB, etc.), customers can configure pipelines and optionally apply transformations (like type casting), and data can be periodically synced directly to customers’ warehouses (ex: Snowflake). We’re actively adding sources/destinations and would appreciate any feature requests.

Here's a 2 min demo of the product https://www.loom.com/share/c7a7e4b4e57c4015b533fd754c510b2e

Pipebird is open source (MIT license) so that any developer can use it. Our aim is to not charge individual developers - we make money selling paid plans that include features like multiple projects, user permissions, additional security features, managed infra, support, etc.

Give us a whirl: https://github.com/pipebird/pipebird. We’d love your feedback and will be here to answer any questions!

Show HN: Open-source infra for building embedded data pipelines
github.com

Hey folks

The product looks great. I had faced such situations in past with different tools.

You should make the PM and Dev community aware of this tool to get better leads and usecases.

Wish you best for the future.

Thanks - much appreciated! We will be posting in some other areas to spread the word.

Thanks for posting. Pipebird sounds really useful. We're running a conference on November 15th called Open Source Analytic Software 2022. It's free and covers all parts of the open source analytic stack. Talks on innovative new tech are very welcome. I would like to invite you to submit a talk. CfP is here: https://altinity.com/osa-con/.

Happy to help you deploy some pipelines once you're ready!