Congrats on the HN launch GrowthBook team! Love what you guys are building!
Launch HN: GrowthBook (YC W22) – Open-source feature flagging and A/B testing
21–30 of 47 posts
Re: Launch HN: GrowthBook (YC W22) – Open-source feature flagging and A/B testing
#22Nice launch. Will Growthbook one day support "Feature Rollouts" like statsig? https://www.youtube.com/watch?v=aFmSVP5Jxmc A lot of big companies need to gradually introduce a new feature to an audience while also seeing how that feature impacts their metrics. If they go with just feature flagging, they have to build their own logging and analysis. But if they go with experiments, then they're overly constrained by th…
Yeah, Statsig is building a really cool product. There's definitely a gap in GrowthBook right now, like you mentioned, between pure feature flags and experiments. We're working on some light-weight event tracking features to try and bridge that gap. So you'll be able to see how many people are using the features, which values they are getting, and easily integrate with tools like DataDog and NewRelic for performance monitoring.
Re: Launch HN: GrowthBook (YC W22) – Open-source feature flagging and A/B testing
#23This looks great, but if I can make a suggestion, a guide in your docs on how to use this with Matomo[1] would go far. Growthbook provides Feature Flag and A/B Testing, but you have to bring your own analytics. Matomo provides those analytics and is also open source and self hosted, and would be an excellent match. There's been an issue in about integrating since September[2], and having a guide with SQL examples for…
Thanks for the suggestion. We added MySQL support so we can work with Matomo now, but we're definitely lacking in documentation and tutorials. There's a similar issue with people using GA4, Snowplow, Segment, and anything else with a pre-defined schema. It should be way easier to use these systems with GrowthBook. Like there's no reason to even write SQL most of the time if we know the Matomo table structure in MySQL. We're hoping to solve that issue in the next few weeks.
Re: Launch HN: GrowthBook (YC W22) – Open-source feature flagging and A/B testing
#24Nice launch. Will Growthbook one day support "Feature Rollouts" like statsig? https://www.youtube.com/watch?v=aFmSVP5Jxmc A lot of big companies need to gradually introduce a new feature to an audience while also seeing how that feature impacts their metrics. If they go with just feature flagging, they have to build their own logging and analysis. But if they go with experiments, then they're overly constrained by th…
Yeah, Statsig is building a really cool product. There's definitely a gap in GrowthBook right now, like you mentioned, between pure feature flags and experiments. We're working on some light-weight event tracking features to try and bridge that gap. So you'll be able to see how many people are using the features, which values they are getting, and easily integrate with tools like DataDog and NewRelic for performance…
What's particularly interesting is Statsig essentially separated the concept of "A/B test" from experiment. Compare https://statsig.com/features/feature-gates and https://statsig.com/features/experiments
I think this isn't just marketing; I think this reflects a deeper understanding that you can do A/B testing on top of a feature flag and not consider it an experiment. It's an innovation that reduces inter-organizational strife on the decision to launch a feature to all users, i.e., ship.
If everything with automated metric analysis is considered an experiment, then the shipping decision only be made after analysis is performed. That leads to tension when Product Managers have already decided to ship and will only rollback if something hurts their metric. Data Scientists will then question why their "experiment" is only running for a day, and then question the decision to ship entirely. You can imagine how many meetings a misdefinition spawns.
The sooner you address these definitions, I think the happier you'll find your enterprise users.
Re: Launch HN: GrowthBook (YC W22) – Open-source feature flagging and A/B testing
#25Looks great. I think every hypergrowth startup I've worked at has gone through the use commercial tool => tool gets crazy expensive as you scale => "let's just roll our own, how hard can it be?" => "oh crap, this actually is more complex than we thought" cycle. There are _so many_ types of services that were similarly disrupted by solid open-source implementations due to their price: business intelligence (Metabase),…
Yeah, that's our exact experience as well. I was super surprised to find out how much of a graveyard open source A/B testing was. Intuit (Wasabi), Facebook (PlanOut), and SeatGuru (Sixpack) open sourced their internal platforms years ago, but they are all abandoned now.
Ah, I forgot about Sixpack. Brings me back to my very first feature launches ever :')
Re: Launch HN: GrowthBook (YC W22) – Open-source feature flagging and A/B testing
#26Growthbook is awesome! We have used the self-hosted version a few months i my company now, and are very happy about it. Talented people behind it, and issues are resolved quickly.
Makes a lot of sense with an experimentation tool that just works on top of your existing customer data. This is the way to go for data tools in my opinion. Makes it so much easier to adopt. What we have done, is just exposing views on top of our regular click-stream data. With this approach, we can focus on having one data source of high quality, in stead of scattering data around to multiple third party tools. And we don't have to deal with lot's of data processing agreements and so on. I hope more and more data tools moves in this direction. Sending data to a third party is starting to become old school :)
Re: Launch HN: GrowthBook (YC W22) – Open-source feature flagging and A/B testing
#27Been using GrowthBook for a few weeks now at Ping Labs and man, this is exactly what we needed.
We're a small startup moving incredibly quick. I was shocked at how much setup time it took to get a simple feature flag into our app with other services like Darkly. We had Growthbook working in roughly 15 minutes with under 10 lines of code changed.
Feels a lot like the "a-ha!" moment I had with Plausible.io. This is all we will need for a long, long time.
Re: Launch HN: GrowthBook (YC W22) – Open-source feature flagging and A/B testing
#28Off topic. @dang, may I suggest making the submission text a bit darker, something like #444 to #555 which is between the current rgb(130,130,130) and black that's used for title and links? Here's the reasons:
- why is the submission text lighter than comments?
- rgb(130,130,130) on #F6F6EF is a bit hard to read
- rgb(130,130,130) is or is too similar to downvoted text color
- rgb(130,130,130) is the same color as visited links, so it makes links users visited not appear as links, like the link in "We did a Show HN 6 months ago (https://news.ycombinator.com/item?id=28088882, ..." to me
Re: Launch HN: GrowthBook (YC W22) – Open-source feature flagging and A/B testing
#29Off topic. @dang, may I suggest making the submission text a bit darker, something like #444 to #555 which is between the current rgb(130,130,130) and black that's used for title and links? Here's the reasons: - why is the submission text lighter than comments? - rgb(130,130,130) on #F6F6EF is a bit hard to read - rgb(130,130,130) is or is too similar to downvoted text color - rgb(130,130,130) is the same color as vi…
[deleted]
Re: Launch HN: GrowthBook (YC W22) – Open-source feature flagging and A/B testing
#30Off topic. @dang, may I suggest making the submission text a bit darker, something like #444 to #555 which is between the current rgb(130,130,130) and black that's used for title and links? Here's the reasons: - why is the submission text lighter than comments? - rgb(130,130,130) on #F6F6EF is a bit hard to read - rgb(130,130,130) is or is too similar to downvoted text color - rgb(130,130,130) is the same color as vi…
Email that stuff in, there's no '@dang' notification or anything.