Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: ClearBrain (YC W18) – Automated Causal Analytics

clearbrain.com

Post reply on HN
Print view View original

11–18 of 18 posts

Re: Show HN: ClearBrain (YC W18) – Automated Causal Analytics

#11

Post by newtothebay »

Can you talk about how to infer causality without running an experiment? From your description, "real-time processing + auto ML + algorithm" still sounds very much observational to me.

I'm asking not as knock against your service, but genuine curiosity about how you manage to solve this incredibly hard problem.

EDIT: From your white paper, it looks like you're running a regression that controls for a bunch of confounders. You also interact the treatment variable with those confounders to get the heterogeneous treatment effect.

My concern with that is that we're not controlling for unobservable confounders, which make causal inference so difficult. If we assume that controlling for observable confounders is enough (we shouldn't!), then correlation and causation are the same.

White paper: https://blog.clearbrain.com/posts/introducing-causal-analyti...

Re: Show HN: ClearBrain (YC W18) – Automated Causal Analytics

#12

Post by bmahmood »

newtothebay wrote: post #11

Can you talk about how to infer causality without running an experiment? From your description, "real-time processing + auto ML + algorithm" still sounds very much observational to me. I'm asking not as knock against your service, but genuine curiosity about how you manage to solve this incredibly hard problem. EDIT: From your white paper, it looks like you're running a regression that controls for a bunch of confoun…

Yep, you're correct that we're using observational studies via a regression to remove confounders and estimate treatment effects. Our confounders are synthetically generated based on the observable variables - we can only make projections of course on digital signals our customers send us (we only use first party data). We are working to incorporate actual experiment data into the algorithm over time as well, to get even closer to the true causal treatment effect.

Re: Show HN: ClearBrain (YC W18) – Automated Causal Analytics

#13

Post by noobhacker »

bmahmood wrote: post #12
newtothebay wrote: post #11

Can you talk about how to infer causality without running an experiment? From your description, "real-time processing + auto ML + algorithm" still sounds very much observational to me. I'm asking not as knock against your service, but genuine curiosity about how you manage to solve this incredibly hard problem. EDIT: From your white paper, it looks like you're running a regression that controls for a bunch of confoun…

Yep, you're correct that we're using observational studies via a regression to remove confounders and estimate treatment effects. Our confounders are synthetically generated based on the observable variables - we can only make projections of course on digital signals our customers send us (we only use first party data). We are working to incorporate actual experiment data into the algorithm over time as well, to get…

Awesome! Could you speak a bit about what you have in mind to incorporate actual experiment data into the algorithm?

Re: Show HN: ClearBrain (YC W18) – Automated Causal Analytics

#16

Post by gingerlime »

bmahmood wrote: post #10
gingerlime wrote: post #9

Hi Bilal! I think I reached out to you in early 2018. Any news about the integration without segment (e.g. Amplitude in our case)? And what’s the pricing model? Couldn’t find much on the site (maybe it’s more limited on mobile?)

Thanks for reaching out again! We're prioritizing support for Segment at this time, but hope to add other integrations next year. Our analytics product is completely free, so getting set up on our joint solution with Segment shouldn't be too expensive. :)

Thank you. What’s your monetization strategy then? (Were you acquired by Segment or something? honest question)

Re: Show HN: ClearBrain (YC W18) – Automated Causal Analytics

#17

Post by aditiyaa1 »

bmahmood wrote: post #6
aditiyaa1 wrote: post #5

Earlier quoted context omitted.

Hi Bilal, Thanks for the overview of the product. This is a really important business problem to solve for many marketing teams. Just using this to prioritize A/B tests in itself pretty valuable. But one of the concerns around this approach is the un-reliability of causal analysis to estimate true effects. The link below refers to a study done at FB that shows observational studies could be erroneous in estimating ef…

Thanks for the great feedback! Yes, some of these limitations expressed in the study are true in the case of ClearBrain - namely we are leveraging observational studies at this time as a prioritized ranking algorithm for which behaviors are most important, but the actual effect sizes themselves may be variable. We're working on improvements, as well as incorporating actual experiment data into our algorithm to make i…

Thanks! What is your strategy around incorporating actual experiment data? Not sure I fully follow here.

Re: Show HN: ClearBrain (YC W18) – Automated Causal Analytics

#18

Post by pfbtgom »

I think you have an interesting product, but I'm having serious issues with your marketing.

Extraordinary claims require extraordinary evidence. How many of your estimated treatment effects have been supported by experiments? Do you have experiments demonstrating that your model generalizes? How accurate are your estimates compared to experimental results?

It's ironic that you're marketing a causal + analytics product without any data. Generating a narrative and basing it off of observational data is the typical trap that many causal claims fall into. Portraying yourselves as statistical experts and pushing unsubstantiated claims is misleading bordering on unethical.

Post reply on HN