Live data from Hacker News

Outlier Detection in SQL

periscopedata.com

11–20 of 36 posts

Re: Outlier Detection in SQL

#11
post #8
post #3

Is it just me or is Periscope allergic to saying what databases it supports? I can find nothing on the website, and I am really hesitant to sign up for more info once companies started actually calling my phone after I did that. Also, none of those queries will work on MySQL.

They support Postgres, Microsoft SQL Server, Redshift, and Oracle I believe.

I know that they also support MySQL. We have Periscope connected to a MySQL database.

Re: Outlier Detection in SQL

#12
post #3

Is it just me or is Periscope allergic to saying what databases it supports? I can find nothing on the website, and I am really hesitant to sign up for more info once companies started actually calling my phone after I did that. Also, none of those queries will work on MySQL.

Is it just me or is Periscope allergic to showing the "Pricing" page? (SaaS anti-pattern) Or is it free?

The blog content is good though

Re: Outlier Detection in SQL

#13
post #12
post #3

Is it just me or is Periscope allergic to saying what databases it supports? I can find nothing on the website, and I am really hesitant to sign up for more info once companies started actually calling my phone after I did that. Also, none of those queries will work on MySQL.

Is it just me or is Periscope allergic to showing the "Pricing" page? (SaaS anti-pattern) Or is it free? The blog content is good though

You can get a free trial, but it's most definitely not free (on the contrary, the price is a lot higher than i would've expected).

Re: Outlier Detection in SQL

#14
post #3

Is it just me or is Periscope allergic to saying what databases it supports? I can find nothing on the website, and I am really hesitant to sign up for more info once companies started actually calling my phone after I did that. Also, none of those queries will work on MySQL.

Periscope Data cofounder here. I wouldn't say we're "allergic" necessarily. ;) Like a lot of startups, we're very young and haven't gotten around to building out the website as much as we'd like to.

`jplitz and `ajones are right. As of now we support MySQL, Postgres, Amazon Redshift, Vertica, SQL Server, Oracle, MemSQL, Sybase, Exasol and Google BigQuery. We add more all the time.

Re: Outlier Detection in SQL

#16
post #12
post #3

Is it just me or is Periscope allergic to saying what databases it supports? I can find nothing on the website, and I am really hesitant to sign up for more info once companies started actually calling my phone after I did that. Also, none of those queries will work on MySQL.

Is it just me or is Periscope allergic to showing the "Pricing" page? (SaaS anti-pattern) Or is it free? The blog content is good though

I won't blow it by sharing the pricing they quoted me, but I thought it was pretty reasonable.

That said, I do have one major gripe with their sales process: Their free trial lasts 7 days. There is no way a company can evaluate a completely new BI system in a week. I begged and pleaded and they would not extend the trial.

I'm not just saying this, but I really really do think we would have bought it if we could have had 30 days. I just can't build enough organizational momentum to spend money on a new system in 7 days.

Re: Outlier Detection in SQL

#17
These are some cool SQL tricks! I like it.

The big caveat with the standard deviation technique is that it assumes a normal distribution. Many datasets are not actually distributed normally (power-law, Poisson, beta, etc, etc) and so the technique won't work. It's a much harder problem to 'generically' detect outliers without knowledge of the underlying distribution.

I don't have any idea how to do it (though a former colleague came up with nice idea of building a histogram and searching for values that occurred after some number of empty bins, implying an outlier). Is there an accepted state-of-the-art for general-purpose outlier detection? Or is that such a broad question as to be meaningless?

Re: Outlier Detection in SQL

#18
I would like to know about more sophisticated techniques for outlier detection. These are stat 101 level. Z scores? You can get into a lot of trouble assuming a normal distribution.

What are credit card companies doing? What's the best way to combine multiple variables that are predictive of an event for outlier detection? Is there a simple framework to automate reporting of these events in real-time?

One way to model this might be to treat the outcome ("Is this event an outlier?") as a 0/1 variable and use one of the many ways to model that type of data–random forests, logistic regression, neural networks, etc. The problem is that this isn't really "outlier" detection anymore.

Re: Outlier Detection in SQL

#20
post #12

Earlier quoted context omitted.

Is it just me or is Periscope allergic to showing the "Pricing" page? (SaaS anti-pattern) Or is it free? The blog content is good though

I won't blow it by sharing the pricing they quoted me, but I thought it was pretty reasonable. That said, I do have one major gripe with their sales process: Their free trial lasts 7 days. There is no way a company can evaluate a completely new BI system in a week. I begged and pleaded and they would not extend the trial. I'm not just saying this, but I really really do think we would have bought it if we could have…

Sorry about that! Just sent you an email.
Post reply on HN