Former Scuba dev, Okay Zed, wrote http://snorkel.logv.org/ . It's open source.
I gotta say, I don't think a snorkel will help the poor person in your logo.
Scuba: Diving into Data at Facebook [pdf]
31–40 of 44 posts
Re: Scuba: Diving into Data at Facebook [pdf]
#32Re: Scuba: Diving into Data at Facebook [pdf]
#33Re: Scuba: Diving into Data at Facebook [pdf]
#34The engineer who built this, Lior Abraham, went on to start data analytics startup Interana (YC W13).
Interana draws much of it's inspiration from Scuba combined with learnings of analyzing massive amounts of data at scale.
Many high-growth companies use Interana for behavioral analysis of their event logs (Asana, Reddit, Imgur, Nextdoor, Bing, Azure, Tinder, SurveyMonkey, Sonos...).
Re: Scuba: Diving into Data at Facebook [pdf]
#35How does Scuba differ from Presto which is also developed by Facebook? It seems that it stores data in-memory and have data expiration feature but also has many common features such as SQL and distributed processing.
This was the reasoning behind a very key architectural decision at Interana that makes it different than Scuba - instead of developing an in memory system, Interana created a custom data store that is heavily optimized around using spinning disk and CPU cache. This makes it incredibly fast and less expensive to operative massive clusters at scale.
Re: Scuba: Diving into Data at Facebook [pdf]
#36Re: Scuba: Diving into Data at Facebook [pdf]
#37Earlier quoted context omitted.
I really miss the awesome tooling like Scuba since I left.
I'll second the post above -- if you miss Scuba, honeycomb.io is for you. https://honeycomb.io/blog/2016/11/honeycomb-faq-in-140-chars...
Re: Scuba: Diving into Data at Facebook [pdf]
#38There's also a blog post about it here https://www.facebook.com/notes/facebook-engineering/under-th... My YC company https://www.interana.com took a lot of lessons from this and is doing something that I think is even better for cos like Reddit, Sonos, Comcast, Bing. You can sign up if you'd like a demo :)
Re: Scuba: Diving into Data at Facebook [pdf]
#39I wonder how often the data is inaccurate given the potentially low sample size?
Re: Scuba: Diving into Data at Facebook [pdf]
#40So it samples data less than or equal to a second old and at a rate determined by the person making the query? I wonder how often the data is inaccurate given the potentially low sample size?
There is no single system-wide imposed sampling rate, so it's up to you to set the sampling rate based on what sort of queries you want to be able to do with good enough accuracy. We have 1:1 rate data for some things (say errors served on a particular service), while a ten or a hundred thousand to one data for other things where there are, say, tens of millions of log lines per second.