Scuba: Diving into Data at Facebook [pdf]
research.fb.com
Scuba: Diving into Data at Facebook [pdf]
1–10 of 44 posts
Re: Scuba: Diving into Data at Facebook [pdf]
#2The article focuses a lot on the implementation, but thankfully you don't need to worry about that when using it. The flexible/quick/easy UI is what seals the deal for me, combined with the fast query times and realtime data -- you can use it to query things that happened literally seconds ago. On the React team we use it to collect all dev-time JS warnings from React that our engineers see so that we can easily track what the most high-firing messages are and how frequently they occur.
I haven't tried them extensively, but honeycomb.io and Interana are both Scuba-inspired products by ex-Facebookers. If this tool sounds at all interesting to you, I'd definitely look into using them.
Re: Scuba: Diving into Data at Facebook [pdf]
#3Re: Scuba: Diving into Data at Facebook [pdf]
#4Seems very similar to BigQuery. I wonder how the architectures compare.
^ This is Google's equivalent (mentioned at the end of the paper in "Related Work")
Re: Scuba: Diving into Data at Facebook [pdf]
#5Re: Scuba: Diving into Data at Facebook [pdf]
#6At Parse I was able to diagnose some seriously complex performance issues for customers by breaking splitting queries (precomputed in logs) into families and looking at probability distributions. It was amazing. Every time the customer told me XYZ was the problem I could send them a screenshot and refocus the conversation where the data sent us.
Some ex Parse employees left FB to build a visualization tool based on the same white papers (honeycomb.io). I'm really hoping to add that to my tool belt again.
Re: Scuba: Diving into Data at Facebook [pdf]
#7I'm so happy to see this posted. I've been at Facebook for two years, and Scuba is hands-down one of my most favorite internal tools we have (and we have a lot of good ones). The article focuses a lot on the implementation, but thankfully you don't need to worry about that when using it. The flexible/quick/easy UI is what seals the deal for me, combined with the fast query times and realtime data -- you can use it to…
Re: Scuba: Diving into Data at Facebook [pdf]
#8I'd also like to echo what spicyj said about the UI, it is very usable. It was common for managers, PMs, and even people in completely non-technical business roles to use Scuba. It was my favorite internal product at FB and the one I miss the most.
Re: Scuba: Diving into Data at Facebook [pdf]
#9Re: Scuba: Diving into Data at Facebook [pdf]
#10This is a special-purpose time-series data warehouse and a UI for querying it. If you are not Facebook, it is almost always better to do projects like this by using a standard data warehouse like Redshift or BigQuery, a queue like Kinesis, and a BI tool like Looker or Tableau. Your data won't be quite as real-time and your queries won't be quite as fast, but it will take much less engineering effort and you'll be abl…