Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

FeatureBase: Open-Source, Real-Time Database Built on Roaring Bitmaps

github.com

Post reply on HN
Print view View original

1–10 of 14 posts

Re: FeatureBase: Open-Source, Real-Time Database Built on Roaring Bitmaps

#3

Post by clowen »

elina123 wrote: post #2

any ideas on real life Use Cases?

I am planning to use it in a project to make the new Congressional API data more approachable for people.

https://api.congress.gov/#/

Hopefully make it easy to find all the bills that your specific congress person was involved in for example.

Re: FeatureBase: Open-Source, Real-Time Database Built on Roaring Bitmaps

#4

Post by jaffee »

elina123 wrote: post #2

any ideas on real life Use Cases?

many! It was originally developed for marketing use cases- helping marketers understand up-to-date use her behavior and find interesting segments.

But really it's useful anytime you need low latency analytics on fresh data.

Re: FeatureBase: Open-Source, Real-Time Database Built on Roaring Bitmaps

#5

Post by JAA1337 »

I've seen bitmaps mentioned a number of times lately. I must admit it is not something I am all that familiar with. Can someone explain to me why bitmaps are more valuable than standard column oriented databases?

I havn't wrapped my head around how this helps speed up queries while data is being ingested.

Re: FeatureBase: Open-Source, Real-Time Database Built on Roaring Bitmaps

#7

Post by kurosknight »

JAA1337 wrote: post #5

I've seen bitmaps mentioned a number of times lately. I must admit it is not something I am all that familiar with. Can someone explain to me why bitmaps are more valuable than standard column oriented databases? I havn't wrapped my head around how this helps speed up queries while data is being ingested.

They are useful for categorical variables. For example, is a record in the "Likes motorcycles" category? They are fast because (well, one reason) bitwise logical operations are very fast for CPUs to do.

Adtech is an example of a sector that benefits from this...they slice and dice datasets a lot to target ad campaigns and such. Being able to do that quickly is useful.

Re: FeatureBase: Open-Source, Real-Time Database Built on Roaring Bitmaps

#8

Post by JAA1337 »

kurosknight wrote: post #7
JAA1337 wrote: post #5

I've seen bitmaps mentioned a number of times lately. I must admit it is not something I am all that familiar with. Can someone explain to me why bitmaps are more valuable than standard column oriented databases? I havn't wrapped my head around how this helps speed up queries while data is being ingested.

They are useful for categorical variables. For example, is a record in the "Likes motorcycles" category? They are fast because (well, one reason) bitwise logical operations are very fast for CPUs to do. Adtech is an example of a sector that benefits from this...they slice and dice datasets a lot to target ad campaigns and such. Being able to do that quickly is useful.

So are you saying that the data is stored in categories which allows for those types of lookups to run faster? Do you have specifics on how the design of a bitmap based database achieves this? How does it maintain these relationships? Just through 0 and 1's?

I guess it's easy for me to visualize both row and column based. Im struggling with the bitmaps concept.

Re: FeatureBase: Open-Source, Real-Time Database Built on Roaring Bitmaps

#9

Post by witchbane »

kurosknight wrote: post #7
JAA1337 wrote: post #5

I've seen bitmaps mentioned a number of times lately. I must admit it is not something I am all that familiar with. Can someone explain to me why bitmaps are more valuable than standard column oriented databases? I havn't wrapped my head around how this helps speed up queries while data is being ingested.

They are useful for categorical variables. For example, is a record in the "Likes motorcycles" category? They are fast because (well, one reason) bitwise logical operations are very fast for CPUs to do. Adtech is an example of a sector that benefits from this...they slice and dice datasets a lot to target ad campaigns and such. Being able to do that quickly is useful.

So is the thinking you run them alongside a traditional RDBMS as sort of a cache or view optimized for bitwise operations?
Post reply on HN