Machine Learning Engineer Guide: Feature Store vs. Data Warehouse
logicalclocks.com
Machine Learning Engineer Guide: Feature Store vs. Data Warehouse
1–10 of 57 posts
Re: Machine Learning Engineer Guide: Feature Store vs. Data Warehouse
#2Other resources on feature stores that help clarify the difference and when to choose it over a data warehouse
https://medium.com/data-for-ai - several blogs discussing feature stores for machine learning
https://www.logicalclocks.com/blog/feature-store-the-missing... - in depth explanation
featurestore.org - list of all feature stores available and in production
Re: Machine Learning Engineer Guide: Feature Store vs. Data Warehouse
#3Author here. I wrote this article because I keep getting the question from prospects - isn't this just a data warehouse? If I missed out on anything or got anything wrong, please let us know here.
Re: Machine Learning Engineer Guide: Feature Store vs. Data Warehouse
#4Great read! I can imagine that a Feature Store could have an impact regarding the challenge of provenance in ML as well. I couldn't find anything on that.
Appreciate the comparison table: Data Warehouse vs Feature Store. Might stea.. re-use that.
Re: Machine Learning Engineer Guide: Feature Store vs. Data Warehouse
#5We have been building a feature first data store for seven years and it feels like feature store is about to become one of the more exciting ways to extract value from data. We see feature stores doing much more than becoming just another silo for ML, but instead a way to get a real-time, centralized view of fragmented data that has to either be federated or put in a data lake to to be queried together. I share more of my thoughts in this blog post: https://www.molecula.com/why-moleculas-feature-based-approac...
Molecula is based on the OSS platform Pilosa (https://www.pilosa.com/) and both Pilosa and Molecula are transitioning to reposition as feature stores over the coming months. Doing machine-scale analytics and ML on the data itself will be a thing of the past.
Re: Machine Learning Engineer Guide: Feature Store vs. Data Warehouse
#6We have been building a feature first data store for seven years and it feels like feature store is about to become one of the more exciting ways to extract value from data. We see feature stores doing much more than becoming just another silo for ML, but instead a way to get a real-time, centralized view of fragmented data that has to either be federated or put in a data lake to to be queried together. I share more…
But is it open-source?
Our Hopsworks Feature Store is
* https://github.com/logicalclocks/hopsworks
as is GoJEK Feast
* https://github.com/feast-dev/feast
I really believe that all Enterprise platform software should have an open-source version if it is to have a meaningful effect on how people work (in this case, how Data Scientists and Data Engineers work together).
Re: Machine Learning Engineer Guide: Feature Store vs. Data Warehouse
#7I get a strong buzzword bingo vibe from this post. On a related note, is there a good reason to ever have something like a "data lake" (and call it like that)? Whenever I've encountered someone bringing up the idea to "build a data lake", a few questions later it became clear they just had a messy pile of incoherent, poorly-understood data and wanted to twist it into something positive by giving that pile a fancy name.
Re: Machine Learning Engineer Guide: Feature Store vs. Data Warehouse
#8We have been building a feature first data store for seven years and it feels like feature store is about to become one of the more exciting ways to extract value from data. We see feature stores doing much more than becoming just another silo for ML, but instead a way to get a real-time, centralized view of fragmented data that has to either be federated or put in a data lake to to be queried together. I share more…
But is it open-source? Our Hopsworks Feature Store is * https://github.com/logicalclocks/hopsworks as is GoJEK Feast * https://github.com/feast-dev/feast I really believe that all Enterprise platform software should have an open-source version if it is to have a meaningful effect on how people work (in this case, how Data Scientists and Data Engineers work together).
Yes, Pilosa is OSS under Apache 2.
Re: Machine Learning Engineer Guide: Feature Store vs. Data Warehouse
#9We have been building a feature first data store for seven years and it feels like feature store is about to become one of the more exciting ways to extract value from data. We see feature stores doing much more than becoming just another silo for ML, but instead a way to get a real-time, centralized view of fragmented data that has to either be federated or put in a data lake to to be queried together. I share more…
This looks like a highly specialised tool. How is it going to integrate with a Data Scientist's favourite tools, such as Jupyter notebooks, Pandas or Spark and especially ML frameworks like TensorFlow, SkLearn etc.?
Re: Machine Learning Engineer Guide: Feature Store vs. Data Warehouse
#10I get a strong buzzword bingo vibe from this post. On a related note, is there a good reason to ever have something like a "data lake" (and call it like that)? Whenever I've encountered someone bringing up the idea to "build a data lake", a few questions later it became clear they just had a messy pile of incoherent, poorly-understood data and wanted to twist it into something positive by giving that pile a fancy nam…
Having one central place to find and access data, even if its messy, can be better for understanding and using the data than each business unit/team/... having their own different place for (still messy) data. It can of course also be utterly pointless if done badly, not maintained, not used, ...