Surfer: Open-Source Personal Data Warehouse

#1

Post by slalani304 » Sat, Dec 14, 2024, 6:42 PM UTC

Surfer: Open-Source Personal Data Warehouse
github.com

Re: Surfer: Open-Source Personal Data Warehouse

#2

Post by ErikBjare » Mon, Dec 16, 2024, 6:51 AM UTC

Nice to finally see this on HN! Excited about it's future

Re: Surfer: Open-Source Personal Data Warehouse

#3

Post by xnx » Tue, Dec 17, 2024, 9:39 PM UTC

Love this idea. It's crazy that every company knows more about my online activity than I know about it myself.

Definitely worried that running the extractors would get my accounts banned for violating policies against automated requests.

Re: Surfer: Open-Source Personal Data Warehouse

#4

Post by bix6 » Tue, Dec 17, 2024, 11:08 PM UTC

Love it and the name!

Is this the best option out there right now? I’ve seen other projects from time to time but not sure what’s stuck.

Re: Surfer: Open-Source Personal Data Warehouse

#5

Post by Carrok » Tue, Dec 17, 2024, 11:41 PM UTC

ErikBjare wrote: post #2

Nice to finally see this on HN! Excited about it's future

It was on HN not long ago. https://news.ycombinator.com/item?id=41325719

Re: Surfer: Open-Source Personal Data Warehouse

#6

Post by Edwardzhang9 » Wed, Dec 18, 2024, 12:04 AM UTC

huge

Re: Surfer: Open-Source Personal Data Warehouse

#7

Post by slalani304 » Wed, Dec 18, 2024, 12:15 AM UTC

xnx wrote: post #3

Love this idea. It's crazy that every company knows more about my online activity than I know about it myself. Definitely worried that running the extractors would get my accounts banned for violating policies against automated requests.

This hasn't happened for any of my accounts and I've been working on this for the last few months.

Re: Surfer: Open-Source Personal Data Warehouse

#8

Post by imglorp » Wed, Dec 18, 2024, 2:52 AM UTC

slalani304 wrote: post #7

xnx wrote: post #3

Love this idea. It's crazy that every company knows more about my online activity than I know about it myself. Definitely worried that running the extractors would get my accounts banned for violating policies against automated requests.

This hasn't happened for any of my accounts and I've been working on this for the last few months.

Even so, would a rate limit be wise?

Re: Surfer: Open-Source Personal Data Warehouse

#9

Post by j45 » Wed, Dec 18, 2024, 3:50 AM UTC

xnx wrote: post #3

Love this idea. It's crazy that every company knows more about my online activity than I know about it myself. Definitely worried that running the extractors would get my accounts banned for violating policies against automated requests.

No need to worry - it's your data.

Large organizations do a takeout/checkout process that sends you a notification when the data is ready for manual exports.

Re: Surfer: Open-Source Personal Data Warehouse

#10

Post by snthpy » Wed, Dec 18, 2024, 6:08 AM UTC

Love the idea!

I still need to look at the implementation later but just wondering if there is any overlap / possibility for leveraging atproto from Bluesky? Probably different use cases but it just made me think of PDSs.