Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Interface for all digital aspects of my life

beepb00p.xyz

Post reply on HN
Print view View original

51–60 of 95 posts

Re: Show HN: Interface for all digital aspects of my life

#51

Post by edraferi »

Very cool!

Intake [0] is another package that might help here. It organizes a set of data sources into

(1) plugins that actually connect to the data source and map the data to standard Python data structures like Data Frames

(2) catalogs that reference the plugins you want to use alongside project specific metadata like usernames/passwords/source URIs

(3) convenience functions that persistence, concatenation, etc

(4) a GUI for browsing data sources

Looks like OP has already written a lot of data access logic... I might fork them into Intake plugins for my personal use.

[0] https://github.com/intake/intake/blob/master/README.md

Re: Show HN: Interface for all digital aspects of my life

#52

Post by karlicoss »

simonw wrote: post #48

My version of this is called Dogsheep, and involves building tools that suck my data from various sources into SQLite databases so I can query it all using Datasette. https://dogsheep.github.io/ I enjoyed reading this author's essay on why he doesn't think loading everything into relational databases is the best approach: https://beepb00p.xyz/unnecessary-db.html I think I have a workarounds for most of the issues he…

Ooooh, I totally forgot to add your link, I remember you commenting, sorry! Will amend now.

Ah I see, so your approach is using databases as an 'intermediate' layer, rather than the main source of 'truth'. My main objection to databases is the fricton with the maintenance and the speed of prototyping, but if it's automatic and used as a cache, I agree it's reasonable. Although I'd still be worried about normalising something wrong, but this is my personal sort of anxiety :)

I've got something similar [0]: if you have a sequence of NamedTuples/dataclasses, you can get a database 'for free'.

[0] https://beepb00p.xyz/unnecessary-db.html#cachew_cachew

Re: Show HN: Interface for all digital aspects of my life

#53

Post by adwi »

GistNoesis wrote: post #44
karlicoss wrote: post #27

Earlier quoted context omitted.

Hey, I've got a crude timeline of events (simply rendered as giant HTMLs), but it's private at the moment. I'll probably share some of it in the future, especially considering that some of the data is public anyway (e.g. online comments or tweets), it makes sense. Next step: yep, I throught about it a bit [0], agree that it would be great as a service-agnostic way of accessing the data, the same way we can see it wit…

I don't like your project. Data aggregation is powerful but very dangerous and I feel that you are handling it carelessly. I also get a strong Borg vibe here. In the current state, people don't know how to handle their data correctly. You are just making it easier for some people siphon other's people data even more. And having their data played against themselves. Most people are not quants. And becoming one is not…

It’s infantilizing to imply people are incapable of handling the responsibility of having access to the personal data they’re creating, as if it’s better to just trust it with private companies that leverage it to monetize us.

I’m sure achieving whatever your sufficient level of “quant” isn’t trivial (congratulations presumably) but a command line interface took shared with the Hackernews community is fairly self-selecting.

I think this is a super interesting project, looking forward to seeing how this gets developed!

Re: Show HN: Interface for all digital aspects of my life

#54

Post by karlicoss »

simonw wrote: post #49
karlicoss wrote: post #15

Earlier quoted context omitted.

I don't mind merging new modules at the moment, so would be happy if you try it out and contribute! But it's also possible to use your module without merging in the upstream HPI package [0] In the long run, I'm not sure how sustainable it is, since there are many different data sources and ways of representing them. This is also particularly difficult when you don't use the service, e.g. as a maintaner I wouldn't hav…

Have you looked at Pluggy much? It's the Python plugin system that was spun out of pytest. I'm really impressed by it - it's a very clean design, and integrates great with Python packaging. I'm using it extensively for Datasette, which means myself or others can add new features to the core software without needing to ask permission and in a way which supports trying out crackpot new ideas without sullying the design…

Thanks, looks great, I'll check this out! So far I haven't even done proper research of potential solutions, because I feel like it would be overengineering at this stage, I'm still figuring out the core.

Re: Show HN: Interface for all digital aspects of my life

#55

Post by eitland »

chrisma0 wrote: post #23
nicbou wrote: post #13

I have been toying with a similar idea for a very long time. I want to have an all-encompassing personal history for the same reasons I want good access and error logs. I want to see when something wrong happens, and I want the means to diagnose the problem and fix the damage. A few good examples: - Did I forget to write down any tax-deductible expenses? - What is my net worth? Is my current lifestyle sustainable? -…

> Which parts of my city have I yet to explore? > Can I search all my conversation with John across 3 messaging apps, email, SMS and phone? These are totally pain points for me / questions I'd love to have some answers for... Anyone tried to get the historic iOS location data to answer the first question?

If you live in Europe it should be possible..?

Re: Show HN: Interface for all digital aspects of my life

#58

Post by ajphdiv »

chrisma0 wrote: post #23
nicbou wrote: post #13

I have been toying with a similar idea for a very long time. I want to have an all-encompassing personal history for the same reasons I want good access and error logs. I want to see when something wrong happens, and I want the means to diagnose the problem and fix the damage. A few good examples: - Did I forget to write down any tax-deductible expenses? - What is my net worth? Is my current lifestyle sustainable? -…

> Which parts of my city have I yet to explore? > Can I search all my conversation with John across 3 messaging apps, email, SMS and phone? These are totally pain points for me / questions I'd love to have some answers for... Anyone tried to get the historic iOS location data to answer the first question?

There is a open source digital forensics project called APOLLO (https://github.com/mac4n6/APOLLO) -- that could be used for personal use to get location information off your phone. Here's a blog post by the author, with a little bit of explanation:

https://www.mac4n6.com/blog/2019/8/21/i-know-where-you-were-...

Re: Show HN: Interface for all digital aspects of my life

#59

Post by themodelplumber »

ozim wrote: post #38

One day you wake up and realize that you spent your life maintaining your system to improve your life instead of actually improving your life.

Starting on a project to improve one's life by engaging latent gifts is one of the smartest things one can do. It is aligned with the best advice psychology can offer. If that means maintaining a measurement system works for you--great. It's not as if we lack data showing that measurement systems are beneficial to human development.

Re: Show HN: Interface for all digital aspects of my life

#60

Post by 21stio »

Communitivity wrote: post #21

Kudos - this looks like a great project and is fully usable now. I have a similar project, DL, that's unfinished. Mine revolves around using a custom API in both Rust and REST to aggregate all my digital life events using ActivityStreams 2.0 and extensions to that, in a manner that is decentralized and ranked/categorized through machine learning. I am still working on it and releasing it Open Source is one of this ye…

Hey, we are building something similar. Please checkout https://metamate.io/blog/most_advanced_hackernews_api. For now, we concentrate on reading public data from various social sources. We just released a HackerNews service and going to add more over the next couple of weeks. Here's a little application build on top of a MetaMate http://showcase.metamate.io/hackernews-activity. Please let me know what you think :)
Post reply on HN