Ask HN: What is your information distillation pipeline?

#1

Post by mariomeissner » Thu, Oct 17, 2019, 4:54 AM UTC

I have been struggling with this for years now. I read articles on Medium, watch YouTube videos, daily check Twitter, HN, Reddit, etc. When I find a paper I like I queue it to to my Mendeley list. When I like an article I add it to Pocket. When I like a tweet I bookmark it. When I have time I try to consume some of the stuff I save. Yet it feels like I'm not making any good use of the information I'm putting into my brain. Many of the things I added I lost interest in because too much time passed, and others I just skim through even though I should be reading it in more detail and taking notes. When I take notes I struggle to organize them and make them easily searchable for posteriority.

This leads me to ask you how you go about dealing with the tons of information that internet makes available to us. I want to call this the "information distillation pipeline".

What sources do you use to learn new things and keep track of the novelties in your field?

How do you decide what is worth reading deeper into, spending more time on, etc?

How do you organize yourself to queue the information you want to dive more into when you have time?

How do you take out the bits of information that actually matter to you? Do you record them somewhere? A diary? A mind map?

How do you browse the information you have recorded? Do you try to memorize it? Add tags? Just basic word search? Complex folder structures?

A related post is https://news.ycombinator.com/item?id=20847508, but it only talks about books. Certainly they are one important source of information and the post gives many good insights, but I would like to revive the conversation with a broader spectrum of sources and focusing more on the whole pipeline and searchability of your saved knowledge.

Re: Ask HN: What is your information distillation pipeline?

#2

Post by ssivark » Thu, Oct 17, 2019, 5:39 AM UTC

> Many of the things I added I lost interest in because too much time passed,

That is great. Fewer things that you need to pay attention to. In fact, this kind of attrition is a great way to reduce the burden. You only want to invest your effort in those things whose value to you are not ephemeral. Check out "It's not what you read, it's what you ignore" by Scott Hanselman https://www.hanselman.com/blog/ItsNotWhatYouReadItsWhatYouIg...

> How do you decide what is worth reading deeper into, spending more time on, etc?

That really depends on your goals, the topic, your background, etc.

> Do you record them somewhere? A diary? A mind map? [...] When I take notes I struggle to organize them and make them easily searchable for posteriority.

I tend to take lots of notes in Markdown files. Recently I've been playing around with Org mode in Emacs with the "Deft" interface. It's full-text fast incremental search is fantastic.

> What sources do you use to learn new things and keep track of the novelties in your field?

At some point (as a grad student) I used to subscribe to a couple of relevant arXiv categories, to skim through ~50 titles/abstracts every day. But see my link to Scott Hanselman's post above.

Also, to quote Sturgeon's law, 90% of everything is crap. And, (as Knuth says) I would rather get to the bottom of things than stay on top of things. So, I consider it more important to invest time/effort into understanding thigns in depth, rather than following every little gust of wind.

The questions are too broad, so it's difficult to give a useful answer. Feel free to reach out to me via the email listen in my profile if you would like to discuss this more specifically.

> When I have time I try to consume some of the stuff I save. Yet it feels like I'm not making any good use of the information I'm putting into my brain.

I think "consumption" is the wrong thing to aim for.

Re: Ask HN: What is your information distillation pipeline?

#3

Post by kevsim » Thu, Oct 17, 2019, 7:15 AM UTC

I tend to focus on two types of value: entertainment stuff and actually-useful-for-my-job stuff.

A lot of what I read here on HN is the former. I'll probably never write my own compiler, but for some reason I can't resist reading an article that goes into gross detail about a 3% speed up in the Rust compiler. I don't sweat retaining the info I read. At best it's stuff to discuss around the lunch table with colleagues.

Then there are things that are truly useful for what I'm working on day to day. However, like with the pure entertainment stuff, I don't sweat about retaining everything I've read and putting it to good use. The reason is that there's a simple rule I try to follow - things that are important keep bubbling up. If we've got a problem and we need to fix something, then I'll dive extremely deep into reading about the problem we've got, and I'll put the stuff I read into practice _immediately_. By doing so I'll learn a lot and retain a lot. If I read an article that I think is interesting and could be useful _someday_ I might file a github issue to remind me to follow up, but usually not. Usually it's fine to let it go and dig in if and when there's a real problem.

Re: Ask HN: What is your information distillation pipeline?

#4

Post by psv1 » Thu, Oct 17, 2019, 11:18 AM UTC

1. Bookmark interesting stuff as it comes up.

2. Review bookmarks whenever you're bored, have some down time or just feel like it.

3. Only stick with material that you find engaging or is somehow essential to a larger project that you're working on. Happily discard stuff that doesn't grab your attention anymore.

4. When it comes to more well defined projects and goals, having a Trello board is really helpful. Around January I made a board with books, tutorials and courses that I wanted to cover this year - tracking my progress there and keeping things up to date has worked out great so far.

5. Don't aim to record, consume and retain everything, don't measure your own efficiency and don't call it an "information distillation pipeline" - you're not a machine, you're just a person reading stuff online, presumably for your own benefit and enjoyment.

Re: Ask HN: What is your information distillation pipeline?

#5

Post by billconan » Thu, Oct 17, 2019, 2:05 PM UTC

I have the same problem. I bookmark a lot and never come back. I lose interest in some of them, which I think is good, because I can focus on what’s left. But even that is too much.

The best way for me to learn is blogging. I usually study a topic first and then explain it to others.

Re: Ask HN: What is your information distillation pipeline?

#6

Post by godot » Thu, Oct 17, 2019, 7:22 PM UTC

I'm pretty sure I'm in the minority among HN crowd, but I don't do any of this. I still read a lot of articles and links. When I find something interesting I tend to read it immediately. (guilty of taking time to read during work; though some of these are work-related, some not.)

After I read it I roughly remember the topic, though not the details. At a later point if a conversation comes up or if something at work relates to that content, I roughly remember something about it, then google (or DDG if you prefer) for the piece again with some keywords I remember.

This has served me quite well for many many years and I don't really spend any of the time most people here spend with note-taking and maintaining a personal knowledge base. There were a few times I can remember myself failing to google a piece I wanted because I either can't remember enough keywords from it or the article is too obscure to find, but majority of the time I'm ok.

Re: Ask HN: What is your information distillation pipeline?

#7

Post by charlieegan3 » Thu, Oct 17, 2019, 7:29 PM UTC

RSS and Notion have made me much more positive about this problem recently.

I’ve found a nice RSS reader I like on my phone and use it to follow HN, blogs, newspaper, podcasts etc. There is only one opening/start in the pipeline. I don’t really read twitter anymore.

I open a reading list on my phone’s browser and use open tabs as a todo. I lose interest in some too but I don’t worry about that any more.

If I hear something interesting in a podcast I note it down in a collection in Notion while I listen.

After reading a book, paper or long article I copy my highlights into notion and summarise them over the course of a few weeks on my commute.

I also keep a notion table of unread books and use that to plan what I read on my ebook reader next.

I also gather information relevant to work in there. Links to docs or notes for my next 1:1.

With many of the notes I never return but I find the writing, summarising and organising helps me remember what I’ve read.

I think I take notes to remember things but i’m not really sure anymore.

Re: Ask HN: What is your information distillation pipeline?

#8

Post by sosilkj » Thu, Oct 17, 2019, 10:38 PM UTC

Possibly useful:

https://zettelkasten.de/posts/collectors-fallacy/

Re: Ask HN: What is your information distillation pipeline?

#9

Post by username90 » Fri, Oct 18, 2019, 7:18 AM UTC

I have an inbuilt system to handle this: I read the article, and if I later remember it then it was important and if I forget then it wasn't. Works well enough with minimal overhead, can't remember any important thing I've missed ;)

Re: Ask HN: What is your information distillation pipeline?

#10

Post by muzani » Fri, Oct 18, 2019, 4:48 PM UTC

0. Before starting, ask yourself whether this is worth reading. 99% of stuff out there is not, and you want to figure that out before you spend half an hour on it. You may want to learn to speed read for this reason; if it can be speed read it's probably too shallow to be worth it.

1. Articles and posts are the worst length for information. Tweets work great because they distill wisdom in only a few words, and super successful people have time to spare writing them. Books are ideal, because they can explore a topic inside out, whereas an article has to stop or doesn't have room to come to terms.

2. Podcasts and videos are poor for the same reason. Elite people rarely have time to do them.

3. In this era, it's become really common to write for the purpose of marketing a service or getting a job. Avoid places where people do this, e.g. LinkedIn, Medium, DEV.

4. When in doubt, don't read it. If it's important, you'll see several references to the same thing.

5. Improve your comprehension. It takes active, tiring, effort to get it up. I'd say 90% comprehension is a good goal, but even 50% is fine.

Long story short, the brain is not designed to comprehend words. It comprehends pictures great. So you want to train your brain to convert words to pictures in milliseconds. A lot of great mathematicians think in diagrams, not formulas. That way you can take on harder books.

6. Follow what you're interested in now, don't structure it too much. A lot of books refer some interesting ideas to other books. Don't be afraid to drop a book and jump to another one. I've probably only read a few pages in Tools of Titans. Mostly it's been a reference to find better books. You also know very different things to other people; the books that fascinate you are different to the ones that fascinate others.