Building personal search infrastructure for your knowledge and code • Hacker News Board

Re: Building personal search infrastructure for your knowledge and code

#111

Post by coleifer » Tue, Jan 28, 2020, 1:49 AM UTC

djhworld wrote: post #44

I've given up with trying to find The One True Note Taking Tool, so have ended up writing my own thing that I tinker with now and again to tune it to exactly what I need. It's essentially a simple web server that sits on top of a bunch of markdown files. The frontend renders the markdown using markdown-it and supports KaTeX for simple inline mathy things, along with the extended markdown stuff like tables etc. I've e…

I have a similar tool I've been using for years now. It is built on sqlite and uses the fts extension to provide full-text search.

Some nice features I've added over the years: bookmarklet and automatic page-screenshotting, tags (and smart auto-tagging), everything markdown supports, file upload and attach, media embed (YouTube link becomes player, eg). Oh, I can also attach email reminders and make to-do lists (with little checkboxes and everything). It started out very simple and has grown over time. Sqlite is a great foundation for projects like this. Strongly recommend.

Re: Building personal search infrastructure for your knowledge and code

#112

Post by bachmeier » Tue, Jan 28, 2020, 2:06 AM UTC

BLKNSLVR wrote: post #52

ahnick wrote: post #38

Earlier quoted context omitted.

Maybe this solves your problems? It creates a database in your browser's LocalStorage. https://noteself.github.io/

And the database can be setup to sync with a self-hosted CouchDB instance.

While that works, the original appeal of Tiddlywiki was that you could open a file in your browser, type away, and save naturally. Once you get into "self-hosted", you just have a regular old wiki. I used it everyday for several years but gave up once the transition happened. I keep trying to go back, but it just can't compete with files edited in a text editor and stored locally.

Re: Building personal search infrastructure for your knowledge and code

#113

Post by Terr_ » Tue, Jan 28, 2020, 2:31 AM UTC

stillwater56 wrote: post #78

Does anyone else find that the simple act of writing notes helps them remember and process better? I spent forever trying to find an ideal note-taking solution, but now I just write things in a single notebook. I rarely review my notes, but I find that simply writing thoughts down consistently has improved my memory and understanding of new concepts.

Yes, the effectiveness of note-taking (particularly handwritten) on memory has been a subject of scientific interest for a while.

I feel it shares aspects with Rubber Duck Debugging: The effort of taking something you "know" and forcing it back out through other brain-circuits (i.e. language and/or simulating a social interaction) helps to fill gaps that your brain would otherwise skip over. The act of hearing/seeing your output also causes other parts of your brain to analyze it as if it were someone else's thought.

I suspect our consciousness isn't nearly as unified as we like to believe.

Re: Building personal search infrastructure for your knowledge and code

#114

Post by 0x262d » Tue, Jan 28, 2020, 2:31 AM UTC

jhoechtl wrote: post #9

ssivark wrote: post #6

Meta-observation. This topic seems to be getting a lot of attention on HN over the last few months, indicating massive interest. Further, looking at the landscape of developments in this space (past all the me-too Markdown note taking apps): Evernote seems to have a fading presence on the landscape, Notion seems to be a (too?) well-funded behemoth startup, Roam is trying some exciting things, and Tiago Forte is putti…

> It’s amazing for how long Emacs’ Org-mode has been largely unparalleled! I use org-mode all the day but frankly OneNote is great too! If OneNote would save in plain text and have a cross-platform gui I would use it (even if it's resource-sucking electron)

I use onenote extensively and the mobile app has gotten steadily a little slower and worse. I would love to have a simpler, faster mobile app that would sync to my computer in plaintext so I could use vim with it....

Basically, onenote is almost there but I would love to leave it

Re: Building personal search infrastructure for your knowledge and code

#115

Post by pottertheotter » Tue, Jan 28, 2020, 3:06 AM UTC

bachmeier wrote: post #106

ssivark wrote: post #6

Meta-observation. This topic seems to be getting a lot of attention on HN over the last few months, indicating massive interest. Further, looking at the landscape of developments in this space (past all the me-too Markdown note taking apps): Evernote seems to have a fading presence on the landscape, Notion seems to be a (too?) well-funded behemoth startup, Roam is trying some exciting things, and Tiago Forte is putti…

> Evernote seems to have a fading presence on the landscape They've made massive changes over the past year. They'll even have a Linux app coming out soon!

Really? What massive changes? I use it regularly but haven't noticed anything massive.

Re: Building personal search infrastructure for your knowledge and code

#116

Post by avisaven » Tue, Jan 28, 2020, 3:06 AM UTC

mikepurvis wrote: post #48

sqs wrote: post #28

Sourcegraph CEO here. I see the doc mentions Sourcegraph for code search (cool!). Something like ripgrep is indeed better for your case, a single person who just needs to search code in local directories on their own machine. I made a PR for our docs at https://github.com/sourcegraph/sourcegraph/pull/8075 that should clarify this. Sourcegraph is a web-based code search tool that automatically syncs and indexes many r…

Another great option for local code/repo search is Hound. I maintain an instance of it at my workplace, but it's so lightweight and easy to deploy that I could easily imagine running an instance of it on my laptop for offline personal use. https://github.com/hound-search/hound

A similar but more structured (though perhaps Hound supports a similar feature set) code searching tool is OpenGrok [1]. It's a bit more setup as it uses Apache Tomcat, but once it is setup it has an incredibly fast and useful code querying tool with really useful abilities to x-ref functions/structures, highlight uses of variables, and integrates git info as well. If you've ever used elixir.bootlin.com to go through the Linux source code, opengrok is effectively a more feature packed open source version of that. I highly recommend taking a look to anyone who spends a lot of time digging through code.

[1] https://oracle.github.io/opengrok/

Re: Building personal search infrastructure for your knowledge and code

#117

Post by archontes » Tue, Jan 28, 2020, 3:46 AM UTC

gwgundersen wrote: post #76

djhworld wrote: post #44

I've given up with trying to find The One True Note Taking Tool, so have ended up writing my own thing that I tinker with now and again to tune it to exactly what I need. It's essentially a simple web server that sits on top of a bunch of markdown files. The frontend renders the markdown using markdown-it and supports KaTeX for simple inline mathy things, along with the extended markdown stuff like tables etc. I've e…

This sounds a lot like a tool I built for myself [1], sans the database. I agree that Markdown + Katex with a local server seems like the right move for most technical people. Lots of things like encryption, backups, and basic text search can be done via other Unix tools. I also agree that the big win is owning your data long-term, even if you get tired of maintaining the software. [1] https://github.com/gwgundersen/…

Sir! I have to say seeing you here that I appreciate your contributions.

Re: Building personal search infrastructure for your knowledge and code

#118

Post by rawoke083600 » Tue, Jan 28, 2020, 4:17 AM UTC

Most stuff (links, photos, docs, etc) I just email it to myself

Re: Building personal search infrastructure for your knowledge and code

#119

Post by bookAlot » Tue, Jan 28, 2020, 4:27 AM UTC

What are your favorite questions in meeting new people? And at work?

That's how you build on knowledge for a more diverse platform. Good luck! From Trump too!

Re: Building personal search infrastructure for your knowledge and code

#120

Post by glinkot » Tue, Jan 28, 2020, 4:34 AM UTC

I use a few things for this (on windows):

- For notes, OneNote, though I'm always on the lookout for an alternative with decent UI and syncing, but using open file formats. Full text search simple enough with this. Code formatting isn't good but there's an addin where the free version formats it as it was copied.

- To search local files, Voidtools Everything is great. Searching instantly by filename is a real time saver.

- If I want full text search of a large base of documents, I used Likasoft Archivarius which cost me $30 about 10 years ago and is still handy. It's the only local desktop search I've found that supports full text indexing of tons of formats like outlook .ost, etc and can look inside archive files

- For backups I've continued to stick with external drives, mirrored periodically with Freefilesync. 3 copies - one as master, two mirrors ensuring one is offsite.