Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building personal search infrastructure for your knowledge and code

beepb00p.xyz

Post reply on HN
Print view View original

61–70 of 162 posts

Re: Building personal search infrastructure for your knowledge and code

#61

Post by lcall »

gotts wrote: post #26
lcall wrote: post #15

I wrote and use daily http://onemodel.org (AGPL, uses postgres), for many reasons listed there :) . One way to think of its current state is a text-mode, easy-to-learn (i hope) infinite mind map of things, where I store and can query effectively everything: calendar, reminders, quasi-anki-like knowledge review, journal, automatic activity log, notes on subjects, very efficiently for the user. (It also stores document…

telnet demo seems to be down at the moment: Trying 52.37.29.12...

If you have possible future interest, there is an announcements list.

Re: Building personal search infrastructure for your knowledge and code

#62

Post by blyry »

mikepurvis wrote: post #48
sqs wrote: post #28

Sourcegraph CEO here. I see the doc mentions Sourcegraph for code search (cool!). Something like ripgrep is indeed better for your case, a single person who just needs to search code in local directories on their own machine. I made a PR for our docs at https://github.com/sourcegraph/sourcegraph/pull/8075 that should clarify this. Sourcegraph is a web-based code search tool that automatically syncs and indexes many r…

Another great option for local code/repo search is Hound. I maintain an instance of it at my workplace, but it's so lightweight and easy to deploy that I could easily imagine running an instance of it on my laptop for offline personal use. https://github.com/hound-search/hound

YES! We have been using hound for several years now, having all hundreds of our org repos searchable in one spot, in a LIGHTNING FAST manner has been an invaluable tool to help our various teams keep up with the legacy sprawl and effectively remove old features and all their dependencies from our sprawly systems. I even wrote a microservice that uses gitlab global hooks to keep hound up to date without polling, and a little c# config generator that runs as a cron job on our gitlab instance and redeploys hound with the newest repos included.

Hound falls short on access control front (we wrapped our instance with a saml proxy), but it's still a 'you either can search every piece of software for \'password\'' or you don't have any access at all. Having to index a specific branch instead of all of them kinda stinks too; for those two specific reasons we have been eyeing sourcegraph, esp. as the gitlab integration matures.

I can't emphasize enough how fast hound is and how pleasurable it is having a regex based code search that doesn't make me wait.

Re: Building personal search infrastructure for your knowledge and code

#63

Post by insomniacity »

fctorial wrote: post #49
insomniacity wrote: post #46

My eternal frustration in this space is that my employer has strict firewalls, web filtering and data-loss prevention software, and remote access is over Citrix with no copy-paste. Consequently, if I build a knowledge base, it is stuck inside the firewall. Equally, if I build it outside, I can't use it at work.

Why don't you host it on an ec2 instance? They won't be blocking amazon ip. Where do you work?

There's definitely no external access without going through the web proxy. And a new uncategorized site would be blocked by the web proxy - and it wouldn't pass review.

I work in a highly regulated industry...

Any workaround would be grounds for termination. So there's no point to my comment really - just curious if anyone else is in the same boat.

Re: Building personal search infrastructure for your knowledge and code

#64

Post by lcall »

lcall wrote: post #15

I wrote and use daily http://onemodel.org (AGPL, uses postgres), for many reasons listed there :) . One way to think of its current state is a text-mode, easy-to-learn (i hope) infinite mind map of things, where I store and can query effectively everything: calendar, reminders, quasi-anki-like knowledge review, journal, automatic activity log, notes on subjects, very efficiently for the user. (It also stores document…

(If one has possible future interest, there is an announcements list, and feedback is also appreciated.)

Re: Building personal search infrastructure for your knowledge and code

#65

Post by Kiro »

user00012-ab wrote: post #30

My problem with a lot of services listed below, is they all eventually go away, and all your data is off somewhere else. Unless you store your data locally in a human readable format (markdown) you are just putting all your data into a system that WILL go away at some point in the future. Google has already had 2-3 services to manage your data that they have closed down. Maybe they are the ones that taught me not to…

Honestly, the risk of me losing my local data is much higher than a note service shutting down. It has happened countless of times and I'm just too sloppy with the backups. Purely personal of course but for me your argument is reversed.

Re: Building personal search infrastructure for your knowledge and code

#66

Post by haddr »

djhworld wrote: post #44

I've given up with trying to find The One True Note Taking Tool, so have ended up writing my own thing that I tinker with now and again to tune it to exactly what I need. It's essentially a simple web server that sits on top of a bunch of markdown files. The frontend renders the markdown using markdown-it and supports KaTeX for simple inline mathy things, along with the extended markdown stuff like tables etc. I've e…

Same here: I'm taking a lot of notes and find SimpleNote/Notepad++/NV work for me for note taking but not for note management. And I also build several tools to manage it. For now I use very simple full-text search (qgrep) and fzf (with my own modifications) to perform search-as-you-type to find notes/source code. qgrep is good for really quick index and it has incremental indexing that works for my byt i'm hitting some problems as it works good for code search but not so good for notes. However i don't feel like using anything that start a server. I just don't think I have that many notes.

Re: Building personal search infrastructure for your knowledge and code

#67

Post by nabnob »

user00012-ab wrote: post #30

My problem with a lot of services listed below, is they all eventually go away, and all your data is off somewhere else. Unless you store your data locally in a human readable format (markdown) you are just putting all your data into a system that WILL go away at some point in the future. Google has already had 2-3 services to manage your data that they have closed down. Maybe they are the ones that taught me not to…

Someone shared this on HN yesterday - https://labs.tomasino.org/gnu-recutils/

It's a set of unix-style tools that let you treat text files as databases.

Re: Building personal search infrastructure for your knowledge and code

#68

Post by klft »

(1) For note taking I stumbled across anno[1] via[2] two weeks ago. It's a python flask application which you run on your localhost. You write markdown which gets stored locally as file and is rendered as html using pandoc[3]. It's really basic but I love it.

(2) For physical documents I use a Fujitsu ScanSnap iX500[4] for scanning. A runtime-licencse of ABBYY FineReader for OCR is included. The resulting PDF has embedded text which I extract using pdftotext[5]. I wrote a python application to search and tag this documents. It loads all the text in-memory which is perfecty fine as I have [1] https://github.com/gwgundersen/anno

[2] https://news.ycombinator.com/item?id=22033792

[3] https://pandoc.org/

[4] https://www.fujitsu.com/global/products/computing/peripheral...

[5] https://en.wikipedia.org/wiki/Pdftotext

Re: Building personal search infrastructure for your knowledge and code

#69

Post by marviel »

ssivark wrote: post #6

Meta-observation. This topic seems to be getting a lot of attention on HN over the last few months, indicating massive interest. Further, looking at the landscape of developments in this space (past all the me-too Markdown note taking apps): Evernote seems to have a fading presence on the landscape, Notion seems to be a (too?) well-funded behemoth startup, Roam is trying some exciting things, and Tiago Forte is putti…

It's a ripe space. I'm using notion mostly right now, but I've also used:

-Coda.io (big, more scriptable player)

-Hypernote (super new player, but with a cool new take on inter-note relationships)

-Tiddlywiki (super customizable, really fast -- but also has a fair amount of footguns)

-Airtable (only played with it a few times but it's usually mentioned in the same breath as notion, I notice)

Hopefully someday we'll achieve Alan Kay's dream :)

Post reply on HN