Live data from Hacker News

Personal computing paves the way for personal library science

bramadams.dev

31–40 of 52 posts

Re: Personal computing paves the way for personal library science

#31

I work in a very interdisciplinary, and somewhat niche, tech/engineering field. For the past 15 years, I've been saving every relevant PDF that I can find -- mostly studies of the sort published by Elsevier and Springer, but also books and presentations. I now have around 10k, which probably makes it the largest private library focused on this particular domain of expertise. It has been extremely useful, especially b…

How would you go about training an llm on 10k large pdfs?

Re: Personal computing paves the way for personal library science

#32

I work in a very interdisciplinary, and somewhat niche, tech/engineering field. For the past 15 years, I've been saving every relevant PDF that I can find -- mostly studies of the sort published by Elsevier and Springer, but also books and presentations. I now have around 10k, which probably makes it the largest private library focused on this particular domain of expertise. It has been extremely useful, especially b…

> I'd add that these days it totally makes sense to download libgen's entire archive, because (1) storage has never been cheaper, and (2) you can use it to train local LLMs.

Hardly. Data hoarding comes with most downsides of hoarding physical objects. It's just smaller, cheaper & easier to process.

There's people that get rid of any physical object they haven't used in the past year (or 2, or 5, whatever). This makes sense. Imho, every object you own falls in 3 categories:

1) Things you use on a (semi?) regular basis. They make your life easier/nicer.

2) Things that are valuable. For flexible metrics of what constitutes value (sentimental, nostalgia, monetary, insurance against 'disaster', ...)

Not having used something in a long time is a good hint it's not valuable.

3) Luggage. Whose value is negative. It doesn't provide anything, just takes up space, drains mental energy (and possibly other resources), and in doing so gets in the way of other pursuits.

Data is no different. For any single piece of it, you either use it from time to time, somehow derive value from it, or it is useless luggage that you drag around at a cost.

Apply good judgement in what to hoard.

Re: Personal computing paves the way for personal library science

#33

I work in a very interdisciplinary, and somewhat niche, tech/engineering field. For the past 15 years, I've been saving every relevant PDF that I can find -- mostly studies of the sort published by Elsevier and Springer, but also books and presentations. I now have around 10k, which probably makes it the largest private library focused on this particular domain of expertise. It has been extremely useful, especially b…

> A local LLM will make it 100x more useful. Also, it might not even need be "local." If I make it available via the web, I can probably sell access to other scientists and engineers in my field.

That's not legal. Just because you own some files does not mean that you own the IP of the content within.

Re: Personal computing paves the way for personal library science

#34

I work in a very interdisciplinary, and somewhat niche, tech/engineering field. For the past 15 years, I've been saving every relevant PDF that I can find -- mostly studies of the sort published by Elsevier and Springer, but also books and presentations. I now have around 10k, which probably makes it the largest private library focused on this particular domain of expertise. It has been extremely useful, especially b…

> A local LLM will make it 100x more useful. Also, it might not even need be "local." If I make it available via the web, I can probably sell access to other scientists and engineers in my field. That's not legal. Just because you own some files does not mean that you own the IP of the content within.

It is, however, the right thing to do.

Re: Personal computing paves the way for personal library science

#35

I work in a very interdisciplinary, and somewhat niche, tech/engineering field. For the past 15 years, I've been saving every relevant PDF that I can find -- mostly studies of the sort published by Elsevier and Springer, but also books and presentations. I now have around 10k, which probably makes it the largest private library focused on this particular domain of expertise. It has been extremely useful, especially b…

> A local LLM will make it 100x more useful. Also, it might not even need be "local." If I make it available via the web, I can probably sell access to other scientists and engineers in my field. That's not legal. Just because you own some files does not mean that you own the IP of the content within.

That’s OpenAI’s business model tho

Re: Personal computing paves the way for personal library science

#36

Earlier quoted context omitted.

If it's research it very probably is at least partially publicly funded. Regardless of whatever the law says, I don't think it's immoral to take it and offer better services around it that will be useful enough that someone decides to pay.

Do you not see the hypocrisy in stating that someone should be able to take something partially publicly funded and profit from it while the creator of said work should not retain some rights over said profit? By extension of the transitive property of the nebulous "partially", the LLM wrapper should be provided for free with complete disregard to the wrapper's creator since it is a derivative of partially publicly f…

The hoarders of IP in this case usually have done 0 work to produce it (Elsevier et al).

Re: Personal computing paves the way for personal library science

#37

Personal computing has stagnated for such a long time, it creates substantial uncertainty about what state it might evolve to if and when the next step actually happens. In this respect local LLM's are simply the tip of the iceberg, pointing out the vast amount of personal information processing that is available in principle but does not actually happen.

One could argue that personal computing (desktop) software piracy lead to web-based SaaS subscription licensing. In theory, mobile app stores solved device software piracy, at the cost of high distribution fees, policy restrictions and telemetry. Thanks to Linux being used at scale in Android and WSL, it's now maintained and capable on the desktop, as a hypothetical foundation for personal computing innovation. But e…

Unfortunately so far Qualcomm was hard at work avoiding making an open platform ARM-based laptop/tablet - to the point of squeezing around MS rules on that through special drivers etc to make Windows think it's dealing with EFI-compliant hardware.

Re: Personal computing paves the way for personal library science

#38
Best quote from the article: "...personal library science is focused on your relationship with your information. How do we store information so that it useful at a later date? How do we transform our information into new valuable assets in different creative domains? How do we do all of this while being flexible enough for the idiosyncrasies, proclivities, likes and dislikes of eight billion distinct individuals? How do we chronicle the information diet of a single person as they learn new things, interact with the world at different phases in their life? How do we make sure we can pass down our best knowledge to generations below?"

Re: Personal computing paves the way for personal library science

#39

I work in a very interdisciplinary, and somewhat niche, tech/engineering field. For the past 15 years, I've been saving every relevant PDF that I can find -- mostly studies of the sort published by Elsevier and Springer, but also books and presentations. I now have around 10k, which probably makes it the largest private library focused on this particular domain of expertise. It has been extremely useful, especially b…

> I'd add that these days it totally makes sense to download libgen's entire archive, because (1) storage has never been cheaper, and (2) you can use it to train local LLMs. Hardly. Data hoarding comes with most downsides of hoarding physical objects. It's just smaller, cheaper & easier to process. There's people that get rid of any physical object they haven't used in the past year (or 2, or 5, whatever). This makes…

i just keep everything ive bothered to keep, (data and stuff) i put it in a sensible place and it only consumes time when looking for it.

the new external drive is so much larger than the oldest ones i put the new stuff on it and use the rest of the space to backup old drives.

friends are constantly purging stuff but they seem unaware how much time and effort it takes.

the significant other wanted to clean up her old photos rather than upgrade the icloud. its an insane amount of work?

Re: Personal computing paves the way for personal library science

#40

Best quote from the article: "...personal library science is focused on your relationship with your information. How do we store information so that it useful at a later date? How do we transform our information into new valuable assets in different creative domains? How do we do all of this while being flexible enough for the idiosyncrasies, proclivities, likes and dislikes of eight billion distinct individuals? How…

My personal favorite https://www.devontechnologies.com/apps/devonthink
Post reply on HN