Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Overflow Offline project

stackoverflow.blog

Post reply on HN
Print view View original

31–40 of 40 posts

Re: The Overflow Offline project

#31

Post by chiph »

I downloaded the databases earlier this year via torrent and attached them to a local SQL Server instance. Some notes:

- The files are big. The data files were 486 GB and the log file (I haven't tried shrinking it yet) is 1.4 TB

- There were no foreign key relationships defined. Nor indexes on commonly queried columns. Easy enough to add them as you work on tuning it.

- To be usable by humans you'll need to set up some full text indexes for searching - they use Elasticsearch. SQL Server's Full-Text catalog doesn't get you where you need.

https://stackexchange.com/performance

Re: The Overflow Offline project

#32

Post by JessicaHicklin »

This project has really been the make or break for us to be able to do the coding in prisons program, as well as the devshop that develops Ed Tech for prisons at Unlocked Labs. Love at the project has been formalized and will make a huge difference in the lives of many justice involved individuals who just want to return to their communities as productive citizens.

Re: The Overflow Offline project

#33

Post by mkathuri »

orblivion wrote: post #23
mkathuri wrote: post #11

Nice to find Kiwix again. Shameless plug, I made my own Kiwix alternative for macOS: https://github.com/technusm1/kiwings

So this is a desktop app, but it uses the server as part of it? The normal Kiwix desktop client doesn't do that right? I'll throw in my own shameless plug: Self-host your Stack Overflow, Wikipedia etc on Sandstorm: https://apps.sandstorm.io/app/5uh349d0kky2zp5whrh2znahn27gwh... Obviously uses kiwix-serve as well. 3 years old, I need to make a better clip for updating it.

I originally built this because Kiwix desktop didn't work with TED videos for me. Kiwix-serve works nicely always, plus I wanted to learn SwiftUI.

Re: The Overflow Offline project

#35

Post by aasasd »

mdaniel wrote: post #29
ComodoHacker wrote: post #19

They could actually try to build a Copilot competitor off their data. /s

I see the "/s" but I actually do wonder if integrating the "prompt" behavior into the question box would help cut down on the absolutely staggering number of duplicate questions. Regrettably, I'm not enough of a GPT expert to know what percentage of the time it would generate gibberish thus making the duplication question problem _worse_

Pretty sure they already show suggestions when the user is typing the question topic, though idk if they catch much. I guess the question here is whether some kind of AI would pop up more relevant suggestions, all the way to “your question is already answered, doofus”.

Re: The Overflow Offline project

#36

Post by aasasd »

ComodoHacker wrote: post #19

They could actually try to build a Copilot competitor off their data. /s

Copilot themselves probably swiped it all already, since SO answers are licensed under CC-BY-SA, which to Github is equal to ‘just take it all and ignore the author’. With the possible small difference that the uploader doesn't agree to the Github ToS.

Re: The Overflow Offline project

#38

Post by hoosieree »

JessicaHicklin wrote: post #32

This project has really been the make or break for us to be able to do the coding in prisons program, as well as the devshop that develops Ed Tech for prisons at Unlocked Labs. Love at the project has been formalized and will make a huge difference in the lives of many justice involved individuals who just want to return to their communities as productive citizens.

Never heard of Unlocked Labs before reading this comment, but that sounds like an outstanding project. Keep up the good work. You didn't promote yourself but I have no affiliation and this is what I found after searching: https://unlockedlabs.org/

I was once a guest in a maximum security prison for a few hours. It gave me the eerie realization that my own freedom is entirely up to the guards and staff. When that door closes you are no longer free - you're at the mercy of a huge system. It's a terrible feeling. Talking with some of the inmates made me realize how much I had subconsciously dehumanized them as a group. It was eye opening.

Listening to stories of people who have been in prison, even if you never meet them, can help build empathy. Here's a good interview with a guy who taught himself to program while in prison: https://corecursive.com/prison-programming-with-rick-wolter/

If you're ever interviewing a formerly incarcerated person for a job, I'd encourage you to try very hard to keep your biases in check. Building a life after prison seems like a supremely difficult task.

[edit] That's what I get for reading the comments before the article! Unlocked Labs is linked in the 2nd sentence. :)

Re: The Overflow Offline project

#39

Post by pbronez »

orblivion wrote: post #23
mkathuri wrote: post #11

Nice to find Kiwix again. Shameless plug, I made my own Kiwix alternative for macOS: https://github.com/technusm1/kiwings

So this is a desktop app, but it uses the server as part of it? The normal Kiwix desktop client doesn't do that right? I'll throw in my own shameless plug: Self-host your Stack Overflow, Wikipedia etc on Sandstorm: https://apps.sandstorm.io/app/5uh349d0kky2zp5whrh2znahn27gwh... Obviously uses kiwix-serve as well. 3 years old, I need to make a better clip for updating it.

How alive is Sandstorm these days? I loved the concept and was sad when the company died.

Re: The Overflow Offline project

#40

Post by orblivion »

pbronez wrote: post #39
orblivion wrote: post #23

Earlier quoted context omitted.

So this is a desktop app, but it uses the server as part of it? The normal Kiwix desktop client doesn't do that right? I'll throw in my own shameless plug: Self-host your Stack Overflow, Wikipedia etc on Sandstorm: https://apps.sandstorm.io/app/5uh349d0kky2zp5whrh2znahn27gwh... Obviously uses kiwix-serve as well. 3 years old, I need to make a better clip for updating it.

How alive is Sandstorm these days? I loved the concept and was sad when the company died.

A rather small number of people are working on it, but it's chugging along. Kenton has the keys still so he does basic vital things. A few new apps are in the pipeline. I'm still rather interested, and am developing another app that I expect to get a lot of attention when it's done.

There's a fund:

https://opencollective.com/sandstormcommunity

The bigger it gets, the more time can be diverted toward it. I was recently paid to upgrade the Etherpad app. But in theory it could go toward core development (which wouldn't be me).

Post reply on HN