Live data from Hacker News

Wikipedia: Database Download

en.wikipedia.org

101–106 of 106 posts

Re: Wikipedia: Database Download

#101
post #73

Earlier quoted context omitted.

I would love to see most sites serve me markdown. I'd happily install a browser extension to mask me as a a AI bot scraper if it means I can just get the text without all the noise.

I have good news. It (almost) exists, it is called Gemini [0] - [0] https://geminiprotocol.net/

Not news to me. I host my own gemlog there, but post rather infrequently.

Websites as gemtext would be even better than as markdown, but less likely to be fed to bots.

Re: Wikipedia: Database Download

#102
post #62

Earlier quoted context omitted.

Will the 2025 zim be available as well?

Main Kiwix dev in charge of scrapers (tools to create ZIM files, even if we do not really scrape technically speaking) here. We are working hard toward upgrading the Wikipedia ZIMs, but it is far from being an easy feat. I'm mostly solo on this, and far from dedicating 100% of my time to this, so it does not move very fast. We are quite close to being able to reach the goal however, probably only a matter of weeks no…

Hey, thank you for what you do! Kiwix was the first project that made me feel like using Github Sponsors to support it; your work is wrapped into countless other educational projects like IIAB.

Is there any specific help you need, or where could folks get involved if they wanted to?

Re: Wikipedia: Database Download

#103
post #83

Earlier quoted context omitted.

Then figure out how and where to store the over one thousand volumes that have 1200 pages each.

From "Plenty of Room at the Bottom"[1]: What would happen if I print all this down at the scale we have been discussing? How much space would it take? It would take, of course, the area of about a million pinheads ... All of the information which all of mankind has every recorded in books can be carried around in a pamphlet in your hand — and not written in code, but a simple reproduction of the original pictures, en…

Yeah electron microscopes are very much not "practically indestructible" on the scale of civilization-wide disturbances.

Going from "25,000x -> encyclopedia on pinhead" to "100x (microfilm scale) -> encyclopedia on a 5x10cm metal sheet" is probably a better bet.

Re: Wikipedia: Database Download

#104

Earlier quoted context omitted.

Genuine question, can you provide multiple explicit examples of such bias? I heard a lot of people railing against bias in Wikipedia, but no one provides any blatant examples of it.

A genuine answer, how about looking up some studies on this subject? Not those done by Wikipedia of course, they claim to be politically neutral after all. Here's a few, from https://www.allsides.com/blog/wikipedia-biased Six studies, including two from Harvard researchers, have found a left-wing bias at Wikipedia: A 2024 analysis [1] by researcher David Rozado that used AllSides Media Bias Ratings [2] found Wikipedi…

> A genuine answer, how about looking up some studies on this subject?

I figured that since you had a strong opinion on the subject you probably had strong evidence and could steer us to more directed reading to understand your viewpoint. Certainly we all should investigate things for ourselves, but sometimes it helps to have a place to start. You've certainly given us plenty to read through and consider. I'll read it with an open mind - some prereading thoughts that come to mind, is the citation bias proportional to factual accuracy (some outlets are more factually accurate than others, so one would expect to see them cited more often)? What's the distribution of the population of potentially citable sources (I.e. Is the bias a reflection of the population, or selection bias)? Is editor sanctions selective in enforcement or are conservative editors more likely to engage in behavior that warrants sanctions?

In other words, are we confusing correlations with causation? I don't know, I'll have to dig into the sources you provided and do my own research. I posit the questions now because it's the only thoughts I can contribute to the discussion at present.

Re: Wikipedia: Database Download

#105
post #62

Earlier quoted context omitted.

Will the 2025 zim be available as well?

Main Kiwix dev in charge of scrapers (tools to create ZIM files, even if we do not really scrape technically speaking) here. We are working hard toward upgrading the Wikipedia ZIMs, but it is far from being an easy feat. I'm mostly solo on this, and far from dedicating 100% of my time to this, so it does not move very fast. We are quite close to being able to reach the goal however, probably only a matter of weeks no…

Thank you for your answer and the hard work!

Re: Wikipedia: Database Download

#106
post #50

Earlier quoted context omitted.

It's not Wikipedia's fault that the vast majority of right wing media consists of pure propaganda, disinformation, and lies.

[citation needed] And you know, I wouldn't be surprised if people hurling those accusations somehow believe that the lies and misinformation are one-sided and partisan. As if leftism has some sort of monopoly on Truth and Goodness bestowed from above. It's really been sickening to see the media outlets just lay down thick trails of bullshit that is designed to distract us, to instill fear, uncertainty, and doubt, to…

This thread is about media organizations, but I think useful context is that in non-polarized situations conservatives and liberals are similarly likely to spread political misinformation while in polarized situations conservatives are relatively more likely to spread political misinformation [1].

[1] https://journals.sagepub.com/doi/full/10.1177/00222429241264...

Post reply on HN