Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Wikipedia: Database Download

en.wikipedia.org

Post reply on HN
Print view View original

51–60 of 106 posts

Re: Wikipedia: Database Download

#51

Post by [deleted] »

amanaplanacanal wrote: post #36
moritzwarhier wrote: post #13

Earlier quoted context omitted.

Is there a "politically neutral" human? And if there was, what could that person reasonably say about politics?

I suspect "politically neutral" is a meaningless phrase. It's just a way for people to tar their political opponents by inference. The problem is: even if you report only facts, there is an editorial function in choosing which facts to report, because it is physically impossible to report all facts. So someone can always point to some sort of bias on choosing which facts to report.

[deleted]

Re: Wikipedia: Database Download

#52

Post by icepat »

froh wrote: post #38
icepat wrote: post #34

Earlier quoted context omitted.

My go-to as well. Pre-war steel.

it's not about pre-war. it's about pre-trinity-nuclear tests. which means uncontaminated by atmospheric radioactive isotopes. it happened at the end of ww-ii but that is not the point.

Yes, however it's also an accepted name for it

> Low-background steel, also known as pre-war steel and pre-atomic steel, is any steel produced prior to the detonation of the first nuclear bombs in the 1940s and 1950s.

https://en.wikipedia.org/wiki/Low-background_steel

Re: Wikipedia: Database Download

#54

Post by wyre »

AStonesThrow wrote: post #47
hsuduebc2 wrote: post #23

Earlier quoted context omitted.

True. Musk for example is publicly attacking it for spreading "left-wing lies" because in his wiki page there are statements like "He has been criticized for making unscientific and misleading statements, including COVID-19 misinformation and promoting conspiracy theories, and affirming antisemitic, racist, and transphobic comments." which are just pure facts. It would be nice to have something like this more decentr…

I was perusing some recent discussions on sources with interest. It seems that Wikipedia's intelligentsia have managed to "blacklist" (deprecate or declare "generally unreliable") practically every prominent source of news in the US that is not centrist or leftist. I kid you not; through a process of attrition they've attacked the very reliability and reputation of every source, including Fox News and the like, and t…

[flagged]

Re: Wikipedia: Database Download

#55

Post by willis936 »

miki123211 wrote: post #46

I wonder how easy it would be to make a practically indestructible, everlasting Wikipedia reader. Something using solar for power, with a rugged and water-resistant enclosure, made of extremely high-quality components that won't break for hundreds of years at least. Maybe add an IRDA port for good measure, to make it possible to transfer all the data out somewhat quickly. You could make hundreds of these and put them…

You could even make it radiation tolerant by printing it.

Re: Wikipedia: Database Download

#56

Post by dijit »

mhitza wrote: post #42
jasoncartwright wrote: post #24

If you've got some spare bandwidth & storage then seeding some of the torrents here is a cheap and fun way of helping Wikipedia out. I've served around 20TB of these dumps in the past year. https://meta.wikimedia.org/wiki/Data_dump_torrents

Do you happen to know why wikipedia didn't embrace torrents as the default download method?

Speculating: because torrents are not especially good at dealing with small modifications?

Most people probably won't seed many versions, so it's a losing effort, and you need to allocate a huge chunk of space for each version.

Deduplicating filesystems are sadly not in vogue.

Re: Wikipedia: Database Download

#57

Post by rokobobo »

webdoodle wrote: post #30
thm wrote: post #18

Might also be worthwhile to download a pre ~2023 dump, because Low-background steel.

> download a pre ~2023 dump, because Low-background steel. Military A.I. was likely in use earlier, and since PSYOPS are the most used and most effective weapon in the U.S. Military's arsenal, you absolutely know it was used. It ain't a war crime the first time...

How much do we know about military AI’s capabilities? As in, is there any evidence that the government/military was ahead of big tech on the AI research front?

Re: Wikipedia: Database Download

#58

Post by we0x »

myself248 wrote: post #10

You can also get it as a .zim file for easy offline browsing with Kiwix. The whole enchilada: https://download.kiwix.org/zim/wikipedia/wikipedia_en_all_ma... Other versions: https://library.kiwix.org/#lang=eng&category=wikipedia

It is useful software for offline use and emergencies. For those who may not know, apart from wikis, they also offer offline documentation(Linux distros like ArchWiki, libraries, etc.), medical libraries(Medicine Plus, etc.), and Stack Exchange.

Re: Wikipedia: Database Download

#59

Post by maronato »

JKCalhoun wrote: post #43

Is there a RAG for Wikipedia? I may not be using the term correctly here. In short, I would love a local LLM + Wikipedia snapshot so that I can have an offline, self-hosted ... Hitchhiker's Guide to Earth.

Huggingface has a few datasets of Wikipedia embeddings.

Here’s a few results: https://huggingface.co/search/full-text?q=Wikipedia+embeddin...

And the first result, which is probably what you’ll want to use: https://huggingface.co/datasets/Upstash/wikipedia-2024-06-bg...

I recommend you go for pgvector or a similar self hosted solution to calculate the similarities instead of a service like Vector.

Re: Wikipedia: Database Download

#60

Post by AStonesThrow »

zzzeek wrote: post #50
AStonesThrow wrote: post #47

Earlier quoted context omitted.

I was perusing some recent discussions on sources with interest. It seems that Wikipedia's intelligentsia have managed to "blacklist" (deprecate or declare "generally unreliable") practically every prominent source of news in the US that is not centrist or leftist. I kid you not; through a process of attrition they've attacked the very reliability and reputation of every source, including Fox News and the like, and t…

It's not Wikipedia's fault that the vast majority of right wing media consists of pure propaganda, disinformation, and lies.

[citation needed]

And you know, I wouldn't be surprised if people hurling those accusations somehow believe that the lies and misinformation are one-sided and partisan. As if leftism has some sort of monopoly on Truth and Goodness bestowed from above.

It's really been sickening to see the media outlets just lay down thick trails of bullshit that is designed to distract us, to instill fear, uncertainty, and doubt, to make us hate one another, to keep us hanging on that channel or that subscription for the next tidbit. It's disgusting and manipulative, and the Right has absolutely no monopoly on those tactics.

Wikipedia is simply a microcosm of the prevailing zeitgeist, so they are as likely to cure systemic bias as a leopard can change its spots.

Post reply on HN