51–60 of 75 posts
Earlier quoted context omitted.
If I can solve the logistics, maybe. I don't have the needed bandwidth and off-site storage at this point.
What sort of sizes are we talking about? I'm thinking if it would be possible to "crowdfund" the storage-costs for a requester-pays s3 bucket for it.
> I'm currently looking for hosting for a large term frequency data file that is necessary for several of the search engine's core functions. I really don't have the bandwidth to serve it myself. It's only a couple of hundred megabytes so it'll probably be solvable somehow. I have a bit of (rented) space available. Most of it on Uberspace. What are the expected traffic requirements? Contact me if you like. Will updat…
Haven't there been estimates that running a globally indexed search engine product costs a billion dollars per year? As always, thank you for marginalia!
Thanks for making this open source. What about the index? Are you open sourcing that also?
If I can solve the logistics of publishing that data, then sure. In its most compressed form it's still of order 100 Gb. The intermediate goal is to have some standardized testing dataset of a couple of hundred megabytes to a gigabyte or so.
I think you should post a ToDo list on the git repo. People can then contribute their skills.
Earlier quoted context omitted.
If I can solve the logistics of publishing that data, then sure. In its most compressed form it's still of order 100 Gb. The intermediate goal is to have some standardized testing dataset of a couple of hundred megabytes to a gigabyte or so.
Cool. Looking forward to see the intermediate dataset. I think you should post a ToDo list on the git repo. People can then contribute their skills.
> I feel GitHub has taken an incredibly toxic turn with its emphasis on social features This the first time I've seen this view of GitHub, and I'm struggling to figure out what specifically he's referring to.
I find a lot of projects because I follow people on github that star or fork really neat projects. I download projects from bitbucket, gitlab, and other self-hosted instances.
I don't know that I've ever contributed a PR back as it's just a more friction-prone process. However, I've opened hundreds of PR's and Issues on github because it's already a part of my daily process.
Thanks again
> I feel GitHub has taken an incredibly toxic turn with its emphasis on social features This the first time I've seen this view of GitHub, and I'm struggling to figure out what specifically he's referring to.
This is a great gift to the community, but I really wish it would be hosted where the community is active. It seems like a lot of people will miss out because they won't be able to discover this. I find a lot of projects because I follow people on github that star or fork really neat projects. I download projects from bitbucket, gitlab, and other self-hosted instances. I don't know that I've ever contributed a PR bac…
Thank god a search engine that try to sell you stuff. Glad it is open sourced. Are they from Scandinavia?
What is Marginalia? Feels like a links database.
It's got an internet search engine, a lo-fi wikipedia mirror, this thing https://search.marginalia.nu/explore/random>, and more.