Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Project Gutenberg – keeps getting better

gutenberg.org

Post reply on HN
Print view View original

251–260 of 300 posts

Re: Project Gutenberg – keeps getting better

#251

Post by gluejar »

Guestmodinfo wrote: post #182
JSeiko wrote: post #2

Hi! I'm one of the programmers at Gutenberg. We've been improving the site a lot over the past few months (and more is coming!). If you haven't visited the page recently, it's worth checking out again: https://www.gutenberg.org/

Hi for the past 20 years I have known about Project Gutenberg and I used to read a lot from it. One of the obstacle that I face is that there is no way to arrange the books in the order of their original publication. Do you know of any such way. Surely we can arrange the books by their release date on Gutenberg but it has long baffled me as it feels to me the most useless way of sorting the books. Thank you for Proje…

only 20% of our books have original publication data in the db. We have a project to add another 40% or so from another database, let us know if you want to help. reply

Re: Project Gutenberg – keeps getting better

#252

Post by gluejar »

svat wrote: post #129
gluejar wrote: post #77

Earlier quoted context omitted.

wikipedians, please help update this article.

In what way? And from what sources? (Wikipedia as a tertiary source is supposed to be a summary of information present in reliable secondary sources — see for instance https://en.wikipedia.org/wiki/Wikipedia:Based_upon . So if the information on the Wikipedia article is incomplete or out of date, where is the correct information available?)

There's quite a lot of information here: https://www.gutenberg.org/about/ All our text is now utf-8. No Plucker! Almost every book is HTML(5).

Re: Project Gutenberg – keeps getting better

#253

Post by gluejar »

cosmos0072 wrote: post #135

From Italy, https://www.gutenberg.org/ gives a 404 error and https://gutenberg.org/ opens a very official-looking page stating "police notice. This site is under judicial seizure" and references a sentence number: "criminal proceedings 52127/20 R.N.R.I. tribunal of Rome" Any idea what's happening? I thought PG published public domain books...

A silly legal tribunal confused PG with pirate sites. We sent the tribunal a letter pointing out their error but it was ignored. The block was served on local dns providers so many Italian users evade the block by using DNS from Google or Cloudflare.

Re: Project Gutenberg – keeps getting better

#256

Post by Guestmodinfo »

gluejar wrote: post #251
Guestmodinfo wrote: post #182

Earlier quoted context omitted.

Hi for the past 20 years I have known about Project Gutenberg and I used to read a lot from it. One of the obstacle that I face is that there is no way to arrange the books in the order of their original publication. Do you know of any such way. Surely we can arrange the books by their release date on Gutenberg but it has long baffled me as it feels to me the most useless way of sorting the books. Thank you for Proje…

only 20% of our books have original publication data in the db. We have a project to add another 40% or so from another database, let us know if you want to help. reply

Yes I am willing to help. Plz include me in your efforts. Thank you for this

Re: Project Gutenberg – keeps getting better

#257

Post by lucb1e »

miki123211 wrote: post #214
lucb1e wrote: post #143

Earlier quoted context omitted.

I'm only a small-scale sysadmin but the way that I understand the internet is that you send abuse notifications to the IP address block owner and, if it doesn't get resolved, you block. The whois/rdap database reveals which IPs all belong to the same hosting provider or ISP, so you can summarize that all to one list of IP addrs + timestamps per some time period The ISP actually knows which subscriber is on that line,…

The problem with this approach is that modern scrapers use hordes of residential proxies and quickly rotate through IP addresses which belong to ASes you get a lot of real traffic from. There's nothing you can do if the ISP won't take any action against the customer.

I know. All the more reason to do it, right? If an ISP can't keep its network clean, then allowing them to send traffic onto the web is just asking for the problem to continue

Show people a useful error, such as "You are using [ISP name] which sends large volumes of abusive traffic (think of spam and DDoS). They allow the attackers to hop around points across their entire network so we cannot block the abusers more selectively. Despite our attempts to contact them, the abuse continues in volumes which we do not see from other ISPs. To access our corner of the internet, use a different ISP. You could try mobile data instead of Wi-Fi or vice versa.", and they can make their own choices about staying with this ISP if more and more websites show this sort of error

If everyone tries to identify people piecemeal, we all need to implement ~200 different identification systems (assuming each country has a central system that everyone is signed up to in the first place), or rely on algorithms to tell who is a bot (I'm currently being misidentified on a daily basis and I'm, eh, not a bot. Trying to buy public transport tickets is currently difficult, for example, because the monopolist in my country blocks me after a few route queries when using a Google browser, and 0 queries from Firefox)

Re: Project Gutenberg – keeps getting better

#258

Post by lucb1e »

tangledhelix wrote: post #233
miki123211 wrote: post #214

Earlier quoted context omitted.

The problem with this approach is that modern scrapers use hordes of residential proxies and quickly rotate through IP addresses which belong to ASes you get a lot of real traffic from. There's nothing you can do if the ISP won't take any action against the customer.

Worse than that - even if they would take action, you can't possibly orchestrate filing all of the complaints. It's a drown-in-quicksand problem, you can't fight quicksand one grain at a time.

> you can't possibly orchestrate filing all of the complaints

To the ISPs? Each IP range has an abuse email address registered and this is specifically exempt from rate limiting at RIPE's WHOIS server. Not sure how it is in other RIRs but I just happen to know of this policy

You can automate the whole thing, provided that you have a reliable way of identifying the undesired traffic which you need anyway for being able to block it by any means. The trouble is in user identification (they'll just use a new IP address from that ISP or hosting provider if you don't tell the provider about the problematic user)

Re: Project Gutenberg – keeps getting better

#259

Post by lucb1e »

Jolter wrote: post #163
lucb1e wrote: post #143

Earlier quoted context omitted.

I'm only a small-scale sysadmin but the way that I understand the internet is that you send abuse notifications to the IP address block owner and, if it doesn't get resolved, you block. The whois/rdap database reveals which IPs all belong to the same hosting provider or ISP, so you can summarize that all to one list of IP addrs + timestamps per some time period The ISP actually knows which subscriber is on that line,…

It’s effective against teenagers maybe. Not so much against Amazon, Meta or wherever botnet/crawler is coming out of China these days from up-and-coming AI companies.

Then block all of Amazon, Meta, or wherever botnet/crawling traffic is coming from that doesn't honor robots.txt, sends DDoS reflection traffic, submits SMTP messages (in large volumes, not just probing) for domains they're not authorized for with SPF, or whatever else applies to the protocol you're using

If they can't keep their ranges clean to a reasonable degree, their customers will need to move if they want to access your part of the internet. New sign-ups will always be hard, so some amount of abuse is expected, but if it's the same abuse traffic for weeks after you've notified them, well, it stops being your problem at some point

Re: Project Gutenberg – keeps getting better

#260

Post by tangledhelix »

acabal wrote: post #225
idoubtit wrote: post #201

Earlier quoted context omitted.

Not the GP, but I also have mixed feelings about Standard Ebooks. They modernise texts for American readers. This means changing the punctuation, merging some words, altering the syntax, etc. When I read an old novel, written two centuries ago in England, the little differences to modern English are part of the charm, and I certainly don't want any Americanism mixed in. For one of my favorite novels, The Forsyte saga…

SE editor in chief here. What you describe is incorrect. The only thing we do is very light sound-alike spelling modernization, like "to-night" -> "tonight". We do not do things like change from en-GB to en-US, replace old words with different modern words, or change text for "American readers", whatever that means. I have no idea where you got that impression. I personally worked on the Forsyte saga. If you think so…

One could argue that this falls into the previous poster's thought about "the little differences to modern English are part of the charm" ...
Post reply on HN