Earlier quoted context omitted.
Never forget Aaron Swartz. All he did was send download requests to JSTOR. https://www.youtube.com/watch?v=9vz06QO3UkQ
I'm going to get a ton of hate for this, but he was doing it with the intent to redistribute the content for free. He didn't own the content. There's a big difference. That being said it's very sad what came about of that. I really don't think the FBI needed to be involved.
A Facebook crawler was making 7M requests per day to my stupid website
221–230 of 416 posts
Re: A Facebook crawler was making 7M requests per day to my stupid website
#222Earlier quoted context omitted.
Next time, have some courtesy for the author and the rest of us by requesting via personal exchange over email instead of hijacking the thread and distracting from the conversation.
Boo hiss. There’s plenty of room here for a polite exchange between professionals. Collapse the thread and move on.
That's it for me.
Re: A Facebook crawler was making 7M requests per day to my stupid website
#223We've had the same issue. They were doing huge bursts of tens of thousands of requests in very short time several times a day. The bots didn't identify as FB (used "spoofed" UAs) but were all coming from FB owned netblocks. I've contacted FB about it, but they couldn't figure out why this was happening and didn't solve the problem. I found out that there is an option in the FB Catalog manager that lets FB auto-remove…
I dont really understand what is the issue. On my welcome page (while all other urls are impossible to guess) i give browser something that requires a few seconds of cpu at 100% to crunch. And tracking some user action in between, visting tarpitted urls etc. In last few years no bot came through. Why bother with robots.txt, just give them something to break their teeths... (I would give you the url, but I just dont w…
Re: A Facebook crawler was making 7M requests per day to my stupid website
#224Earlier quoted context omitted.
This pattern needs to die. Whenever I paste a link in any sort of messaging app - iMessages to Slack - it puts a thumbnail and summary, polluting the entire conversation with tons of noise. Messaging apps have no business in looking up the URL. Just let it pass as a link. Fuck everything about this and we need to push back on this nonsense.
I like link previews. I don't like when they are managed server-side instead of client-side.
Re: A Facebook crawler was making 7M requests per day to my stupid website
#225Earlier quoted context omitted.
Next time, have some courtesy for the author and the rest of us by requesting via personal exchange over email instead of hijacking the thread and distracting from the conversation.
> ... and the rest of us ... Please don't try to police the thread and speak for yourself only.
Re: A Facebook crawler was making 7M requests per day to my stupid website
#226Are you sure it's a crawler and not a proxy of some sort? Eg. one of your links is on something high traffic in facebook, and all requests are human, running through fb machines.
Re: A Facebook crawler was making 7M requests per day to my stupid website
#227Earlier quoted context omitted.
So, just to be clear, you are perfectly fine with someone taking something someone else created and violating the terms upon which they were given that thing? e.g. If I took code you wrote and lets say released under an MIT license and claimed I wrote it and didn't give you any credit, and in fact released it under another license entirely, you'd be fine with that?
It's perfectly valid to criticize the original license choice. GPLv3 is a very restrictive license, especially for what is essentially a micro blog (though I dislike the license for most open source software anyway). Add on the original author going after a bit of CSS, not even the main effort of the project in question, and you've got my "petty" comment.
Re: A Facebook crawler was making 7M requests per day to my stupid website
#228Earlier quoted context omitted.
Super optimized in a SEO sense, not in a pleasing the HN crowd kind of sense I'd assume ;)
You're right, but for some reason the whole SEO thing just winds me up. It's my opinion that 'good' SEO makes sites worse for actual people to use.
Re: A Facebook crawler was making 7M requests per day to my stupid website
#229Hi Napolux, It looks like your site is using a theme based on my website ( https://ruudvanasseldonk.com/ , source at https://github.com/ruuda/blog ). That is fine — it is open source after all, licensed under the GPLv3. But I can’t find the source code for your site, and I can’t find any prominent notices saying that you modified my source. Could you please add those?
Re: A Facebook crawler was making 7M requests per day to my stupid website
#230Earlier quoted context omitted.
Sure man no problem. The code for my theme is here BTW with credits To your original blog https://github.com/napolux/coding.napolux.com/
Thanks, I’m flattered to see it be used as inspiration :) I searched quickly but I didn’t find that repository. You might want to link it somewhere in your footer or from a comment in the html.