Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A Facebook crawler was making 7M requests per day to my stupid website

coding.napolux.com

Post reply on HN
Print view View original

181–190 of 416 posts

Re: A Facebook crawler was making 7M requests per day to my stupid website

#181

Post by Nextgrid »

mikece wrote: post #56
systemvoltage wrote: post #13

Earlier quoted context omitted.

This pattern needs to die. Whenever I paste a link in any sort of messaging app - iMessages to Slack - it puts a thumbnail and summary, polluting the entire conversation with tons of noise. Messaging apps have no business in looking up the URL. Just let it pass as a link. Fuck everything about this and we need to push back on this nonsense.

A funny story (well, funny because it didn't happen to me) was chronicled on a recent episode of Corey Quinn's "Whiteboard Confessions" -- https://www.lastweekinaws.com/podcast/aws-morning-brief/whit... -- where Slackbot's auto URL unfurling feature "clicked" an SNS alert unsubscribe link when a tech posted the a report including that link into a slack channel. If nobody lost their job over this triggering a SEV-1 al…

As far as I know the iOS implementation mitigates that by generating the preview on the sender side. The receiver side will not make a request to the URL unless they manually click on it.

Re: A Facebook crawler was making 7M requests per day to my stupid website

#183

Post by jasonlotito »

thelean12 wrote: post #172
ruuda wrote: post #163

Earlier quoted context omitted.

I checked the source and it does indeed mention https://wordpress.org/themes/blankslate/ , but judging from the screenshot there, that theme is just really a blank theme with no style at all, and it was used to include a different stylesheet. The real style is at https://coding.napolux.com/wp-content/themes/coding.napolux.... , which looks like normalize.css followed by a Wordpress adaptation of my stylesheet. The si…

Come on, this is super petty. And that includes your license choice and your enforcement for this.

So, just to be clear, you are perfectly fine with someone taking something someone else created and violating the terms upon which they were given that thing?

e.g. If I took code you wrote and lets say released under an MIT license and claimed I wrote it and didn't give you any credit, and in fact released it under another license entirely, you'd be fine with that?

Re: A Facebook crawler was making 7M requests per day to my stupid website

#184

Post by ruuda »

RhodesianHunter wrote: post #142
ruuda wrote: post #140

Hi Napolux, It looks like your site is using a theme based on my website ( https://ruudvanasseldonk.com/ , source at https://github.com/ruuda/blog ). That is fine — it is open source after all, licensed under the GPLv3. But I can’t find the source code for your site, and I can’t find any prominent notices saying that you modified my source. Could you please add those?

Despite how it sounds, I ask this with zero judgment and pure curiosity. Why do you care?

I made my site open source so others who like it can take a look at how it works, or give it their own twist. Visitors of my site can see that they have that freedom, but visitors of an adaptation might not know if it’s not stated anywhere.

Re: A Facebook crawler was making 7M requests per day to my stupid website

#185

Post by Drakim »

jimmaswell wrote: post #177
mastermojo wrote: post #160

Earlier quoted context omitted.

The author probably cares about either copy-left or the right to tinker. Some people believe it's important and others don't. Basically any derivative product out of GPL3 code needs to either be open-source/copy-left, or if it uses the code as a library needs to let end users substitute that library for their own version.

Worse than unimportant, I find copyleft harmful to the developer community as a whole. Software is hardly "free" when you have to bend to the author's demands to license your entire rest of your project under their ideology if you want to use it, and I think we should stop calling it so. Maybe "Conditionally Free Software" instead. You're hardly helping the world releasing libraries in terms that nobody but hobbyists…

Copyleft is about maximizing the users's rights, not the developers's rights. Companies can't take linux, put it on a router for sale, and then say that their customers/users aren't allowed to know what's going on on the box in terms of backdoors and spying. The users have a right to look at the source code if they wish.

Re: A Facebook crawler was making 7M requests per day to my stupid website

#186

Post by thelean12 »

jasonlotito wrote: post #183
thelean12 wrote: post #172

Earlier quoted context omitted.

Come on, this is super petty. And that includes your license choice and your enforcement for this.

So, just to be clear, you are perfectly fine with someone taking something someone else created and violating the terms upon which they were given that thing? e.g. If I took code you wrote and lets say released under an MIT license and claimed I wrote it and didn't give you any credit, and in fact released it under another license entirely, you'd be fine with that?

It's perfectly valid to criticize the original license choice.

GPLv3 is a very restrictive license, especially for what is essentially a micro blog (though I dislike the license for most open source software anyway).

Add on the original author going after a bit of CSS, not even the main effort of the project in question, and you've got my "petty" comment.

Re: A Facebook crawler was making 7M requests per day to my stupid website

#187

Post by oh_sigh »

TheSpiceIsLife wrote: post #180
johnwheeler wrote: post #165

Earlier quoted context omitted.

Next time, have some courtesy for the author and the rest of us by requesting via personal exchange over email instead of hijacking the thread and distracting from the conversation.

Boo hiss. There’s plenty of room here for a polite exchange between professionals. Collapse the thread and move on.

The question is why does everyone need to read what really could first be something done on a private communication.

Re: A Facebook crawler was making 7M requests per day to my stupid website

#188

Post by smallgovt »

lun4r wrote: post #29

We've had the same issue. They were doing huge bursts of tens of thousands of requests in very short time several times a day. The bots didn't identify as FB (used "spoofed" UAs) but were all coming from FB owned netblocks. I've contacted FB about it, but they couldn't figure out why this was happening and didn't solve the problem. I found out that there is an option in the FB Catalog manager that lets FB auto-remove…

>> The bots didn't identify as FB (used "spoofed" UAs)

That's surprising. What were the spoofed user agents that they used?

We've run into this issue also, but all Facebook bot activity had user agents that contained the string "facebookexternalhit".

Re: A Facebook crawler was making 7M requests per day to my stupid website

#190

Post by paulpauper »

lun4r wrote: post #29

We've had the same issue. They were doing huge bursts of tens of thousands of requests in very short time several times a day. The bots didn't identify as FB (used "spoofed" UAs) but were all coming from FB owned netblocks. I've contacted FB about it, but they couldn't figure out why this was happening and didn't solve the problem. I found out that there is an option in the FB Catalog manager that lets FB auto-remove…

twitter does the same thing. it sends a bunch of spoofed US visitors from Korea and Germany and US. The bots are spoofed to make it harder to filter them.
Post reply on HN