Viewing profile — stummjr

This repo may be helpful to you: https://github.com/satwikkansal/wtfpython It’s a collection of snippets to explain behaviors that may be considered unexpected.

I disagree with this article in so many ways. I have a CS degree and I work with many people who don’t, and they are just as good as (or even better than) me at all the points rais…

Ngrok is just awesome! A huge shout out to the developers!

Crawl-delay is not in the standard robots.txt protocol, and according to Wikipedia, some bots have different interpretations for this value. That's why maybe many websites don't ev…

That's kind of what Scrapy's AUTO_THROTTLE middleware does.

Yeah, but that's not just because of web scraping. Plagiarism has been an issue for centuries.

Scrapy is asynchronous, but it provides many settings that you can use to avoid DDoS a website, such as limiting the amount of simultaneous requests for each domain or IP address. …

Scrapinghub is 100% remote from day zero. Nowadays there are ~140 people spread around the world, covering almost all timezones.

Hey! I work for Scrapinghub. Feel free to ask any questions.

Hey, Valdir from Scrapinghub here! Feel free to ask any questions you might have about the platform.

This is such an inspiring story! I guess a lot of people today go the opposite way, renting or buying places where they know there is a decent internet access. I'm guilty about it.…

Hey, I'm the author. Feel free to ask any questions.

Hey, of course. We are glad that you are interested in testing Kumo. Please email us (help at scrapinghub . com) your user id, organization ids and the project ids you want to migr…

Hey, I'm the author of this post. Feel free to ask any questions or to suggest topics for the next month's post on the "Scrapy tips from the pros" series. :)