Viewing profile — pablohoffman

Last time this question was asked on HN was in 2017 ( https://news.ycombinator.com/item?id=15694118 ), a lot has changed in the last 5 years in the world of web scraping (legal lan…

Hey folks, I'm the one interviewed, happy to answer any follow up question here.

Not surprised, I'm pretty low profile :)

Portia from Scrapinghub is 100% open source: http://scrapinghub.com/portia/

We released a similar open source tool for visual scraping, earlier this year, called Portia: https://github.com/scrapinghub/portia It's been getting quite a bit of traction and we…

Not yet.

Yes, you just need to select a different field type ("text", instead of "html").

We are using CDH4.2 and have had a very positive experience so far. Cloudera has in fact been an inspiration for us to follow, you guys have really struck the right balance between…

We are not plainly complaining about MongoDB, nor saying it's useless. We are just explaining why it's a poor choice for a specific use case: storing scraped data. FWIW, we still u…

We went with HBase. Cassandra would have been suitable too, but we already use Hadoop for data processing so it was a natural choice within the infrastructure ecosystem. We will wr…

The title of this post is misleading, it makes you think GitHub is actually broken, when the article is about complaining how their fork feature works.

I initially submitted this post, but then deleted it and resubmitted to the original post on Common Crawl blog: http://news.ycombinator.com/item?id=3208853 I now regret since this …

I found strange that no one mentioned Privnote ( https://privnote.com ), a much older/popular service that does this but more securely since it generates URLs with keys in the frag…

Did it work?

I've been using tiled window managers for 7 years now (currently on awesome, which I switched 3 years ago) and the interface has always remained pretty consistent. I wonder if Linu…

After checking a few registrars, I found they only sell third-level domains (.com.ng, .net.ng), not second-level domains (like "amazi.ng") which is probably what most people here w…