Is data shared between accounts if two accounts both want to retrieve information from the same exact URL?
Show HN: Web scraping page analyzer
11–20 of 52 posts
Re: Show HN: Web scraping page analyzer
#12Is there a way to access an authenticated web site?
Re: Show HN: Web scraping page analyzer
#13Is there a way to access an authenticated web site?
Sadly not for now. Our company has a solution for that (for some websites), but currently this tool does not have this functionality, since I wanted it be as simple as possible. Maybe in the future.
Re: Show HN: Web scraping page analyzer
#14Is data shared between accounts if two accounts both want to retrieve information from the same exact URL?
Nope, there is no caching now, every run of the tool has a single instance and writes the output into separate file. I'm using it to test stability of cloud when multiple users are using it and to test proxies. It would not be much of a test if one user opened the demo page and then every other use would just get the results from a file. But when I'm happy with how it works I will add caching.
Re: Show HN: Web scraping page analyzer
#15Earlier quoted context omitted.
Are you Jaroslov ? :)
Jaroslav, yes, I'm the author. Did you notice any problems or ways how I can improve it?
Not that i can see from a surface view, i think documentation can be improved :). Personally like the idea of APIFY, saw it a few months ago. Are you guys hiring ? :D
Re: Show HN: Web scraping page analyzer
#16Just wanted to say great tool!
Thank you, still lot's of things to improve (for example 404 handling) but it's great to see positive feedback.
Re: Show HN: Web scraping page analyzer
#17I entered a URL and pressed enter, wondering why nothing happened. Only then did I scroll down to find the 'Analyze' button. I wasn't looking for specific attributes, and the strong color contrast of that section made it look like nothing else of interest would come below.
Re: Show HN: Web scraping page analyzer
#18I entered a URL and pressed enter, wondering why nothing happened. Only then did I scroll down to find the 'Analyze' button. I wasn't looking for specific attributes, and the strong color contrast of that section made it look like nothing else of interest would come below.
Oh... Clearly I need to work on my UX skills, I will improve that in next iteration.
Re: Show HN: Web scraping page analyzer
#19Awesome tool! How do you handle scraping data that's hiding behind layers of ~fancy~ JS libraries? Is it as simple as triggering click events, pausing for loading, and then grabbing the information?
Re: Show HN: Web scraping page analyzer
#20Nice work! It seems that it only supports microdata and not RDFa at the moment?