SiteOne Crawler – big tool with no users
crawler.siteone.io
SiteOne Crawler – big tool with no users
1–10 of 18 posts
Re: SiteOne Crawler – big tool with no users
#2For some odd reason, the tool didn't get any replies on Reddit, HN, AlternativeTo.net or GitHub, despite looking fairly sophisticated. The dev is also non-professional.
Maybe I am bad at digging, but I couldn't find anything about siteone.io. The entire situation is strange.
Older discussion with one reply: https://news.ycombinator.com/item?id=41292651
Re: SiteOne Crawler – big tool with no users
#3For some odd reason, the tool didn't get any replies on Reddit, HN, AlternativeTo.net or GitHub, despite looking fairly sophisticated. The dev is also non-professional. Maybe I am bad at digging, but I couldn't find anything about siteone.io. The entire situation is strange. Older discussion with one reply: https://news.ycombinator.com/item?id=41292651
yeah and it looks really full-featured and well-done, while many other crawlers/scrapers with way less get way more stars
Re: SiteOne Crawler – big tool with no users
#4For some odd reason, the tool didn't get any replies on Reddit, HN, AlternativeTo.net or GitHub, despite looking fairly sophisticated. The dev is also non-professional. Maybe I am bad at digging, but I couldn't find anything about siteone.io. The entire situation is strange. Older discussion with one reply: https://news.ycombinator.com/item?id=41292651
Very small amount of sourcecode: Under 20k by my count. Are they outsourcing the actual crawl to a different tool?
Re: SiteOne Crawler – big tool with no users
#5For some odd reason, the tool didn't get any replies on Reddit, HN, AlternativeTo.net or GitHub, despite looking fairly sophisticated. The dev is also non-professional. Maybe I am bad at digging, but I couldn't find anything about siteone.io. The entire situation is strange. Older discussion with one reply: https://news.ycombinator.com/item?id=41292651
The parent, SiteOne, seems to be a well-established IT company in The Czech Republic. Hopefully, they will gain more traction in the near future.
Re: SiteOne Crawler – big tool with no users
#6For some odd reason, the tool didn't get any replies on Reddit, HN, AlternativeTo.net or GitHub, despite looking fairly sophisticated. The dev is also non-professional. Maybe I am bad at digging, but I couldn't find anything about siteone.io. The entire situation is strange. Older discussion with one reply: https://news.ycombinator.com/item?id=41292651
Very small amount of sourcecode: Under 20k by my count. Are they outsourcing the actual crawl to a different tool?
It seems to contain the crawler: https://github.com/janreges/siteone-crawler/blob/main/src/Cr...
Re: SiteOne Crawler – big tool with no users
#7Hi andrei-akopian. I am the main author of SiteOne Crawler and thank you for this post!
For about 20 years, I have been leading the development and infrastructure of the Czech webdev company SiteOne, so I wouldn't say that I am not a professional ;) I have around 50,000 hours of practice. But I definitely know a lot of better programmers than me and many of them are at SiteOne :)
However, after the extremely premature birth of my son, I had to change my role at SiteOne, but all the more I wanted to help my colleagues with a useful tool that we generally lacked and at the same time I wanted to give myself joy in difficult times.
I want to implement a number of other useful improvements into the crawler (some of them are also described in the documentation and roadmap) and work on its promoting in parallel. The more people will use it, the more it will help to optimize the website, the more it will help developers and testers with various needs, the more joy it will give me :)
By the way, the GUI application https://github.com/janreges/siteone-crawler-gui is just a visual wrapper over the command-line version of SiteOne Crawler and integrates it into itself.
Re: SiteOne Crawler – big tool with no users
#8For some odd reason, the tool didn't get any replies on Reddit, HN, AlternativeTo.net or GitHub, despite looking fairly sophisticated. The dev is also non-professional. Maybe I am bad at digging, but I couldn't find anything about siteone.io. The entire situation is strange. Older discussion with one reply: https://news.ycombinator.com/item?id=41292651
Very small amount of sourcecode: Under 20k by my count. Are they outsourcing the actual crawl to a different tool?
SiteOne Crawler offers 2 open-source tools:
1) command-line version (which is the core of the crawler itself): https://github.com/janreges/siteone-crawler
2) a graphical application for Win/macOS/Linux, which is just a wrapper over the command-line version and displays the output from it through Xterm: https://github.com/janreges/siteone-crawler-gui
Re: SiteOne Crawler – big tool with no users
#9For some odd reason, the tool didn't get any replies on Reddit, HN, AlternativeTo.net or GitHub, despite looking fairly sophisticated. The dev is also non-professional. Maybe I am bad at digging, but I couldn't find anything about siteone.io. The entire situation is strange. Older discussion with one reply: https://news.ycombinator.com/item?id=41292651
yeah and it looks really full-featured and well-done, while many other crawlers/scrapers with way less get way more stars
Yes, the tool has a lot of features, settings and usage options. I believe that many people will like the tool and help promote it themselves.
Re: SiteOne Crawler – big tool with no users
#10For some odd reason, the tool didn't get any replies on Reddit, HN, AlternativeTo.net or GitHub, despite looking fairly sophisticated. The dev is also non-professional. Maybe I am bad at digging, but I couldn't find anything about siteone.io. The entire situation is strange. Older discussion with one reply: https://news.ycombinator.com/item?id=41292651
Maybe people ignore it because there are already many tools like this. I was tempted to not look, but checked it out anyway