Show HN: Sitebulb, a website crawler and auditor for SEOs
1–10 of 75 posts
Re: Show HN: Sitebulb, a website crawler and auditor for SEOs
#2Hey HN,
Sitebulb is a desktop crawler for Windows and Mac, specifically designed for SEO consultants/agencies. Me and my business partner have been building it for the last 2 years or so, and we're finally launching it today.
It's main differentiating factors: 1. Scale – it can comfortably crawl 500,000+ page websites despite being a desktop program. 2. Reporting – it does a lot of data manipulation and processing so you don't have to. 3. Visualization – it has tons of useful graphs, including the Crawl Maps, which help you visualise site structure.
Our aim was to give it the reporting capability of a SaaS crawler, with the convenience of a desktop crawler.
Looking forward to hearing your feedback on our new product. Thanks, HN community!
Re: Show HN: Sitebulb, a website crawler and auditor for SEOs
#3Hey HN, Sitebulb is a desktop crawler for Windows and Mac, specifically designed for SEO consultants/agencies. Me and my business partner have been building it for the last 2 years or so, and we're finally launching it today. It's main differentiating factors: 1. Scale – it can comfortably crawl 500,000+ page websites despite being a desktop program. 2. Reporting – it does a lot of data manipulation and processing so…
Interesting that it’s all a desktop app. What problem do you think this solves compared to something that runs in the cloud ? Apart from the cost structure, I can’t think of anything myself.
Re: Show HN: Sitebulb, a website crawler and auditor for SEOs
#4Hey HN, Sitebulb is a desktop crawler for Windows and Mac, specifically designed for SEO consultants/agencies. Me and my business partner have been building it for the last 2 years or so, and we're finally launching it today. It's main differentiating factors: 1. Scale – it can comfortably crawl 500,000+ page websites despite being a desktop program. 2. Reporting – it does a lot of data manipulation and processing so…
Interesting that it’s all a desktop app. What problem do you think this solves compared to something that runs in the cloud ? Apart from the cost structure, I can’t think of anything myself.
The other big thing, in comparison to cloud software, is convenience. You can setup a crawl and start it running -and see URLs being crawled - within a minute.
On cloud software that's simply not possible, due to the way that everything is scheduled.
There are a few other small things, such as being able to view Audits offline (what we call 'train mode').
The cost structure can be a big limiting factor though, especially for smaller companies. Sitebulb effectively remove all limitations around number of domains, number of projects, total number of URLs crawled etc...
Re: Show HN: Sitebulb, a website crawler and auditor for SEOs
#5I've been testing Sitebulb for a few months now and I'm really impressed, you've done solid work guys :) good luck with the launch
Re: Show HN: Sitebulb, a website crawler and auditor for SEOs
#6I've been testing Sitebulb for a few months now and I'm really impressed, you've done solid work guys :) good luck with the launch
Thank you! And thanks for the beta feedback, it was super helpful.
Re: Show HN: Sitebulb, a website crawler and auditor for SEOs
#7Hey HN, Sitebulb is a desktop crawler for Windows and Mac, specifically designed for SEO consultants/agencies. Me and my business partner have been building it for the last 2 years or so, and we're finally launching it today. It's main differentiating factors: 1. Scale – it can comfortably crawl 500,000+ page websites despite being a desktop program. 2. Reporting – it does a lot of data manipulation and processing so…
Interesting that it’s all a desktop app. What problem do you think this solves compared to something that runs in the cloud ? Apart from the cost structure, I can’t think of anything myself.
The one advantage that I can think of is this can run on websites that are in development and not accessible to something running in the cloud. A lot of enterprise websites have their dev/stage behind a VPN and being able to run this against those without having to find out how to jump through hoops would be really nice (which is looks like this is capable of doing since you just need to feed it a URL). On top of that you also don't have to worry about what they're doing with the data output by the program on their server.
Re: Show HN: Sitebulb, a website crawler and auditor for SEOs
#8Ditto on the sitebulb love. It's a well-done product and a welcome challenger to the status-quo of SEO software out there.
I will say the cost gives me hesitation but you've put a lot of work into it so I understand the justification.
For the visualization, does the crawl map limit the connections? I was expecting to see more of a web with pages linked from the entire site. Can you tell us more about that?
Thanks
Re: Show HN: Sitebulb, a website crawler and auditor for SEOs
#9This looks really good. My only comment I have is that if I am purchasing a piece of desktop software I expect there to be at least the option of an annual license.
Monthly feels more like a service than a product and I think of desktop software as a product.
But I realize Desktop software pricing is evolving so this may just be old school thinking on my part.
Also, if I'm a Mac user (I am) I expect to be able to buy / pay through the app store. I know it sucks because Apple takes a huge cut but there is a level of trust (misplaced or not) and ease of use that comes with the app store. I am much more likely to use an app if it is in there. Just like on Linux I am much more likely to use an app if it is in a package manager.
Speaking of that, I don't know what desktop framework was used but, is there anything preventing a Linux version?
Re: Show HN: Sitebulb, a website crawler and auditor for SEOs
#10Ditto on the sitebulb love. It's a well-done product and a welcome challenger to the status-quo of SEO software out there. I will say the cost gives me hesitation but you've put a lot of work into it so I understand the justification. For the visualization, does the crawl map limit the connections? I was expecting to see more of a web with pages linked from the entire site. Can you tell us more about that? Thanks
Price is always a difficult one trying to get the cost/value balance right. We did some pricing sensitivity testing before launch so I'm hopeful we've not got it too wrong.
Regarding Crawl Maps, yeah it does have some limitations on, which I've written about here - https://sitebulb.com/resources/guides/crawl-maps-faqs/
Although from your comment I think you might be thinking it is a link map, rather than a crawl map. So with the Crawl Map it is mapping out how each URL/node was found when the crawler traversed the site. So each node will only ever have one edge/link.
A link map ends up a LOT more messy, although it's on our roadmap to try and build one of these too!