61–70 of 286 posts
Cool! But how did you get the initial dataset of 643,000+ Shopify stores (data as per your “About” page) in the first place, to then scrape the products from their /products.json feeds? Or did you just try a huge list of domain names at random?
https://www.shopify.com/robots.txt lists a lot of sitemap files, which tend to be a good starting point.
I'm sorry, but I have to question where this heartfelt story about looking out for your wife is in any way real? The website certainly doesn't look like a side project, it has a fully fledged system for merchants to advertise on Agora for a fee, an affiliate system offering $50 commissions to onboard merchants and the ToS and Privacy policy link to a website with the following mission statement: > We buy, build, and…
Earlier quoted context omitted.
https://www.shopify.com/robots.txt lists a lot of sitemap files, which tend to be a good starting point.
Did this suddenly get changed? Nothing but "# ,: # ,' | # / : # --' / # \/ />/ # /" is shown now.
Earlier quoted context omitted.
Weird. I think it did change. Google cache shows a 2229 line file: https://webcache.googleusercontent.com/search?q=cache%3Ahttp...
Seems it might be looking at the referrer. Loading https://www.shopify.com/robots.txt from clicking the link shows the weird line while opening it in a private browser window shows the right one.
I'm sorry, but I have to question where this heartfelt story about looking out for your wife is in any way real? The website certainly doesn't look like a side project, it has a fully fledged system for merchants to advertise on Agora for a fee, an affiliate system offering $50 commissions to onboard merchants and the ToS and Privacy policy link to a website with the following mission statement: > We buy, build, and…
Agora also doesn't return red shoes for the search query "red shoes". Seems like you haven't fully solved the problem yet :) From a technical perspective, crawling 25M products is impressive but the search itself doesn't provide much value to me. I already use large e-commerce sites (amazon, wallmart, ...) and targeted ones (Nordstrom, SSENSE, ...). Sure I may not be searching through all the shopify, wix stores but…
[flagged]
Earlier quoted context omitted.
https://www.shopify.com/robots.txt lists a lot of sitemap files, which tend to be a good starting point.
Did this suddenly get changed? Nothing but "# ,: # ,' | # / : # --' / # \/ />/ # /" is shown now.
# / # ,:
# ,' |
# / :
# --' /
# \/ />/
# /
# ,: # ,' | # / : # --' / # \/ />/ # /
Ref: https://html.spec.whatwg.org/multipage/parsing.html https://developer.mozilla.org/en-US/docs/Web/CSS/white-space...
I'm sorry, but I have to question where this heartfelt story about looking out for your wife is in any way real? The website certainly doesn't look like a side project, it has a fully fledged system for merchants to advertise on Agora for a fee, an affiliate system offering $50 commissions to onboard merchants and the ToS and Privacy policy link to a website with the following mission statement: > We buy, build, and…
OP here. Yup, haven't launched the holding company yet but idea is to have an LLC for all my software projects. Still a work-in-progress in both thinking and execution. Agora specifically came from a personal need and is obviously still an MVP project. Spun up a Merchant Page and Affiliate Program page in a few hours on Webflow using a template. There is a merchant dashboard built but the 'affiliate program' is a tes…