Show HN: I scraped 200M Shopify products to build a search engine

So, Shopify does have something similar called the shop.app which is only for Shopify stores. My best guess is that the e-commerce platforms are solving a different problem: store creation. I'll also note that each individual e-commerce platform isn't incentivized to aggregate and send users to stores not built with their platform. When we launched in December, we only had Shopify stores. Now we have Shopify and WooCommerce, and working on support for Squarespace and Wix sites as well.

For user traffic, the primary strategy is to build features with a viral loop back to the product. I mentioned this in another comment but we have have the concept of making shareable 'lists' of products you like. This is already working for us, at a small scale.

The general plan is to aggregate as many e-commerce products as possible from different platforms, keep improving the search experience, add automated filters to ensure high quality products, and then keep layering in features that drive a viral loop back to Agora.

For users (demand-side), the problem we solve: There are user groups that currently have a fragmented experience. For example, a specialized solar technician (just throwing out a random example) has to look through a handful of speciality stores to find and compare products that are only sold there. I think there are specific user-groups we can go after that really feel the pain right now of this process. Additionally, as the number of e-commerce platforms increases, it becomes tougher for every day users to find products they are looking for. They have to either go to Amazon or go store-by-store to discover products. The shop.app solves it for Shopify store but there's also millions of sites on WooCommerce, Squarespace, Wix, etc. We get around the empty-state problem with the crawler and now have merchants signing up to get their products indexed.

For merchants (supply-side), the problem we solve: If they sell on their own website, they have to compete against non-product pages on Google. For example, if you sell "red shoes" on your own site, you have to compete against the IMDB entry for the movie "Red Shoes" for people to find you. Additionally, if they sell on their own website and use Amazon (or any physical retailer) for distribution, they give up a percent of their margin. This increases your sell-through but is a smaller amount of money in your pocket.

I'll note that I've seen this problem first hand. In 2016 I launched a game called The 2016 Election Game, which was like Cards Against Humanity for the 2016 US elections. Sold about 5k units fulfilling order myself. And then again in 2020 called DoneWith2020, which was like Cards Against Humanity for the absurdity of the 2020 year. Sold about 34k units using mostly dropshipping. I remember losing out on search / discovery by choosing to sell on my own store but made a much higher margin on each sale (i.e. made about $15 on each $24.99 unit sold). We did work with a company to get on Amazon but always preferred people purchasing on our own site. It was also really hard to get high intent traffic to my store from ads. Would have been nice to send people searching for "funny card game" to my site. Now if everyone has my same dark sense of humor once they landed on the site, is up for debate.

The goal isn't to catch up to Shopify, WooCommerce, etc. but to rather aggregate products across platforms. I do think we can index most of e-commerce products sold on these platforms (my best guess is that it's somewhere between 10 - 20 billion products). This is obviously a very tough data hosting and search problem at that scale. Even Mongo, which is what we use as our primary database, has a limit of 2 billion records.

I agree that it's a traffic problem. Everything comes down to getting users. Based on the number of merchants signing up, we are validating that others have the supply-side problem. It's a matter of nailing down the demand-side problem (i.e. finding the right user groups, building the right features for engagement, etc). We use 'search' as the conduit, assuming that exceptional search will lead to more traffic. But agreed that there are several other factors to solve.

Generally, Google Shopping shows big retailers or ads. E-commerce stores have a tough time competing against these big retailers, on both Google Shopping and a normal Google Search.

That all said, we are running tests to track engagement of a "search result page" indexed by Google or Bing. For example, searching for a "backpack" and then landing on the below link with selection to choose from.

https://www.searchagora.com/search?query=backpack