Tracking supermarket prices with Playwright • Hacker News Board

Re: Tracking supermarket prices with Playwright

#211

Post by bob_theslob646 » Sun, Aug 11, 2024, 10:31 AM UTC

bane wrote: post #179

RasmusFromDK wrote: post #104

Nice writeup. I've been through similar problems that you have with my contact lens price comparison website https://lenspricer.com/ that I run in ~30 countries. I have found, like you, that websites changing their HTML is a pain. One of my biggest hurdles initially was matching products across 100+ websites. Even though you think a product has a unique name, everyone puts their own twist on it. Most can be handled w…

> One of my biggest hurdles initially was matching products across 100+ websites. Even though you think a product has a unique name, everyone puts their own twist on it. Most can be handled with regexes, but I had to manually map many of these (I used AI for some of it, but had to manually verify all of it) In the U.S. at least, big retailers will have product suppliers build slightly different SKUs for them to make…

Would that still matter if you just compare by description?

Re: Tracking supermarket prices with Playwright

#212

Post by bob_theslob646 » Sun, Aug 11, 2024, 10:42 AM UTC

PigiVinci83 wrote: post #160

Nice article, enjoyed reading it. I’m Pier, co founder of https://Databoutique.com , which is a marketplace for web scraped data. If you’re willing to monetize your data extractions, you can list them on our website. We just started with the grocery industry and it would be great to have you on board.

This looks like a really cool website but my only critique is how are you verifying that the data is actually real and not just generated randomly?

Re: Tracking supermarket prices with Playwright

#213

Post by kinderjaje » Sun, Aug 11, 2024, 1:07 PM UTC

A few years ago, we had a client and built a price-monitoring app for women's beauty products. They had multiple marketplaces, and like someone mentioned before, it was tricky because many products come in different sizes and EANs, and you need to be able to match them.

We built a system for admins so they can match products from Site A with products from Site B.

The scraping part was not that hard. We used our product https://automatio.co/ where possible, and where we couldn't, we built some scrapers from scratch t using simple CURL or Puppetteer.

Thanks for sharing your experience, especially that I didn't use Playwright before.