Live data from Hacker News

Tracking supermarket prices with Playwright

sakisv.net

11–20 of 213 posts

Re: Tracking supermarket prices with Playwright

#11

Very cool! I did something similar in Canada ( https://grocerytracker.ca/ )

Oh nice!

A thorny problem in my case is that the same item is named in 3 different ways between the 3 supermarkets which makes it very hard and annoying to do a proper comparison.

Did you have a similar problem?

Re: Tracking supermarket prices with Playwright

#12
post #11

Very cool! I did something similar in Canada ( https://grocerytracker.ca/ )

Oh nice! A thorny problem in my case is that the same item is named in 3 different ways between the 3 supermarkets which makes it very hard and annoying to do a proper comparison. Did you have a similar problem?

Absolutely! It’s made it difficult to implement some of the cross-retailer comparison features I would like to add. For my charts I’ve just manually selected some products, but I’ve also been trying to get a “good enough but not perfect” string comparison algorithm working.

Re: Tracking supermarket prices with Playwright

#15
post #3

Ah, I love this. Nice work! I really wish supermarkets were mandated to post this information whenever the price of a particular SKU updated. The tools that could be built with such information would do amazing things for consumers.

[flagged]

Re: Tracking supermarket prices with Playwright

#16
post #11

Earlier quoted context omitted.

Oh nice! A thorny problem in my case is that the same item is named in 3 different ways between the 3 supermarkets which makes it very hard and annoying to do a proper comparison. Did you have a similar problem?

Absolutely! It’s made it difficult to implement some of the cross-retailer comparison features I would like to add. For my charts I’ve just manually selected some products, but I’ve also been trying to get a “good enough but not perfect” string comparison algorithm working.

Ah yes.

My approach so far has been to first extract the brand names (which are also not written the same way for some fcking reason!), update the strings, and then compare the remaining.

If they have a high similarity (e.g. >95%) then they could be automatically merged, and then anything between 75%-95% can be reviewed manually.

Re: Tracking supermarket prices with Playwright

#17
One problem that the author notes is that so much rendering is done client side via javascript.

The flip side to this is that very often you find that the data populating the site is in a very simple JSON format to facilitate easy rendering, ironically making the scraping process a lot more reliable.

Re: Tracking supermarket prices with Playwright

#18
post #3

Ah, I love this. Nice work! I really wish supermarkets were mandated to post this information whenever the price of a particular SKU updated. The tools that could be built with such information would do amazing things for consumers.

As someone who actively works on these kind of systems, it's a bit more complicated than that. The past few years we worked on migrating from some old system from the 80's designed for LAN use only, to a cloud based item catalogue system that finally allowed us the ability to easily make pricing info more available to consumers, such as through an app.

Re: Tracking supermarket prices with Playwright

#19
post #16

Earlier quoted context omitted.

Absolutely! It’s made it difficult to implement some of the cross-retailer comparison features I would like to add. For my charts I’ve just manually selected some products, but I’ve also been trying to get a “good enough but not perfect” string comparison algorithm working.

Ah yes. My approach so far has been to first extract the brand names (which are also not written the same way for some fcking reason!), update the strings, and then compare the remaining. If they have a high similarity (e.g. >95%) then they could be automatically merged, and then anything between 75%-95% can be reviewed manually.

I am not by any mean an expert but maybe using some LLMs or a sentence transformer here could help to do the job?

Re: Tracking supermarket prices with Playwright

#20
post #10
post #3

Ah, I love this. Nice work! I really wish supermarkets were mandated to post this information whenever the price of a particular SKU updated. The tools that could be built with such information would do amazing things for consumers.

Thanks! If Greece's case is anything to go by, I doubt they'd ever accept that as it may bring to light some... questionable practices. At some point I need to deduplicate the products and plot the prices across all 3 supermarkets on the same graph as I suspect it will show some interesting trends.

fyi, I posted this on /r/greece
Post reply on HN