Live data from Hacker News

Scraping Recipe Websites

benawad.com

51–60 of 204 posts

Re: Scraping Recipe Websites

#52

I highly useful tool in my household for dealing with the SEO/tracking scourge that recipe blogs have become is https://www.paprikaapp.com/ . Hoping someday to have some spare time to integrate this with https://grocy.info/ and have a pipeline for recipe -> preparation automation.

Big fan of this app, and I love it so I don't have to keep revisiting the sites. This is one of the few apps, I've purchased multiple times: for my iOS eco-system (I typically cook with my iPad), my android phone (so I can add recipes on the go), and my partner's devices so she can add recipes to our list.

We've developed a little workflow where we put all the recipes we want to try into an "incoming" category, and then move them to one of our custom categories when we make it and decide it's worth keeping. This is a reaction to becoming recipe hoarders when using a site like Pintarest for something similar.

The iOs has a really subtle, but nice feature when you're cooking with the app. It prevents the iPad from going to sleep and locking which you have messy fingers.

Re: Scraping Recipe Websites

#53

Here's the question... why is it so difficult to do this in Android? Seriously, AndroidDriver for Selenium was last updated 2013... and importing it throws an HttpClient error now. Update that client and you get a class duplication hell that is impossible to exit. All I needed was to interact with 2-3 fields on a webpage but it's been eight hours and now I hate my life.

Checkout BrowserStack -- it's dead easy -- and even if you're not using their platform, their docs are good for showing the Selenium/Driver usage.

Re: Scraping Recipe Websites

#55
post #20
post #12

Earlier quoted context omitted.

Having thought about this and my own use of cookbooks as loose inspiration rather than actually to follow in detail, I have come to the conclusion that the "fluff" is what most of the recipe-reading public want. In particular, the fluff has value even if you never cook the recipe! Which saves a lot of time and inconvenience on your part while still giving the same warm fluffy feelings. Actual "I have these things and…

I have come to the conclusion that the "fluff" is what most of the recipe-reading public want. I've often heard the prevailing reason why this happens on the web is because of SEO and 'bounce rates'. More time spent on the site improves ranking, so the actual recipe is pushed below the fold so users have to scroll down thereby adding more time on the site. Have often wondered if any SEO wonks with the inside baseball…

I have done lot of work on cocktails (major brand) and they key is the structured mark-up we are trying some tests on longer form listings ie >200 words.

I think that food is much more dispersed apart from super stars like Nigella, Jamie and the BBC so its harder for the average food blogger.

Also don't discount the target audience for these pages might not be the average developer / nh reader. Also don't discount the

Re: Scraping Recipe Websites

#56
post #5

But how will I read about "Dakota", an avid yoga enthusiast who just happens to be a mom, who enjoys making healthy and savory meals for her family while blogging? Seriously, I hope this spells an end to the Google ranking imposed nonsense that makes the simple act of searching for a recipe so insufferable.

Does it pose an actual problem? When I search for recipes, I type in the food I want + recipe and then open the top 5 or so links. I quick scan for a list of ingredients. If I don't easily spot on in a few seconds I move on. I'll do this until I have a couple different lists of ingredients for making the item. This ends up taking less than a minute or two. That just isn't a significant portion of time compared to how…

This ends up taking less than a minute or two.

For you to open 5 websites, dismiss the cookie permission request on each one, dismiss the notifications request, scroll down to find the ingredients list, dismiss the scrolling activated newsletter signup, read the ingredients list, and click the 'next page' button to see the instructions to work out what you can tweak, all in an average of 24s per site is very impressive.

Re: Scraping Recipe Websites

#58
post #37

Any recommendations for a js lib that does all the "easy" scraping (microdata, og tags, jsonld, etc)?

I thought that's what this blog post was going to be about but it's just an ad for their app. I just need the scraping functionality.

While they do end with pushing their product, I think they did a good job of outlining how they scrape the recipes. They inform the reader about json+ld, microdata, and how to scrape the sites that don't use those. They even link to a JS lib that handles the parsing for you. I think calling it "just an ad" is inaccurate.

> There are libraries like https://github.com/digitalbazaar/jsonld.js/ to parse JSON-LD + Microdata for you.

Re: Scraping Recipe Websites

#59
post #5

But how will I read about "Dakota", an avid yoga enthusiast who just happens to be a mom, who enjoys making healthy and savory meals for her family while blogging? Seriously, I hope this spells an end to the Google ranking imposed nonsense that makes the simple act of searching for a recipe so insufferable.

It's a running joke in our house. I start off wanting to make some mashed potatoes, and time and time again, I have to suffer through someone's life story--the camping trip in North Dakota when Susan's husband first discovered his love of homemade sour cream--etc. Makes me wonder if a super barebones recipe site that literally just has recipes and absolutely no fluff would be something people would gravitate towards.

> Makes me wonder if a super barebones recipe site that literally just has recipes and absolutely no fluff would be something people would gravitate towards

In German there's chefkoch.de. It's full of ads but its core is basically a big DB of user-submitted recipes without fluff. Don't you have something like that in English?

Re: Scraping Recipe Websites

#60
post #48

Really nice! I often copy and paste recipes into text files I have locally so this is a great alternative. One feature request (if I may be so bold): it would be great to offer an imperial metric convertor. This is predominantly one of the reasons I keep copies of recipes I find and use.

It's really the conversion to weight (grams, from cups/tbsp/tsp/hogsheads) that would be valuable. It's just so much easier to clean to stick a scale under the mixing bowl.
Post reply on HN