Live data from Hacker News

Scraping Recipe Websites

benawad.com

111–120 of 204 posts

Re: Scraping Recipe Websites

#111

Be careful with this, some recipes are subject to copyright law. I think you can list ingredients of a recipe with no problem, but once you get to exact measurements and prep it somehow switches over to falling under copyright law. There used to be a bunch of open sourced recipe repos/databases...but almost all of them are gone.

Can you provide links to those open source recipe resources?

Aren't there dumps of the bygone ones anywhere?

Re: Scraping Recipe Websites

#112
post #77

Earlier quoted context omitted.

It's definitely grown worse now, but I think that this originated from recipe sites that people actually used to follow, because the blogs were interesting and we got to know the writers, and what's changed is more that we're jumping to the first Google hit and we expect them just to grant us the information we wanted. There is a difference between opening up a recipe site, like a favorite blog, or the New York Times…

I think it's also about different types of recipe collections. There are cookbooks that are all recipes. This seems to be what the HN crowd is looking for when they search the internet. There are cookbooks where each recipe is accompanied by a little story. These seem to sell well, judging by the number of them that appear on bookstore shelves. And then there are cookbooks where all of the anecdotes are in the front…

There's also cookbooks that are almost more like a textbook, with technical cooking information followed by recipes (almost like textbook information followed by exercises).

Re: Scraping Recipe Websites

#114

Earlier quoted context omitted.

It's a running joke in our house. I start off wanting to make some mashed potatoes, and time and time again, I have to suffer through someone's life story--the camping trip in North Dakota when Susan's husband first discovered his love of homemade sour cream--etc. Makes me wonder if a super barebones recipe site that literally just has recipes and absolutely no fluff would be something people would gravitate towards.

> Makes me wonder if a super barebones recipe site that literally just has recipes and absolutely no fluff would be something people would gravitate towards In German there's chefkoch.de. It's full of ads but its core is basically a big DB of user-submitted recipes without fluff. Don't you have something like that in English?

I usually add "BBC" to my search term, which ① gets a straightforward page and ② ensures the measurements and oven temperatures will be metric.

The further from the UK one is, the less useful this is. The measurements should only be a problem in the US and Canada, but common ingredients can change -- e.g. the fat content of cream, or whether canned tomatoes are salted or sweetened. Also, if you're from the rest of the world, you might wonder why they've forgotten the herbs and spices :-)

Re: Scraping Recipe Websites

#115
post #77

Earlier quoted context omitted.

It's definitely grown worse now, but I think that this originated from recipe sites that people actually used to follow, because the blogs were interesting and we got to know the writers, and what's changed is more that we're jumping to the first Google hit and we expect them just to grant us the information we wanted. There is a difference between opening up a recipe site, like a favorite blog, or the New York Times…

I think it's also about different types of recipe collections. There are cookbooks that are all recipes. This seems to be what the HN crowd is looking for when they search the internet. There are cookbooks where each recipe is accompanied by a little story. These seem to sell well, judging by the number of them that appear on bookstore shelves. And then there are cookbooks where all of the anecdotes are in the front…

I wouldn't mind the "recipe at the end" format if that were the actual case.

But it's not at the end. The actual structure of a modern recipe article is

1) Header

2) Blogpost

3) Click to read button

4) More blogpost, structured in a format that looks like an informal recipe - bulleted ingredients without quantities, discussion of steps.

5) Ad.

6) More blogpost

7) More ad that looks confusingly like a kind of recipe.

8) More blogpost

9) Nav that looks like the end of the blogpost.

10) More blogpost

11) Ad

12) Recipe ingredients

13) Ad

14) Recipe directions

15) 20 pages of chumbox, some of which structured in a way that kind of looks like a recipe at a glance.

Combined with some wonky JS that doesn't show the content until a half-second after I scroll it into view.

When I've got the recipe it feels like I've found Waldo.

Re: Scraping Recipe Websites

#116
post #77
post #5

But how will I read about "Dakota", an avid yoga enthusiast who just happens to be a mom, who enjoys making healthy and savory meals for her family while blogging? Seriously, I hope this spells an end to the Google ranking imposed nonsense that makes the simple act of searching for a recipe so insufferable.

It's definitely grown worse now, but I think that this originated from recipe sites that people actually used to follow, because the blogs were interesting and we got to know the writers, and what's changed is more that we're jumping to the first Google hit and we expect them just to grant us the information we wanted. There is a difference between opening up a recipe site, like a favorite blog, or the New York Times…

Many of the recipe sites are following a cargo cult-like methodology to SEO and only have stories before their recipes because of how they perceive Google's ranking algorithm will rank their content.

Re: Scraping Recipe Websites

#119

Earlier quoted context omitted.

It's a running joke in our house. I start off wanting to make some mashed potatoes, and time and time again, I have to suffer through someone's life story--the camping trip in North Dakota when Susan's husband first discovered his love of homemade sour cream--etc. Makes me wonder if a super barebones recipe site that literally just has recipes and absolutely no fluff would be something people would gravitate towards.

“Why doesn’t everyone who is putting information out there for free not cater to exactly my needs”. What an utter load of bollocks. These are people who are creating something that they enjoy doing and giving you information you apparently need for free. I don’t understand why anyone would trash their desire to write something that is personal and/or interesting to them about it. And that’s setting aside that these u…

It's not about trashing someone for doing something they love. It's about a prescribed format for food blogs that has all but taken over anything food related on the internet. I highly doubt it's a coincidence that thousands of food bloggers adapted a "five pages about me and then the recipe" template for their writing. As others are saying, it's an SEO tactic, and it's probably annoying to the bloggers too.

There's no arguing about it making recipes more "readable" when you have to scan down page after page to get to the actual ingredient list. That's super annoying. To put it into perspective, if the recipe was at the top, and the blogger wanted to spend five chapters talking about their life afterwards, more power to them, I wouldn't care a bit.

Re: Scraping Recipe Websites

#120

Earlier quoted context omitted.

And, the real farce is that Google mistakenly sees all that scrolling up and down the page, looking for the recipe as 'engagement' and "Dwell time".

Is that mistaken? Does no one click on ads on the page while doing this?

the algo thinks you're having a good time.
Post reply on HN