Be careful with this, some recipes are subject to copyright law. I think you can list ingredients of a recipe with no problem, but once you get to exact measurements and prep it somehow switches over to falling under copyright law. There used to be a bunch of open sourced recipe repos/databases...but almost all of them are gone.
Scraping Recipe Websites
41–50 of 204 posts
Re: Scraping Recipe Websites
#42Any recommendations for a js lib that does all the "easy" scraping (microdata, og tags, jsonld, etc)?
In Python I use the "extruct" package from the scrapy people. It's not very good with syntax errors in the markup.
Re: Scraping Recipe Websites
#43Earlier quoted context omitted.
It's a running joke in our house. I start off wanting to make some mashed potatoes, and time and time again, I have to suffer through someone's life story--the camping trip in North Dakota when Susan's husband first discovered his love of homemade sour cream--etc. Makes me wonder if a super barebones recipe site that literally just has recipes and absolutely no fluff would be something people would gravitate towards.
“Why doesn’t everyone who is putting information out there for free not cater to exactly my needs”. What an utter load of bollocks. These are people who are creating something that they enjoy doing and giving you information you apparently need for free. I don’t understand why anyone would trash their desire to write something that is personal and/or interesting to them about it. And that’s setting aside that these u…
It's a sweet world view, but the truth is that the majority of these cooking sites are filling in content for SEO and ad purposes, and the "stories" are fictions written by Tom, a 22 year old freelancer who isn't a yoga enthusiast but is just trying to make ends meet at 3¢/word.
Re: Scraping Recipe Websites
#44Earlier quoted context omitted.
I have come to the conclusion that the "fluff" is what most of the recipe-reading public want. I've often heard the prevailing reason why this happens on the web is because of SEO and 'bounce rates'. More time spent on the site improves ranking, so the actual recipe is pushed below the fold so users have to scroll down thereby adding more time on the site. Have often wondered if any SEO wonks with the inside baseball…
I can't validate this but have been told this by multiple authors and SEO professionals. So it's anecdotal. One blogger apologized to me and told me she was embarrassed doing it but it's an industry practice. She explained it's because other recipe sites use automated scraping tools and republish their recipes in an effort to outrank original authors. The personal fluff helps slow them down. Also, a lot of recipes ar…
Thanks. This makes sense. I was wondering what these fluff pieces have to do with SEO, because surely the search engines aren’t monitoring all users and how long they spend on each page in order to rank the usefulness of the content in their search results.
Bounce rates and how long a visitor stays on a page matter for those who own the sites and/or do some sort of marketing on them (ads, their own products, their services, etc.).
Re: Scraping Recipe Websites
#45But how will I read about "Dakota", an avid yoga enthusiast who just happens to be a mom, who enjoys making healthy and savory meals for her family while blogging? Seriously, I hope this spells an end to the Google ranking imposed nonsense that makes the simple act of searching for a recipe so insufferable.
Now we need an bot that parses the comments and applies AI to do . . . something with the people who say the substituted half the ingredients for what they had and left the others out to reply and tell them what recipe they actually made and that they can stuff their 1 star review.
Re: Scraping Recipe Websites
#46But how will I read about "Dakota", an avid yoga enthusiast who just happens to be a mom, who enjoys making healthy and savory meals for her family while blogging? Seriously, I hope this spells an end to the Google ranking imposed nonsense that makes the simple act of searching for a recipe so insufferable.
Does it pose an actual problem?
When I search for recipes, I type in the food I want + recipe and then open the top 5 or so links. I quick scan for a list of ingredients. If I don't easily spot on in a few seconds I move on. I'll do this until I have a couple different lists of ingredients for making the item. This ends up taking less than a minute or two. That just isn't a significant portion of time compared to how long I'll spend comparing different recipes to find a common theme to follow.
Maybe it is because I never follow a single recipe but instead combine the common themes from a couple that the whole life story before the recipe shtick isn't something that bothers me.
Re: Scraping Recipe Websites
#47Earlier quoted context omitted.
It's a running joke in our house. I start off wanting to make some mashed potatoes, and time and time again, I have to suffer through someone's life story--the camping trip in North Dakota when Susan's husband first discovered his love of homemade sour cream--etc. Makes me wonder if a super barebones recipe site that literally just has recipes and absolutely no fluff would be something people would gravitate towards.
“Why doesn’t everyone who is putting information out there for free not cater to exactly my needs”. What an utter load of bollocks. These are people who are creating something that they enjoy doing and giving you information you apparently need for free. I don’t understand why anyone would trash their desire to write something that is personal and/or interesting to them about it. And that’s setting aside that these u…
They're being rightly trashed because it's a SEO tactic and nothing more. Between the preamble ranking them higher and Adsense requiring "substance" in order to monetise the page, there is a systematic issue that's leading to what is essentially useless information to what I would say is the majority of people.
They're very welcome to write their life story, but I bet if google changed their algorithm slightly you'd see it disappear - and I'd say that would be a good thing.
Re: Scraping Recipe Websites
#48Really nice! I often copy and paste recipes into text files I have locally so this is a great alternative.
One feature request (if I may be so bold): it would be great to offer an imperialmetric convertor. This is predominantly one of the reasons I keep copies of recipes I find and use.
Re: Scraping Recipe Websites
#49But how will I read about "Dakota", an avid yoga enthusiast who just happens to be a mom, who enjoys making healthy and savory meals for her family while blogging? Seriously, I hope this spells an end to the Google ranking imposed nonsense that makes the simple act of searching for a recipe so insufferable.
My understanding of why there's a life story copy at the top of recipes is that a recipe is not copyrightable, but a story is. And, an AI can generate a story one time relatively easy. https://lizerbramlaw.com/2015/04/07/copyright-protect-recipe...
Is there evidence that the "annoying recipe sites" in question include algorithmic stories, or are you speaking hypothetically?
Re: Scraping Recipe Websites
#50Is this post illegal as it contains information on how to commit crimes such as copyright infringement?
Even if the action was a crime, discussion about an illegal action, including instructions on how to do it, are not illegal. It might be illegal in very exact situations where you are giving information directly to a person you believe will use that information to commit a crime, but sharing the information in general as a topic of discussion is protected (at least in the US, laws do differ in other countries but I find the US law as the best one to go by).