Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Scraping Recipe Websites

benawad.com

Post reply on HN
Print view View original

131–140 of 204 posts

Re: Scraping Recipe Websites

#131

Post by logfromblammo »

crankylinuxuser wrote: post #79
tylerrobinson wrote: post #49

Earlier quoted context omitted.

Is there evidence that the "annoying recipe sites" in question include algorithmic stories, or are you speaking hypothetically?

I don't have any direct proof that recipe sites are doing this. I only look at just how many "stories" there are, and how many recipes. There just looks like too much writing. And it too is also very formulaic. And it's also 'if I was to do a recipe site, I'd use a text generator'

Sounds cheaper than paying a freelancer $0.025 per word, too.

I suppose if you were just starting a blogspam recipe site, you could initially pay freelancers for the first articles, and use them as your training corpus. But since this sort of templated recipe site is already evil, just scrape all the other recipe sites and use their articles as your corpus.

"When we vacationed in { madlib( international_city ) }, we ate a { r.Name }, and it was delicious. When we returned home, we tried to recreate the recipe, but it was never quite right. After months of trying, it's finally perfect."

"As a child, I remember my grandmother making { r.Name } and eating it with all my cousins. It was her secret recipe. She never told anyone. When she died, we were all very sad because we thought the recipe was gone forever. But I found this in her old { madlib( noun ) }, and now I'm the most popular cousin at the reunion."

Repeat ad nauseam.

Re: Scraping Recipe Websites

#132

Post by phito »

memset wrote: post #129

Interesting! I wrote https://plainoldrecipe.com (open source!) to solve this, an inadvertently discovered many of the metadata tags described here. The irony is that the content is required for SEO purposes, but once you’ve landed on the page you don’t want to see it. I wonder if there would be a way to write SEO that only the google bot sees and hide it from humans...

Your header says "plan old recipe"

Re: Scraping Recipe Websites

#133

Post by CrazyStat »

partiallypro wrote: post #41

Be careful with this, some recipes are subject to copyright law. I think you can list ingredients of a recipe with no problem, but once you get to exact measurements and prep it somehow switches over to falling under copyright law. There used to be a bunch of open sourced recipe repos/databases...but almost all of them are gone.

> I think you can list ingredients of a recipe with no problem, but once you get to exact measurements and prep it somehow switches over to falling under copyright law.

In the US, recipes are not protected by copyright, including precise measurements and instructions for how to combine them.

If you write sufficiently creative instructions those may be protected by copyright, but I can get around that by simply not replicating those instructions. I can give exactly the same list of ingredients and a less-creative set of instructions for the same dish without infringing your copyright.

Copyright.gov has a circular which explains this with a couple examples [1].

[1] https://www.copyright.gov/circs/circ33.pdf

Re: Scraping Recipe Websites

#134

Post by csa »

scrollaway wrote: post #43
thawaway1837 wrote: post #40

Earlier quoted context omitted.

“Why doesn’t everyone who is putting information out there for free not cater to exactly my needs”. What an utter load of bollocks. These are people who are creating something that they enjoy doing and giving you information you apparently need for free. I don’t understand why anyone would trash their desire to write something that is personal and/or interesting to them about it. And that’s setting aside that these u…

It's a sweet world view, but the truth is that the majority of these cooking sites are filling in content for SEO and ad purposes, and the "stories" are fictions written by Tom, a 22 year old freelancer who isn't a yoga enthusiast but is just trying to make ends meet at 3¢/word.

This is so incredibly accurate.

Re: Scraping Recipe Websites

#135

Post by peterwwillis »

So far the best way I've found to search for recipes is to search in a foreign language. Translate what you're looking for, then search and translate back to English. There are still recipe blogs, but 5 instead of 5,000, and usually an authentic dish, not what Michelle The Stir Fry Queen From Michigan thinks constitutes a "Moroccan" dish because it has cinnamon and tomatoes.

Would love to see someone put together a search engine that excludes recipe blogs and penalizes SEO.

Re: Scraping Recipe Websites

#136

Post by hawski »

memset wrote: post #129

Interesting! I wrote https://plainoldrecipe.com (open source!) to solve this, an inadvertently discovered many of the metadata tags described here. The irony is that the content is required for SEO purposes, but once you’ve landed on the page you don’t want to see it. I wonder if there would be a way to write SEO that only the google bot sees and hide it from humans...

There is a way to present different things for the google bot and humans, but it can and should result in Google ban. You're probably aware and I'm a bit too verbatim.

Re: Scraping Recipe Websites

#137

Post by csa »

galfarragem wrote: post #85
onion2k wrote: post #62

Earlier quoted context omitted.

mashed potatoes Steam some potatoes. (Don't boil them. No one wants watery mash.) Add far more butter than anyone would think is reasonable. (Like, 1/2lb of butter to 1lb of potatoes. Maybe more.) Add salt and pepper. (No, more than that. You've under-seasoned them.) Mash them until they're the consistency you want. (For really smooth mash use an electric hand mixer instead of a masher.) To make them better still, pu…

I would buy a cooking book with recipes explained like this one. With real world tricks and explaining "why" you do it in that way. A raw list of ingredients is, in my opinion, almost useless.

America’s Test Kitchen (TV) / Cooks Illustrated (books) will scratch your itch.

My foodie friend and I use a lot of their recipes.

The cool thing is that sometimes you will disagree with their preferences (totally normal), so you can use one of their variants that has the attributes that you want. Or similarly, you can see how changing certain ingredients changes the outcome and personalize your version of the recipe accordingly. It definitely saves some experimental batches.

Re: Scraping Recipe Websites

#140

Post by otakucode »

SamBam wrote: post #77
kmbfjr wrote: post #5

But how will I read about "Dakota", an avid yoga enthusiast who just happens to be a mom, who enjoys making healthy and savory meals for her family while blogging? Seriously, I hope this spells an end to the Google ranking imposed nonsense that makes the simple act of searching for a recipe so insufferable.

It's definitely grown worse now, but I think that this originated from recipe sites that people actually used to follow, because the blogs were interesting and we got to know the writers, and what's changed is more that we're jumping to the first Google hit and we expect them just to grant us the information we wanted. There is a difference between opening up a recipe site, like a favorite blog, or the New York Times…

There also might be an issue that recipes themselves can not be copyrighted. Article content and writing alongside the recipe can be copyrighted, but the recipe itself is not eligible for copyright protection at all. So if you're trying to make money off of a site, if it only had recipes, there is no protection whatsoever from someone setting up an identical copy and monetizing it themselves.
Post reply on HN