Live data from Hacker News

Scraping Recipe Websites

benawad.com

31–40 of 204 posts

Re: Scraping Recipe Websites

#31
Here's the question... why is it so difficult to do this in Android?

Seriously, AndroidDriver for Selenium was last updated 2013... and importing it throws an HttpClient error now. Update that client and you get a class duplication hell that is impossible to exit.

All I needed was to interact with 2-3 fields on a webpage but it's been eight hours and now I hate my life.

Re: Scraping Recipe Websites

#33
post #20
post #12

Earlier quoted context omitted.

Having thought about this and my own use of cookbooks as loose inspiration rather than actually to follow in detail, I have come to the conclusion that the "fluff" is what most of the recipe-reading public want. In particular, the fluff has value even if you never cook the recipe! Which saves a lot of time and inconvenience on your part while still giving the same warm fluffy feelings. Actual "I have these things and…

I have come to the conclusion that the "fluff" is what most of the recipe-reading public want. I've often heard the prevailing reason why this happens on the web is because of SEO and 'bounce rates'. More time spent on the site improves ranking, so the actual recipe is pushed below the fold so users have to scroll down thereby adding more time on the site. Have often wondered if any SEO wonks with the inside baseball…

I've also read that it's to do with copyright - the recipes themselves can't be copyrighted, but the text around them can. Scraper republishes your ingredients list: not a lot you can do. Scraper republishes your fluffy anecdote and pictures: BLAMMO!

Re: Scraping Recipe Websites

#35
post #5

But how will I read about "Dakota", an avid yoga enthusiast who just happens to be a mom, who enjoys making healthy and savory meals for her family while blogging? Seriously, I hope this spells an end to the Google ranking imposed nonsense that makes the simple act of searching for a recipe so insufferable.

My understanding of why there's a life story copy at the top of recipes is that a recipe is not copyrightable, but a story is.

And, an AI can generate a story one time relatively easy.

https://lizerbramlaw.com/2015/04/07/copyright-protect-recipe...

Re: Scraping Recipe Websites

#36
post #7

Are there any legal issues with scraping recipe sites in a commercial app like that? I'm assuming ingredients and directions are "facts" so can't be copyrighted, but what about the pictures?

While a recipe isn't protected by copyright in the US (and many other countries, including the UK), the wording of the recipe could well be an original literary work, the layout of the page could attract a copyright (as it does in cookbooks) and you're right that the images would be protected.

All that said, if the import is being used for personal use only and not being edited, then it's little different to printing it out and putting it in a binder. I don't know much about US fair-use laws, but in the UK it would seem that reproducing a recipe in an app for your own use would qualify as fair dealing thanks to being personal study.

That only applies if the imports are specific to the person importing them, of course. If they're shared or published, then it's a different story. Also, if you're importing more than one recipe, so it's a significant amount of the published work, then that'd be an issue too. You can't import a whole cookbook and claim it's personal study, but one recipe out of dozens is probably fine.

Re: Scraping Recipe Websites

#37

Any recommendations for a js lib that does all the "easy" scraping (microdata, og tags, jsonld, etc)?

I thought that's what this blog post was going to be about but it's just an ad for their app. I just need the scraping functionality.

Re: Scraping Recipe Websites

#38
post #7

Are there any legal issues with scraping recipe sites in a commercial app like that? I'm assuming ingredients and directions are "facts" so can't be copyrighted, but what about the pictures?

I assume this would go into DMCA territory, since your hosting user submitted content. As long as you don't host the scraped recipes and images publicly, I imagine it would live in a legal grey area if you had a notice that you must be allowed to use the image you upload in your jurisdiction.

It'd be similar to trying to go after google because someone uploaded a copyrighted work to their google drive. I know they have to deal with it if you share the link, but they don't go out of their way to remove content you uploaded to your google drive and never shared.

Re: Scraping Recipe Websites

#39

Here's the question... why is it so difficult to do this in Android? Seriously, AndroidDriver for Selenium was last updated 2013... and importing it throws an HttpClient error now. Update that client and you get a class duplication hell that is impossible to exit. All I needed was to interact with 2-3 fields on a webpage but it's been eight hours and now I hate my life.

I believe that the Webdriver/Chromedriver approach is the current recommended way of doing this.

Re: Scraping Recipe Websites

#40
post #5

But how will I read about "Dakota", an avid yoga enthusiast who just happens to be a mom, who enjoys making healthy and savory meals for her family while blogging? Seriously, I hope this spells an end to the Google ranking imposed nonsense that makes the simple act of searching for a recipe so insufferable.

It's a running joke in our house. I start off wanting to make some mashed potatoes, and time and time again, I have to suffer through someone's life story--the camping trip in North Dakota when Susan's husband first discovered his love of homemade sour cream--etc. Makes me wonder if a super barebones recipe site that literally just has recipes and absolutely no fluff would be something people would gravitate towards.

“Why doesn’t everyone who is putting information out there for free not cater to exactly my needs”.

What an utter load of bollocks. These are people who are creating something that they enjoy doing and giving you information you apparently need for free. I don’t understand why anyone would trash their desire to write something that is personal and/or interesting to them about it.

And that’s setting aside that these usually make the recipes far more readable and interesting to the vast majority of people.

Post reply on HN