Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: An API for scraping recipe web pages

choppingboard.recipes

Post reply on HN
Print view View original

1–10 of 113 posts

Re: Show HN: An API for scraping recipe web pages

#3

Post by Freak_NL »

It looks good and seems to work with some arbitrarily picked recipes on the usual large recipe websites, although more obscure links cause some bugs¹. Is the source code on GitHub? Are you handling specific websites such as allrecipes.com with bespoke code?

Technically the project is interesting, but if you want to offer a commercial API you might run into copyright and fair use issues (as with any scraping tool). Not so much a problem for personal use, but expect angry letters from the major recipe websites for violating their terms of use (i.e., this is a threat to their business model).

1: Try this link: https://www.thespruce.com/peking-duck-recipe-694920 . You'll see a lot of garbage HTML and XML entities in the JSON that can be filtered and replaced fairly easily.

Re: Show HN: An API for scraping recipe web pages

#4

Post by oneeyedpigeon »

This really seems to be an exercise in 'structuring recipe data' rather than the ins-and-outs of scraping. Seems like a much-needed task; is there anything approaching a 'standard' for recipe data already?

    "ingredients": [
        "600g pineapple, peeled, chopped"
    ]
This seems like a prime candidate for improvement; something like the following would seem to be more useful:

    "ingredients": [{
        "ingredient": "pineapple",
        "quantity": {
            "value": 600
            "unit": "g",
        },
        "preparation": [ "peeled", "chopped" ]
    }]
Using this kind of data, I can imagine automated ordering, management of several planned recipes in conjunction, personalised timing estimations, etc. I think the whole field of cooking is ripe for some quality api-based disruption!

Re: Show HN: An API for scraping recipe web pages

#5

Post by oneeyedpigeon »

Freak_NL wrote: post #2

"prepTime": { "text": "15 minutes", "iso": "PT15M", "minutes": 15 } Why are you triplicating the time periods? I would limit that to just the ISO 8601 value (PT15M) and let the UI choose the proper rendering of the value.

Agreed; also more useful for internationalisation to keep textual descriptions out of the data, wherever possible.

Re: Show HN: An API for scraping recipe web pages

#6

Post by Freak_NL »

oneeyedpigeon wrote: post #4

This really seems to be an exercise in 'structuring recipe data' rather than the ins-and-outs of scraping. Seems like a much-needed task; is there anything approaching a 'standard' for recipe data already? "ingredients": [ "600g pineapple, peeled, chopped" ] This seems like a prime candidate for improvement; something like the following would seem to be more useful: "ingredients": [{ "ingredient": "pineapple", "quant…

Whenever I read a recipe online I get this nagging feeling that there should be some widely used open standard for describing recipes that can do all sorts of awesome stuff (like a web crawler such as DuckDuckGo being able to answer "I have a pineapple, cream, and the usual pantry basics, what can I cook?"), but this seems to go directly against the business models of the big recipe sites, so it will not likely see much uptake.

Re: Show HN: An API for scraping recipe web pages

#7

Post by simooooo »

oneeyedpigeon wrote: post #4

This really seems to be an exercise in 'structuring recipe data' rather than the ins-and-outs of scraping. Seems like a much-needed task; is there anything approaching a 'standard' for recipe data already? "ingredients": [ "600g pineapple, peeled, chopped" ] This seems like a prime candidate for improvement; something like the following would seem to be more useful: "ingredients": [{ "ingredient": "pineapple", "quant…

MyFitnessPal has the same issue where the quantity values mixed between different units, but also between volume(cup) vs weight(grams). So its a pain in the neck to know how heavy a cup of pasta is.

I can see the same issue arising with this tool.

Re: Show HN: An API for scraping recipe web pages

#8

Post by ygra »

oneeyedpigeon wrote: post #4

This really seems to be an exercise in 'structuring recipe data' rather than the ins-and-outs of scraping. Seems like a much-needed task; is there anything approaching a 'standard' for recipe data already? "ingredients": [ "600g pineapple, peeled, chopped" ] This seems like a prime candidate for improvement; something like the following would seem to be more useful: "ingredients": [{ "ingredient": "pineapple", "quant…

What I'm missing the most in this context is for how many people those ingredients actually are. That's necessary if you want to scale the recipe to fewer or more people.

Re: Show HN: An API for scraping recipe web pages

#9

Post by cbcoutinho »

simooooo wrote: post #7
oneeyedpigeon wrote: post #4

This really seems to be an exercise in 'structuring recipe data' rather than the ins-and-outs of scraping. Seems like a much-needed task; is there anything approaching a 'standard' for recipe data already? "ingredients": [ "600g pineapple, peeled, chopped" ] This seems like a prime candidate for improvement; something like the following would seem to be more useful: "ingredients": [{ "ingredient": "pineapple", "quant…

MyFitnessPal has the same issue where the quantity values mixed between different units, but also between volume(cup) vs weight(grams). So its a pain in the neck to know how heavy a cup of pasta is. I can see the same issue arising with this tool.

That's the beauty of 'user-generated content'. You can scan whatever you want, and if it's not in their system you can just put the details in yourself. That includes choosing what a portion is
Post reply on HN