31–40 of 134 posts
I very simple plumbing tools like these.
This reminds me of the Soupault ( https://soupault.app/ ) philosophy for building static sites. You write it in any language you want like, pass it trough Pandoc or AsciiDoctor as preprocessor, and postprocess with Lua and CSS selectors.
Earlier quoted context omitted.
Thanks! I'll look at that.
> (Of course, for the combined feed this would be problematic.) Not so; unlike the HTML element which applies to a document, the xml:base attribute is applied to an element and its descendants. The typical pattern (as shown in the RFC 4287 example) is to put it on each entry’s . In your markup, you’ll end up with each entry having its URL in three places: http://example.com/item …
Edit: the package I'm using to generate the feeds does not support that attribute yet, so it'll have to wait a bit for my PR to hopefully be accepted: https://github.com/jpmonette/feed/pull/158
Thanks for the pointers!
Nice, but it's essentially scraping? Scraping is brittle. Some design changes and the whole thing breaks. Maintenance nightmare.
This is why APIs exist.
Nice, but it's essentially scraping? Scraping is brittle. Some design changes and the whole thing breaks. Maintenance nightmare.
Not sure for the downvote reason. Like I say scraping is brittle. I've done hundreds of scraping projects in the past. A recent one was taking 50 shopping sites looking for what products they covered. Safe to say you look at it a month later and the XPaths/CSS selectors/whatever scraping method you used has changed. This is why APIs exist.
Earlier quoted context omitted.
Not sure for the downvote reason. Like I say scraping is brittle. I've done hundreds of scraping projects in the past. A recent one was taking 50 shopping sites looking for what products they covered. Safe to say you look at it a month later and the XPaths/CSS selectors/whatever scraping method you used has changed. This is why APIs exist.
Ideally sites would just publish RSS feeds themselves, but not all of them do — let alone an API…
It also begs the question of why they don't make their content that way, perhaps it was by choice. Especially if you use this kind of tool to syndicate things.
Not quite sure of the novelty of this one given that people have scraped things for decades.
There's probably a wiki endpoint for their example, maybe, maybe not. A lot of wiki's stuff is free to download and they have extensive endpoints for acquiring data.
Two things I am using:
Twitter to RSS: https://github.com/RSS-Bridge/rss-bridge
Arbitrary RSS feeds: https://feedity.com
Nice, but it's essentially scraping? Scraping is brittle. Some design changes and the whole thing breaks. Maintenance nightmare.
Earlier quoted context omitted.
Same problem with SSL encryption. But with both RSS and SSL, if the user use a modern client, he shouldn't bother if its internally using RSS, Atom, SSL or TLS. I for one know most of my pals do know about RSS, but haven't heard the term 'Atom feed'.
“SSL” is finally dying as a term; the significant majority of what I see calls it TLS now. But the problem didn’t exist in the same way with SSL/TLS: SSL was actively killed off in favour of TLS so that regardless of what you call it, you’re actually dealing with TLS. But with feeds, RSS is still supported, and so the mindshare problem happens: people hear about RSS and so implement the inferior and problematic RSS r…
I think that's even worse. For most ordinary people that has a bunch of unintended meanings. "My Facebook has a feed, do you mean subscribing to your site on Facebook? Why do I see a bunch of code when I click the link?"
In case anyone wants to detect the selectors automatically, here's a small python library I wrote that does it for you: https://github.com/lorey/mlscraper