WWW::Mechanize fam where you at?
Beautiful Soup
41–50 of 95 posts
Re: Beautiful Soup
#42Why is this kind of post allowed on HN? It's not recent nor relevant and not specific in any meaningful way (literally linked to the homepage). I occasionally see posts just linking to Wikipedia articles as well, same sort of feel as this. At the least, OP should have to offer some sort of discussion point or tidbit from the linked content.
Re: Beautiful Soup
#43whats new about it? why is this post here? Beautiful Soup has been around for a long long time.
Re: Beautiful Soup
#44Why is this kind of post allowed on HN? It's not recent nor relevant and not specific in any meaningful way (literally linked to the homepage). I occasionally see posts just linking to Wikipedia articles as well, same sort of feel as this. At the least, OP should have to offer some sort of discussion point or tidbit from the linked content.
Because it's nice to exist outside the news cycle of what's "current" every once in a while :)
Re: Beautiful Soup
#45Why is this kind of post allowed on HN? It's not recent nor relevant and not specific in any meaningful way (literally linked to the homepage). I occasionally see posts just linking to Wikipedia articles as well, same sort of feel as this. At the least, OP should have to offer some sort of discussion point or tidbit from the linked content.
Because it's nice to exist outside the news cycle of what's "current" every once in a while :)
You have other social media for that. Low effort posts do not invite meaningful discussion and add moderation load.
Re: Beautiful Soup
#46Is Beautiful Soup still the best way to scrape the web with python? IIRC, Beautiful Soup doesn't handle javascript, so at least for JS you're forced to use something else. I'm also looking forward to seeing how people scrape the web once Web Assembly becomes prevalent.
I’ve found Playwright to be a really great tool for scraping
This sounds interesting. Any resources for a beginner? I use Selenium regularly.
Re: Beautiful Soup
#47Why is this kind of post allowed on HN? It's not recent nor relevant and not specific in any meaningful way (literally linked to the homepage). I occasionally see posts just linking to Wikipedia articles as well, same sort of feel as this. At the least, OP should have to offer some sort of discussion point or tidbit from the linked content.
Personally these and the Wikipedia posts are my favorite posts on here. News is cool, but there's a lot of cool things that don't change very often, and I love seeing those things too.
Also, in response to your "low effort posts do not invite meaningful discussion" from a different comment, I don't see how this is any lower effort than every other link only post (i.e. the vast majority)? And there's over 40 comments on this thread now talking about other scrapers, projects you can do with scrapers, better docs, tangential use cases and how to handle them, etc. Seems like a lot of people have a variety of things to say about this, I don't see how that's not "meaningful discussion".
EDIT: I also disagree with requiring a couple of sentences from the submitter. If they have something to say they can say it, otherwise it's fine if they don't try to influence the discussion - it's more interesting to see where the random commenters take something, then trying to chart a course.
Re: Beautiful Soup
#48Great memories with this library, one of my all time favs. It is fast? no. But it had a fantastic mission: extracting data from malformed HTML. Might be less common now but back then (~10+ years ago) it was still rampant. Many if not most parsers would barf on any deviation from the standard, leaving you to hand-roll regex solutions and ugly corner cases. BS covered a LOT of these cases without forcing you to write t…
I work in finance and this thing is still indispensable for us. A LOT of financial info is only presented on a dynamically rendered HTML page that was last written in the bad old days.
Absolutely. Until you have had to deal with this kind of problem first-hand, you have no idea how much of a relief it is that it exists.
Sometimes, finding and using the right library can completely turn around a f'd project.
Re: Beautiful Soup
#49I found lxml.html a lot easier to work with than bs4, in case that helps anyone else. https://lxml.de/lxmlhtml.html
On the off chance you were not aware, bs4 also supports[0] getting parse events from html5lib[1] which (as its name implies) is far more likely to parse the text the same way the browser would 0: https://www.crummy.com/software/BeautifulSoup/bs4/doc/index.... 1: https://pypi.org/project/html5lib/
BeautifulSoup is an API for multiple parsers
https://beautiful-soup-4.readthedocs.io/en/latest/#installin... :
BeautifulSoup(markup, "html.parser")
BeautifulSoup(markup, "lxml")
BeautifulSoup(markup, "lxml-xml")
BeautifulSoup(markup, "xml")
BeautifulSoup(markup, "html5lib")
Re: Beautiful Soup
#50Earlier quoted context omitted.
Because it's nice to exist outside the news cycle of what's "current" every once in a while :)
You have other social media for that. Low effort posts do not invite meaningful discussion and add moderation load.
However, I don't recall hearing the moderators complain about it.
I'm guessing most of the difficult moderation would be on the newsy more-controversial posts anyways.