Earlier quoted context omitted.
I work in finance and this thing is still indispensable for us. A LOT of financial info is only presented on a dynamically rendered HTML page that was last written in the bad old days.
Any poignant examples?
[deleted]
Beautiful Soup
91–95 of 95 posts
Re: Beautiful Soup
#92Great memories with this library, one of my all time favs. It is fast? no. But it had a fantastic mission: extracting data from malformed HTML. Might be less common now but back then (~10+ years ago) it was still rampant. Many if not most parsers would barf on any deviation from the standard, leaving you to hand-roll regex solutions and ugly corner cases. BS covered a LOT of these cases without forcing you to write t…
Same. I had to parse HTML written by eBay sellers. They were professional sellers, not webmasters.
Re: Beautiful Soup
#93Why is this kind of post allowed on HN? It's not recent nor relevant and not specific in any meaningful way (literally linked to the homepage). I occasionally see posts just linking to Wikipedia articles as well, same sort of feel as this. At the least, OP should have to offer some sort of discussion point or tidbit from the linked content.
I used to feel this way too, but then I realized that we need to welcome the newcomers (not just newcomers to HN, but also newcomers to this material, and especially young people) who haven't yet encountered these things for the first time. For us grizzled oldtimers, they may be classics or perennials, but not for everybody. So it's ok for these posts to be part of the mix.
One of my favorite things about HN is that it has a lot of both serious oldtimers and high school students. It's actually a place where one can, at least sporadically, get the technical mentorship that many of us longed for, but missed, early in our careers.
Wikipedia submissions are a special case and somewhat different: https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que....
Re: Beautiful Soup
#94Re: Beautiful Soup
#95Earlier quoted context omitted.
This sounds interesting. Any resources for a beginner? I use Selenium regularly.
Playwright for Python has really good documentation: https://playwright.dev/python/ I used it for my https://shot-scraper.datasette.io/ tool, and wrote a bit about CLI-driven scraping using that tool here: https://simonwillison.net/2022/Mar/14/scraping-web-pages-sho...
Thanks!