41–50 of 77 posts
My guess would be that some companies are doing it (I worked at a major tech company that is/was), just not publicizing this fact as crawling/scraping is such a gray legal area.
Earlier quoted context omitted.
Userscripts for the win. Consider Violentmonkey over Tampermonkey though.
> Consider Violentmonkey over Tampermonkey though. Why?
https://github.com/Tampermonkey/tampermonkey
> This repository contains the source of the Tampermonkey extension up to version 2.9. All newer versions are distributed under a proprietary license.
Earlier quoted context omitted.
You don't have max-width set on the text, so unless you have your browser window resized to very small size the paragraphs will span your whole screen.
I have the opposite problem. I typically have many browser windows open at the same time but only two screens, and many sites that I use are designed to assume that everyone has full-screen browsers.
With this setup, many sites work, but a few... a few have a top ad banner, a side banner and a footer of 'cookie acceptance'... then add in a 'subscribe to our email' and a google login prompt.... (Game Wiki's.. I game in smaller windows too -- what good is a multi tasking computer if you don't use it?)
> can you write a web scraper in your browser? The answer is: YES, you can! So why is nobody doing it? Completely agree with this sentiment. I just spent the last couple of months developing a chrome extension, but recently also did an unrleated web scraping project where I looked into all the common tools like beautiful soup, selenium, playwright, pupeteer, etc, etc. All of these tools were needlessly complicated an…
You might like Tamper monkey. You can add a button to kick it off or whatever your heart desires. Tampermoney also works around CORs issues with relative ease.
I wrote a prototype of a browser extension that scraped your bookmarks + 1 degree, and indexed everything into an in-memory search index (which gets persisted in localstorage). I took over the new tab page with a simple search UI, with instant type-ahead search. Rough aspects: a) It requires a _lot_ of browser permissions to install the extension, and I figured the audience who might be interested in their own search…
>Some walled garden sites seem completely unscrapable Any examples besides Linkedin? Tell me what sites you're trying to target and I'll have a look to see what can be done with them. It takes some pretty evil Javascript obfuscation to block me and only one site has been able to do that. I doubt that the sites you're hitting are anywhere near that evil, lol. I would appreciate it if you have a good example that I cou…
IIRC I ended up building an iframe based scraper for sites that didn't yield any content with just a fetch - and I think built a fallback mechanism so that if fetch didn't work, I'd queue it up in the iframe scraper. The problem with that is that there are various heavily used security headers that prohibit loading in an iframe. (And the reason for iframe vs just loading in a tab and injecting my extension's script is that I wanted it to be able to run "in the background" without being super distracting for the user - the tab changing favicon every second or two was pretty annoying.)
Earlier quoted context omitted.
Too much contrast makes it hard to read too. (With bonus eye strain.)
Is there a happy in-between? Maybe not. What looks perfect to one user might appear atrocious to another. What is a poor website operator to do??? I dislike black-on-white and don't understand gray-on-black which seems to be popular now due to gamma settings being cranked up to 11 or something. I try to use some color as an in-between but that may take some time to "perfect".
Since browsers allow users to configure a default font and background color then one possible "happy in-between" would be to set no background color, and set no font color, thereby allowing each user agent (i.e., browser) to display the site with that user's default background and font colors.
In that case, each viewer should get their preferred colors, all without you doing anything.