Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google begins requiring JavaScript for Google Search

techcrunch.com

Post reply on HN
Print view View original

51–60 of 158 posts

Re: Google begins requiring JavaScript for Google Search

#51

Post by pluc »

orf wrote: post #23
pluc wrote: post #11

Earlier quoted context omitted.

There's absolutely no need for JavaScript on a page that has a text input and two buttons and that has worked without JS for three decades. Given Google's reputation for privacy and the constant attempts at selling their users out, it's fair to assume that the reason they're requiring JavaScript is not noble.

> There's absolutely no need for JavaScript on a page that has a text input and two buttons The whole web is evil then. Hacker news has JavaScript for simple upvote buttons, is it also evil?

I don't disagree with you. I use NoScript which lets me selectively enable every JS source a site has ever since marketers and advertisers have weaponized it, and you'd be surprised what you find and what works with minimal JS. If anything, it's very educational.

Re: Google begins requiring JavaScript for Google Search

#52

Post by marginalia_nu »

supriyo-biswas wrote: post #50
marginalia_nu wrote: post #25

To be fair, if my search engine is anything to go on, about 0.5-1% of the requests I get are from human sources. The rest are from bots, and not like people who haven't found I have an API, but bots that are attempting to poison Google or Bing's query suggestions (even though I'm not backed by either). From what I've heard from other people running search engines, it looks the same everywhere. I don't know what Googl…

I run a semi-popular website hosting user-generated content, although it's not a search engine; the attacks on it have surprised me, and I've eventually had to put in the same kinds of restrictions on it. I was initially very hesitant to restrict any kind of traffic, relying on ratelimiting IPs on critical endpoints that needed low friction, and captchas on the higher friction with higher intents, such as signup and…

> I'm appalled by some of the discussions here. Was I "enshittifying" my website out of unbridled "greed"? I don't think so. But every time I come here, I find these accusations, which makes me think that as a website with technical users, we can definitely do better.

It's if nothing else very evident most people fundamentally don't understand what an adversarial shit show running a public web service is.

Re: Google begins requiring JavaScript for Google Search

#53

Post by supriyo-biswas »

palmfacehn wrote: post #42
marginalia_nu wrote: post #25

To be fair, if my search engine is anything to go on, about 0.5-1% of the requests I get are from human sources. The rest are from bots, and not like people who haven't found I have an API, but bots that are attempting to poison Google or Bing's query suggestions (even though I'm not backed by either). From what I've heard from other people running search engines, it looks the same everywhere. I don't know what Googl…

My impression is that there's less effort for them to go directly to headless browsers. There are several foot guns in using a raw HTML parsing lib and dispatching HTTP requests. People don't care about resource usage, spammers even less and many of them lack the skills.

A major player in this space is apparently looking for people experienced in scraping without using browser automation. My guess is that not running a browser results in using far fewer resources, thus reducing their costs heavily.

Running a headless browser also means that any differences in the headless environment vs. a "headed" one can be discovered, as well as any of your Javascript executing within the page, which significantly makes it difficult to scale your operation.

Re: Google begins requiring JavaScript for Google Search

#54

Post by [deleted] »

yla92 wrote: post #47
nilslindemann wrote: post #24

Just tested (ignoring AI search engines, non-english, non-free): Search engines which require JavaScript: Google, Bing, Ecosia, Yandex, Qwant, Gibiru, Presearch, Seekr, Swisscows, Yep, Openverse, Dogpile, Waldo Search engines which do not require JavaScript: DuckDuckGo, Yahoo Search, Brave Search, Startpage, AOL Search, giveWater, Mojeek

Kagi.com works without JS

[deleted]

Re: Google begins requiring JavaScript for Google Search

#55

Post by marginalia_nu »

supriyo-biswas wrote: post #53
palmfacehn wrote: post #42

Earlier quoted context omitted.

My impression is that there's less effort for them to go directly to headless browsers. There are several foot guns in using a raw HTML parsing lib and dispatching HTTP requests. People don't care about resource usage, spammers even less and many of them lack the skills.

A major player in this space is apparently looking for people experienced in scraping without using browser automation. My guess is that not running a browser results in using far fewer resources, thus reducing their costs heavily. Running a headless browser also means that any differences in the headless environment vs. a "headed" one can be discovered, as well as any of your Javascript executing within the page, wh…

My experience is that headless browsers use about 100x more RAM, and at least 10x more bandwidth and 10x more processing power, and page loads take about 10x as long time to finish (vs curl). Though these numbers may be a bit low, there are instances you need to add another zero to one or more of them.

There's also considerably more jank with headless browsers, since you typically want to re-use instances to avoid incurring the cost of spawning a new browser for each retrieval.

Re: Google begins requiring JavaScript for Google Search

#56

Post by dageshi »

supriyo-biswas wrote: post #50
marginalia_nu wrote: post #25

To be fair, if my search engine is anything to go on, about 0.5-1% of the requests I get are from human sources. The rest are from bots, and not like people who haven't found I have an API, but bots that are attempting to poison Google or Bing's query suggestions (even though I'm not backed by either). From what I've heard from other people running search engines, it looks the same everywhere. I don't know what Googl…

I run a semi-popular website hosting user-generated content, although it's not a search engine; the attacks on it have surprised me, and I've eventually had to put in the same kinds of restrictions on it. I was initially very hesitant to restrict any kind of traffic, relying on ratelimiting IPs on critical endpoints that needed low friction, and captchas on the higher friction with higher intents, such as signup and…

There's a certain relatively tiny audience that has congregated on HN for whom hating ads is a kind of religion and google is the great satan.

Threads like this are where they come to affirm their beliefs with fellow adherents.

Comments like yours, those that imply there might be some valid reason for a move like this (even with degrees of separation) are simply heretical. I think these people cling to an internet circa 2002ish and the solution to all problems with the modern internet is to make the internet go back to 2002.

Re: Google begins requiring JavaScript for Google Search

#57

Post by shiomiru »

marginalia_nu wrote: post #25

To be fair, if my search engine is anything to go on, about 0.5-1% of the requests I get are from human sources. The rest are from bots, and not like people who haven't found I have an API, but bots that are attempting to poison Google or Bing's query suggestions (even though I'm not backed by either). From what I've heard from other people running search engines, it looks the same everywhere. I don't know what Googl…

If it's such a common issue, I would've thought Google already ignored searches from clients that do not enable JavaScript when computing results?

Besides, you already got auto-blocked when using it in a slightly unusual way. Google hasn't worked on Tor since forever, and recently I also got blocked a few times just for using it through my text browser that uses libcurl for its network stack. So I imagine a botnet using curl wouldn't last very long either.

My guess is it had more to do with squeezing out more profit from that supposed 0.1% of users.

Re: Google begins requiring JavaScript for Google Search

#60

Post by supriyo-biswas »

shiomiru wrote: post #57
marginalia_nu wrote: post #25

To be fair, if my search engine is anything to go on, about 0.5-1% of the requests I get are from human sources. The rest are from bots, and not like people who haven't found I have an API, but bots that are attempting to poison Google or Bing's query suggestions (even though I'm not backed by either). From what I've heard from other people running search engines, it looks the same everywhere. I don't know what Googl…

If it's such a common issue, I would've thought Google already ignored searches from clients that do not enable JavaScript when computing results? Besides, you already got auto-blocked when using it in a slightly unusual way. Google hasn't worked on Tor since forever, and recently I also got blocked a few times just for using it through my text browser that uses libcurl for its network stack. So I imagine a botnet us…

Given that curl-impersonate[1] exists and that a major player in this space is also looking for experience with this library, I'm pretty sure forcing the execution of JS using DOM stuff would be a much more effective deterrent to prevent scraping.

[1] https://github.com/lwthiker/curl-impersonate

Post reply on HN