Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Does Google execute JavaScript?

stephanboyer.com

Post reply on HN
Print view View original

91–100 of 114 posts

Re: Does Google execute JavaScript?

#91

Post by valarauca1 »

dom0 wrote: post #46
mootothemax wrote: post #28

I'm convinced that Google has several Googlebots that are run depending on how popular a site is. That is, new and low traffic sites are crawled by less intelligent bots, and as the site gets more visitors or better rankings, more complicated and resource intensive bots are deployed. How this might work with the most popular sites out there, the Amazons and Wikipedias of this world - I'm not so sure about that. If I…

Or perhaps there are also heuristics in place to determine which strategy to follow, ie. heuristics to see whether executing JS would be worth it, would yield additional content. So, say, when crawling documentation, where JS doesn't give any of that (eg. Sphinx' JS search), it could decide - nah, not doing JS, not worth it. I'd expect that there are also other heuristics and different strategies for crawling to bett…

    heuristics to see whether 
    executing JS would be worth 
    it, would yield additional 
    content.
You are literally describing vanilla page rank. If a large number of links are found to a page, but that page doesn't contain the contents the link rate suggest it should contain... either link rate has failed, or JavaScript should be executed.

Re: Does Google execute JavaScript?

#92

Post by ComodoHacker »

nrjdhsbsid wrote: post #17
Gigablah wrote: post #13

Earlier quoted context omitted.

And here I was thinking that it was Apple and the iPhone that effectively killed Flash. Silly me.

They disliked flash for different reasons. Google for search and apple because flash games were too powerful, jobs wanted those games turned into iPhone apps. The offical reasons were speed and security, which was obviously a farce since all the major browser vendors wrote their own PDF viewers which is both slow and known for tons of security issues as well. They could have just as easily wrote their own fast and se…

>The offical reasons were speed and security, which was obviously a farce

Security wasn't a farce at all. "Flash is a big hole" (which is true) translates to "Chrome is a big hole".

Re: Does Google execute JavaScript?

#93

Post by dispose13432 »

jupiter90000 wrote: post #36
teddyh wrote: post #16

Earlier quoted context omitted.

If that was true, you would think they would have responded better to the threat of Facebook than with Google+ and their distasteful pushing of it on all their platforms. What do I mean with “the threat of Facebook”? In the old days, before today’s large “social media” sites, people made their own web pages on places like GeoCities or on simpler social-media-like sites like LiveJournal, etc. Those sites all had conte…

Very interesting points. Kind of a tangent, I had kind of felt nostalgic about the days of the personal/hobby websites you appear to allude to that seemed so prolific before Facebook/etc, and wondered why things had changed so much. It makes sense to me that now someone who used to be motivated to build a site about their life or a topic of interest now may often just sign up for a service like FB and occasionally do…

> It seems to require much less effort for folks, which is perhaps why they do this

It's also much easier to find readers.

Compare:

Facebook: become friends with your co-workers, now they see your picture

Blog: Please go to jupiter90000 (that's how many zeros?).blogspot.com to see my once a week pic updates!!

Re: Does Google execute JavaScript?

#94

Post by cbr »

andrewstuart2 wrote: post #66

This is a subject that really irks the engineering side of me. It's utterly ridiculous that engineering and efficiency decisions are so deeply affected by whether or not the largest search engine will properly index your content. Why is it that Google doesn't get flak for not discovering content that's engineered to send the absolute minimum over the wire, cache intelligently in localStorage and IndexedDB, and scale…

    It's utterly ridiculous that engineering and efficiency
    decisions are so deeply affected by whether or not the
    largest search engine will properly index your content.
There being more sites that can only be crawled and indexed well if you run js is probably good for Google relative to competitors. Anything that makes crawling the web harder increases the barrier to entry, making it harder for sites like DuckDuckGo to serve search results from their own crawl.

(Disclosure: I work for Google, on unrelated stuff.)

Re: Does Google execute JavaScript?

#95

Post by cbr »

tyingq wrote: post #47

Google's announcement when they started parsing javascript: https://webmasters.googleblog.com/2014/05/understanding-web-...

They had already been doing it for a while by 2014. For example, I saw them doing it on my site in January 2012: https://www.jefftk.com/p/googlebot-running-javascript

(Disclosure: I work for Google, on unrelated stuff, though I didn't at the time I wrote that blog post.)

Re: Does Google execute JavaScript?

#97

Post by user5994461 »

lern_too_spel wrote: post #67
dx034 wrote: post #45

Earlier quoted context omitted.

The chances of becoming a part of the search team are still very low. Even most Googlers won't know the exact details.

But within Google, you could ask somebody and get the right answer.

And outside Google, you could look for Google employees on linkedin who may be in position to have the answer and ask nicely ;)

Re: Does Google execute JavaScript?

#98

Post by andrewstuart2 »

paulddraper wrote: post #89
andrewstuart2 wrote: post #83

Earlier quoted context omitted.

Isomorphic. Thank you, I was searching my brain for that word for like half an hour. :-) > Why should your code care if it is running on my computer or yours? It shouldn't. But my users already care about perceived latency, and that is directly limited by the speed of light. My users want feedback as quickly as possible that their input has been received, and that something is happening in response. Thanks to the spe…

Yeah, I understand the server-side rendering vs. client-side updating, and the design benefits of API-driven development. And unfortunately, a lot of popular JS frameworks haven't done a great job about helping with these. Closure Library/Templates was meant to render server-side and bind JS functions after render, or create client-side dynamically. (Interestingly, the historic reasons were performance, not SEO.) Rea…

Yeah, I think I'm probably thinking much more heavily of the heavily-data-driven, dynamic web application use case since that's the kind of thing I've been working on for 5+ years now. I imagine that the vast majority of the internet content actually consists of much more long-form prose that doesn't benefit quite so much from a deferred-rendering approach since it varies little if any from user to user. In fact, that would probably be an overall systemic loss since now the same work is being done many times to render the same content, when it could be done once and cached for all.

And I don't expect Google or anyone to be able to support every edge case, either. I really would just like some sort of better solution that involves a global minimum of effort to achieve the same thing -- indexing what the user actually sees (non-private info, at least), and helping users discover sites that will give them a great experience and not just sites that give indexers a great experience.

Re: Does Google execute JavaScript?

#100

Post by inlined »

paulddraper wrote: post #89
andrewstuart2 wrote: post #83

Earlier quoted context omitted.

Isomorphic. Thank you, I was searching my brain for that word for like half an hour. :-) > Why should your code care if it is running on my computer or yours? It shouldn't. But my users already care about perceived latency, and that is directly limited by the speed of light. My users want feedback as quickly as possible that their input has been received, and that something is happening in response. Thanks to the spe…

Yeah, I understand the server-side rendering vs. client-side updating, and the design benefits of API-driven development. And unfortunately, a lot of popular JS frameworks haven't done a great job about helping with these. Closure Library/Templates was meant to render server-side and bind JS functions after render, or create client-side dynamically. (Interestingly, the historic reasons were performance, not SEO.) Rea…

[Disclosure: I work at Google but don't work on anything related to the crawler]

All this anger aside, I'm actually pretty impressed with the world we live in and proud of my company. Think about how far we've come that merely crawling and indexing the vastness of the internet is so mundane now. Now we should expect the whole internet to be downloaded and executed. That's got to be a great security and integrity problem. Surely someone had tried to break out of the sandbox. Can that be abused to affect SEO of other sites. The easy answer is "spin up a new VM for each page" but that would slow the indexing process down by orders of magnitude.

Post reply on HN