If you don't want the deal with the ambiguity of whether your AJAX will run or not, I'll shamelessly suggest https://www.prerender.cloud/ which is helping a few sites who couldn't get Google to execute their AJAX.
Does Google execute JavaScript?
81–90 of 114 posts
Re: Does Google execute JavaScript?
#82Earlier quoted context omitted.
Google brings something that no amount of fancy engineering and elegant solutions can provide: New Users. Because of that it is necessary to dance to their SEO tune.
Isn't it ironic, though? Google exists because they had the best solution in 1998 for helping users discover the content that already existed. Of course, now that they're gigantic, they now are the primary force that's adding unneeded engineering complexity to keep content in a format they can already read. So who's really helping whom?
>So who's really helping whom?
Offtopic, but Google's propensity to migrate anything popular to their own hosted content is a better example of this. They find ways to present the good stuff without the end user ever visiting your site. At some point, this starves off the sources.
Re: Does Google execute JavaScript?
#83This is a subject that really irks the engineering side of me. It's utterly ridiculous that engineering and efficiency decisions are so deeply affected by whether or not the largest search engine will properly index your content. Why is it that Google doesn't get flak for not discovering content that's engineered to send the absolute minimum over the wire, cache intelligently in localStorage and IndexedDB, and scale…
> result in hundreds of thousands of wasted engineering hours trying to enable the idea of server-side and client-side rendering with the same code It this problem really that difficult? Why? Why should your code care if it is running on my computer or yours? Isomorphic JS has been around for years. Build your product on bloated tech stack relying on a increasingly poorly planned web of dependencies, and I'll agree i…
Isomorphic. Thank you, I was searching my brain for that word for like half an hour. :-)
> Why should your code care if it is running on my computer or yours?
It shouldn't. But my users already care about perceived latency, and that is directly limited by the speed of light. My users want feedback as quickly as possible that their input has been received, and that something is happening in response. Thanks to the speed of light, this would ideally take place instantly right in front of their eyeballs. That can't happen yet, so as much as can realistically happen on my user's CPU, memory, and storage is the next best thing.
> They do crawl REST APIs. Specifically, ones using Hypertext Markup Language.
What I meant to say was JSON, so I'm contributing to my own pet peeve of saying "REST" and meaning "JSON." :-p
HTML is awesome and does a wonderful job of letting me mark content in a way that it can be efficiently rendered, semantically, and be both human-readable and marginally machine readable. There are two problems, though. The first is that full documents (since the article points out AJAX is not performed by Google) are incredibly repetitive and wasteful, especially when retrieving the same content fragments multiple times.
The second is that it is strongly coupling content and presentation, two orthogonal concepts, much earlier than is optimal. Sure, you can cache full documents and display them when requested again, but the more common case is that a large subset of what I just displayed to my user will be displayed again, with one new item, but has still invalidated my cache because the granularity is at the full-page presentation level, and not the business domain object level. If, instead, I cache and render business objects on the client side, I can be more intelligent and granular with my caching strategy, react much more quickly to my users' feedback, and have a much smaller impact on their constrained devices. Not only that, but transmitting structured business objects instead of presentation-structured content lets me more efficiently reuse that data across devices for which HTML may not be the most effective way to present the data to them.
My personal architectural bents aside, the truth remains that content discovery agents (e.g. indexers) should not be treated as content delivery agents with such a huge influence on content format. This ends up creating (IMO) too much influence over external engineering decisions, rather than allowing engineers to think critically about the right architecture that gives users the best possible experience.
Most importantly, I'm not saying that all the engineering effort should be placed upon the discovery agents. Of course there are limits on how much they can discover on their own, and (as always in matters involving many parties) there need to be good conversations about the state of things, and what we think is the right direction to go to support each other and our users. It's just been my opinion lately that this is not so much a conversation anymore as a unidirectional stream of "best practices" coming from a single group.
Re: Does Google execute JavaScript?
#84My pet theory is that Google actually developed chrome as a web crawler and that consumer release was to ensure that Google would always be able to crawl pages (since sites would always want to work properly with chrome) It also explains why they effectively killed flash and Java applets. They were competing technologies that weren't owned by Google and not crawlable. If they would have taken off, Google's position a…
Steve Jobs killed Flash by refusing to put it in the iPhone.. Google nailed down the coffin. (And we owe both our eternal gratitude)
Re: Does Google execute JavaScript?
#85Earlier quoted context omitted.
And here I was thinking that it was Apple and the iPhone that effectively killed Flash. Silly me.
They disliked flash for different reasons. Google for search and apple because flash games were too powerful, jobs wanted those games turned into iPhone apps. The offical reasons were speed and security, which was obviously a farce since all the major browser vendors wrote their own PDF viewers which is both slow and known for tons of security issues as well. They could have just as easily wrote their own fast and se…
>They could have just as easily wrote their own fast and secure implementations of flash and Java instead of disabling them. Especially for Java given Google's deep experience with it and that they already have their own runtime.
Yup, Google should:
1. Reverse Engineer a proprietary language/API (Flash).
2. Write a secure VM for it (Flash and Java), trying to hit a moving target.
or
Get everyone to use an open, standardized tech (HTML5/js).
Can you give me one reason Google should prop up Flash?
Re: Does Google execute JavaScript?
#86Earlier quoted context omitted.
Isn't it ironic, though? Google exists because they had the best solution in 1998 for helping users discover the content that already existed. Of course, now that they're gigantic, they now are the primary force that's adding unneeded engineering complexity to keep content in a format they can already read. So who's really helping whom?
>So who's really helping whom? Offtopic, but Google's propensity to migrate anything popular to their own hosted content is a better example of this. They find ways to present the good stuff without the end user ever visiting your site. At some point, this starves off the sources.
"That's some nice data you go there, guys. It'd be a shame if... somebody scraped it and kept that page view and ad revenue for themselves."
Again ironically, this will put Google out of business if they keep it up, unless they can start to collect all that data on their own or otherwise incentivize content producers to allow them access.
Re: Does Google execute JavaScript?
#87My pet theory is that Google actually developed chrome as a web crawler and that consumer release was to ensure that Google would always be able to crawl pages (since sites would always want to work properly with chrome) It also explains why they effectively killed flash and Java applets. They were competing technologies that weren't owned by Google and not crawlable. If they would have taken off, Google's position a…
I'm using Google Chrome in an instrumented fashion on https://urlscan.io/ and I've always thought that using Chrome that way would make a pretty good crawler. Maybe this is me being naive, but it's certainly not surprising that crawlers do execute JavaScript at this point in time.
Re: Does Google execute JavaScript?
#88I think one of the most frustrating things about indexing from Google is the complete lack of transparency. I understand that it helps Google slow down the arms race of search engines, but it also means that devs doing 100% banal work need to sift through mountains of rumors and spin up sites to test assumptions.
I have literally heard every combination of practices with regards to SEO and have no idea what is truly correct. Every source contradicts each other, Google employee statements contradict those, etc.
Re: Does Google execute JavaScript?
#89Earlier quoted context omitted.
> result in hundreds of thousands of wasted engineering hours trying to enable the idea of server-side and client-side rendering with the same code It this problem really that difficult? Why? Why should your code care if it is running on my computer or yours? Isomorphic JS has been around for years. Build your product on bloated tech stack relying on a increasingly poorly planned web of dependencies, and I'll agree i…
Isomorphic. Thank you, I was searching my brain for that word for like half an hour. :-) > Why should your code care if it is running on my computer or yours? It shouldn't. But my users already care about perceived latency, and that is directly limited by the speed of light. My users want feedback as quickly as possible that their input has been received, and that something is happening in response. Thanks to the spe…
Yeah, I understand the server-side rendering vs. client-side updating, and the design benefits of API-driven development. And unfortunately, a lot of popular JS frameworks haven't done a great job about helping with these.
Closure Library/Templates was meant to render server-side and bind JS functions after render, or create client-side dynamically. (Interestingly, the historic reasons were performance, not SEO.)
React and Meteor have good server-side stories. Angular 2 is getting one.
I would say there is a lot of low hanging fruit in just avoiding most client-side JS. Take http://wiki.c2.com/ -- the "original" wiki. That should all be static. Same with blogs, documentation, and lots of other public, indexable content.
Re: Does Google execute JavaScript?
#90They have been for almost three years...