Now, what kinds of V8 vulnerabilities can we exploit to get inside Google? Said every intelligence agency everywhere.
How Googlebot crawls JavaScript
31–40 of 114 posts
Re: How Googlebot crawls JavaScript
#32For people wondering about Ajax requests, Googlebot is performing them very well together with SVG rendering.
For example this URL:
https://www.chemeo.com/predict?smiles=CCCC
is performing the drawing of the molecule using RaphaelJS, then pulling the corresponding molecule from the database using Ajax and updating the page. Googlebot is performing all the steps perfectly well to add the end index the page.
It is very annoying because this is not important in our case, what we want is the good indexing of the main data pages, not these pages... I do not want to block the bot yet, but I need to figure out a way to have the main page better ranked.
Re: How Googlebot crawls JavaScript
#33Earlier quoted context omitted.
IMHO: Websites that don't have "realtime" content should always stick with traditional HTML. I'm a Webdeveloper myself and i don't like the JavaScript Frontend trend. Many Devs use Frontend JS in places where it's absolutely not needed. If you're building an App that updates in realtime, shows informations while it's created, i'm fine with Frontend JS, but it's an overkill for most content pages. Sure, it depends on…
I mostly agree, but at the same time, the rise of native apps has raised the bar of what people expect in terms of UX. Take Hacker News and Reddit, primarily content based sites and a good fit for the classic server rendered HTML approach. Still a lot of people prefer using native apps to access that content. You can only get so far by adding some CSS to make the site responsive, but you won't be anywhere near the UX…
But you can still progressively enhance with JS to achieve that nice UI, and often it will be more usable because it's built on a solid RESTful foundation that is close to browser behaviour and therefore user expectation.
My experience with JS only apps is that they're often less usable, more brittle, and often don't work at all in IE
Re: How Googlebot crawls JavaScript
#34I feel that dynamic websites are not websites, but applications. Even after this thorough research, I'd still be very wary of turning a primarily content-based site into a dynamic app. A plain HTML site is accessible, and will be accessible in a 1000 years. A site depending on (external) JavaScript sources will force John Titor to travel back in time to find version 1.x of jQuery. Starting with JavaScript abandons pr…
For those that ignore it, John Titor[0] was a time traveler sent back in time to acquire some obsolete IBM machine which is needed in the future to debug some legacy code. [0] http://en.wikipedia.org/wiki/John_Titor
You're probably not a native speaker of English, but of Latin. In Latin, `ignorare` can mean `not to know` in addition to its meaning of `not to pay attention to`, but in English, it only has the meaning of `not to pay attention to`.
Vale.
Re: How Googlebot crawls JavaScript
#35I've long thought that the need for a high performance sandboxed JavaScript VM was the real impetus for Google's investment in v8, and that Chrome was just a useful opportunity to leverage it and to get external contribution. Is there any evidence that this is the case?
Re: How Googlebot crawls JavaScript
#36For people wondering about Ajax requests, Googlebot is performing them very well together with SVG rendering. For example this URL: https://www.chemeo.com/predict?smiles=CCCC is performing the drawing of the molecule using RaphaelJS, then pulling the corresponding molecule from the database using Ajax and updating the page. Googlebot is performing all the steps perfectly well to add the end index the page. It is very…
However, if you search on Google:
"Property Prediction for Butane" site:https://www.chemeo.com
Re: How Googlebot crawls JavaScript
#37Earlier quoted context omitted.
I mostly agree, but at the same time, the rise of native apps has raised the bar of what people expect in terms of UX. Take Hacker News and Reddit, primarily content based sites and a good fit for the classic server rendered HTML approach. Still a lot of people prefer using native apps to access that content. You can only get so far by adding some CSS to make the site responsive, but you won't be anywhere near the UX…
But you can still progressively enhance with JS to achieve that nice UI, and often it will be more usable because it's built on a solid RESTful foundation that is close to browser behaviour and therefore user expectation. My experience with JS only apps is that they're often less usable, more brittle, and often don't work at all in IE
Progressive enhancement work well for simple stuff. Like progressively enhancing a form post, or a "like" button which just sends an Ajax request. But as the complexity grows, progressive enhancement doesn't really scale and you end up with two separate versions of your site/app.
I agree that Javascript only apps are often less usable, because the devs making them aren't testing enough on different browsers and devices. But the trend of the "Javascript only" approach is certainly driven by more than just frontend devs that want to use shiny new things (even if that is a factor as well).
Re: How Googlebot crawls JavaScript
#38I feel that dynamic websites are not websites, but applications. Even after this thorough research, I'd still be very wary of turning a primarily content-based site into a dynamic app. A plain HTML site is accessible, and will be accessible in a 1000 years. A site depending on (external) JavaScript sources will force John Titor to travel back in time to find version 1.x of jQuery. Starting with JavaScript abandons pr…
> A plain HTML site is accessible, and will be accessible in a 1000 years. A site depending on (external) JavaScript sources will force John Titor to travel back in time to find version 1.x of jQuery.
Not to sound completely apathetic, but so what? Most of us aren't building sites that we expect to be around in 10 years, much less 1000 years. The ephemeral nature of what we're building isn't lost on us - we're trading that guaranteed longevity for an improved development process (though some obviously disagree).
Frequently, writing a traditional website with any sort of meaningful UI interactions was/is kind of a mess. Most of us don't write these applications (and you're right, they are applications) because we have any particular affinity for JavaScript, but because it makes the whole process much nicer. It still sucks, it's just nicer.
Sure, progressive enhancement is a thing. And it's a great idea. In practice, top-down directives will probably be something akin to "Sure, do that, but do it on your own time and not at the expense of anything else." The realized benefits are very low (the % of users with JavaScript disabled is incredibly small), and saying something like "our site won't be accessible in 1000 years otherwise" is likely to get you mostly blank stares. It's a pretty big investment with very little benefit to most companies.
Sure, 50 years down the road if these sites still exist they'll probably be nigh-unusable without some sort of "ES6 emulator mode", but so what? I don't think we'll go wanting for any historical artifacts from this time period. If we do, it'll be because future generations have no interest in our generation - not because we didn't produce enough relics.
Re: How Googlebot crawls JavaScript
#39I feel that dynamic websites are not websites, but applications. Even after this thorough research, I'd still be very wary of turning a primarily content-based site into a dynamic app. A plain HTML site is accessible, and will be accessible in a 1000 years. A site depending on (external) JavaScript sources will force John Titor to travel back in time to find version 1.x of jQuery. Starting with JavaScript abandons pr…
I think the trend of "turning a primarily content-based site into a dynamic app", and indeed most of what has been referred to as "Web progress", "moving the Web forward", etc. comes from the desire of content producers to obtain and maintain more control over their content. Look at how browsers have evolved to de-emphasise features which give the user control while adding those that are author-targeted.
We're moving from browsers being viewers for simple HTML documents (which can be copied, shared, and linked via simple means), to a platform for running complex applications written in JavaScript which often render data retrieved in proprietary formats from proprietary APIs. The "open by default" nature of plain HTML has become the "closed by default" of the data processed by web apps, much like with many native apps. Native app platforms (e.g. mobile) are also gradually becoming more "closed by default"; I'm not sure if that's a related trend.
Google can say more sternly that using JavaScript can have ugly consequences for user experience and accessibility, but they are swimming upstream: The web seems to be moving on to fancy new technologies regardless of what their SEO says.
Part of the reason is because Google themselves are doing this in many of their products... some of their employees probably disagree with "JS everything", but they're in the minority.
Re: How Googlebot crawls JavaScript
#40Now, what kinds of V8 vulnerabilities can we exploit to get inside Google? Said every intelligence agency everywhere.
From 2006 to 2010, my primary role at Google was JavaScript execution in the indexing pipeline. I knew I was likely executing every known JavaScript engine exploit out there plus a good number of 0-days, and ran the javascript engine in a single-threaded subprocess with a greatly restricted set of allowed system calls.
Certainly the right combination of kernel zero-days and JS interpreter exploits could be used to take over the machine, but it would be non-trivial.