Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Googlebot's recent improvements might revolutionize web development

blog.workhere.io

Post reply on HN
Print view View original

61–70 of 97 posts

Re: Googlebot's recent improvements might revolutionize web development

#61

Post by CHY872 »

This seems like an odd thing to be shouting about.

As far as I can see, throughout this thread the performance benefits of single page apps are touted as being fantastic, making it worthwhile to use the new technology etc.

When has performing operations efficiently ever been the domain of the web? Websites in my experience have the worst performance of almost any software I use! I've seen developers cite 200ms or longer to load a page as being a good benchmark - that seems pretty awful to me.

If getting this tiny performance improvement (which often results in poorer performance on the first load (not ideal for many)) is so critical, why do the same developers not invest in writing more performant server apps? Yes, often the database is a bottleneck, but these problems can in general be worked around (either by use of faster queries or caching etc).

Why attempt to get a small performance benefit by saving 30-odd kB of HTML on each page load (static and so essentially free for the server), when one could get a much larger performance benefit by optimising the backend?

Almost all serious sites will still see their page load being limited by the time it takes to produce the page. It's possible to write really fast websites (try http://forum.dlang.org/) but no one seems to do it :(

If anything but almost all of your website is static, you won't be saving all that much time.

Re: Googlebot's recent improvements might revolutionize web development

#62

Post by andrenotgiant »

mixonic wrote: post #18
andrenotgiant wrote: post #8

Taking a step back: The "Page" paradigm is still very much alive, despite these recent javascript parsing advances. 1. Google still needs a URL-addressable "PAGE" to which it can send Users. 2. This "PAGE" needs to be find-able via LINKS (javascript or HTML) and it needs to exist within a sensible hierarchy of a SITE. 3. This "PAGE" needs to have unique and significant content visible immediately to the user, and on…

I'd debate the phrase "step back". If you replace all your references to PAGE with URL, you get closer to a real meaning. URLs for single-page applications are a serialization of application state. The fact that we now have an application platform (JavaScript/HTTP) providing sharable, mostly-human-readable state sharing (URLs) and is also indexed and searchable is nothing short of incredible. Yes, the basic abstracti…

The point I was trying to make is this:

With every new capability from Googlebot comes new opportunities for us to screw it up as developers.

If we were to replace PAGE with URL, and URL is simply a serialization of application STATE, we could easily end up with infinite URLs that lead to STATES that are not really that different, unique or appealing as answers to queries users type into Google.

When deciding how to build Search-accessible Web Apps, and specifically what to expose to Google, we need to keep in mind that Google likes PAGES that follow the requirements I detailed above.

Re: Googlebot's recent improvements might revolutionize web development

#63

Post by workhere-io »

CHY872 wrote: post #61

This seems like an odd thing to be shouting about. As far as I can see, throughout this thread the performance benefits of single page apps are touted as being fantastic, making it worthwhile to use the new technology etc. When has performing operations efficiently ever been the domain of the web? Websites in my experience have the worst performance of almost any software I use! I've seen developers cite 200ms or lon…

If anything but almost all of your website is static, you won't be saving all that much time.

Single page apps can easily be static (static HTML page + static JSON). The point of this would be to decrease the download size for each new page visited by the user.

Re: Googlebot's recent improvements might revolutionize web development

#64

Post by CHY872 »

workhere-io wrote: post #63
CHY872 wrote: post #61

This seems like an odd thing to be shouting about. As far as I can see, throughout this thread the performance benefits of single page apps are touted as being fantastic, making it worthwhile to use the new technology etc. When has performing operations efficiently ever been the domain of the web? Websites in my experience have the worst performance of almost any software I use! I've seen developers cite 200ms or lon…

If anything but almost all of your website is static, you won't be saving all that much time. Single page apps can easily be static (static HTML page + static JSON). The point of this would be to decrease the download size for each new page visited by the user.

I think you missed my point. In each web page downloaded there's a bunch of (basically constant) static data - to download your javascript files, and set up your document - your template (or similar). This is the only data that single page apps can eliminate - everything else must either be queried from the server or can already be cached.

Some sites obviously inline CSS or JavaScript, but that can be eliminated if necessary (and only affects the first page load anyway).

This information is free to generate on the server side, so it's not slowing down that computation at all (it's just a stringbuilder function, essentially). Furthermore, the transfer time is generally not the deciding factor - it's the server side time to put the rest of the information together.

To give one example, I went to a typical website - the Guardian (it's a fairly standard high-traffic news website). Chrome informs me that in order to request one article, it took 160ms to load the html - 140ms of waiting and 20ms of downloading. Now, the RTT is about 14ms, so that's about 110ms of generating the web page and 20ms of actually downloading it. It's about 30kB of compressed HTML (150kB uncompressed), most of it's 'static content' - inlined CSS and JS.

Them using the single page model would reduce the page download time (apart from the first page) by an absolute maximum of 20ms - which means that the time to load each page has been reduced by about 12%.

This is fine, but almost all of the data is just the result of string concatenations and formatting - i.e. free processing (or at least almost-free processing). It's getting the rest of the data together that's somehow taking the 100ms (or crap implementations).

The cost of moving data around on websites is typically small compared to the actual production time of the content. That's why we see people preferring to inline huge amounts of CSS etc on each web page and having people download it time after time - because it's only about 10kB compressed the data transfer is inconsequential, and normally is dominated by the RTT.

Spending all the time writing these frameworks because of performance benefits is a fallacy - the data still has to be generated somewhere, and if it happens dynamically it's slow as hell. The savings can never become that great - at most they lead to 20-30ms of improvements if bandwidth is acceptable.

Writing the frameworks because they make development easier is a much more reasonable argument.

This still all detracts away from the fact that non-static websites are typically dog slow and they shouldn't be.

Re: Googlebot's recent improvements might revolutionize web development

#65

Post by Flenser »

jacquesm wrote: post #7

One potential problem here is that google will use this to widen the gap between it and the 'one page apps' web and other search engines (such as duckduckgo) that can't match it in resources. How strong of an advantage that will be in the long run is uncertain, I would rather see a web that ships pages with actual content in them than empty containers for a variety of reasons (most of which have to do with accessibil…

At the head of the index it probably won't make much difference, big sites will render the initial page on the server, it's the long tail where it will be client only. It may make scaling a website easier though as frameworks will start making adding or moving rendering from client to server trivial. So the path from the long tail to the head will be easier to navigate. (if you'll excuse me mixing my metaphors)

Re: Googlebot's recent improvements might revolutionize web development

#67

Post by rgbrgb »

rcsorensen wrote: post #31
marknutter wrote: post #26

Earlier quoted context omitted.

The lifting ain't that heavy. And besides, wouldn't you want your server spending its precious cycles on things the clients absolutely cannot handle?

Like `ssorallen says, slow CPUs and limited battery life is an issue. In addition, latency on mobile networks is rough and should be avoided for your users whenever possible. You have two choices when taking an SPA approach: 1) Give your users a webpage that is usable, readable, and navigable as soon as they get it. 2) Give your users the skeleton of a webpage that they then have to accept the latency of javascript p…

SPAs reduce the latency of data fetches because each new resource is a few bytes of JSON rather than some kb of HTML.

Re: Googlebot's recent improvements might revolutionize web development

#68

Post by rcsorensen »

rgbrgb wrote: post #67
rcsorensen wrote: post #31

Earlier quoted context omitted.

Like `ssorallen says, slow CPUs and limited battery life is an issue. In addition, latency on mobile networks is rough and should be avoided for your users whenever possible. You have two choices when taking an SPA approach: 1) Give your users a webpage that is usable, readable, and navigable as soon as they get it. 2) Give your users the skeleton of a webpage that they then have to accept the latency of javascript p…

SPAs reduce the latency of data fetches because each new resource is a few bytes of JSON rather than some kb of HTML.

Of subsequent data fetches. Not the first page load. Using JSON fetches to update documents also gives you the ability to not throw away the DOM/CSS and maintain the execution context.

If you use the right technology, you can give a usable application out of the gate without waiting and use JSON data fetches for every subsequent request.

Re: Googlebot's recent improvements might revolutionize web development

#69

Post by voltagex_ »

workhere-io wrote: post #57
justinph wrote: post #17

Earlier quoted context omitted.

Ok, I see your point. But, have Yahoo or Bing or DuckDuckGo made the transition to be able to crawl the web with a full JS & DOM rendering engine? I doubt it. By eschewing that compatibility we're setting a very high bar for what any competitor to google would have to achieve. I like google. I just don't think it's good to have one company own a market so completely.

But, have Yahoo or Bing or DuckDuckGo made the transition to be able to crawl the web with a full JS & DOM rendering engine? They can just use PhantomJS ( http://phantomjs.org/ ), which is free and open source.

They could, but I wonder what it'd take to scale it to crawling that number of pages.

I think only Bing would have the cash and resources to build that.

Re: Googlebot's recent improvements might revolutionize web development

#70

Post by drakaal »

Optimism rather than fact.

There is a lot more to it. I am pretty well known as an SEO, and while I would love this to be true it isn't.

Google's improvements to GoogleBot are mostly targeting spam, and obfuscation of content. The ideas is not to discover content as much as it is to avoid having content hidden.

Previously you could have a webpage that appeared to be about Puppies, but then used any number o dynamic methods to instead show naked cam girls. Google worked to fix this.

Google is now doing some indexing of named anchors, and this allows for linking to a page with in a page as it were. But that is a Long ways from building indexable single page applications.

-Brandon Wirtz SEO (formerly Greatest Living American) http://www.blackwaterops.com

Post reply on HN