Live data from Hacker News

Googlebot's recent improvements might revolutionize web development

blog.workhere.io

51–60 of 97 posts

Re: Googlebot's recent improvements might revolutionize web development

#51
post #35

So you want to have the URL for every content but you don't want to provide the content as HTML, instead expecting that once the page is by client, the client only then separately loads the content? Only because it's easier to you to program, you want to deliver me the content much slower than you can? There is some strange logic there.

Only the first load is slower. If you click on any links within the page they will render much faster as you skip the whole full page load thing.

Re: Googlebot's recent improvements might revolutionize web development

#52

Earlier quoted context omitted.

Clientside rendering doesn't need to be heavy at all. In fact, you could do it with just $.getJSON('/api/users', function(data) { $('#users').text(data.content) }. Sure, that requires jQuery, but most "normal" sites require jQuery, too.

Sure! And now your poor little mobile user has to wait for the page to download, then to execute javascript, then to wait for the API response, then wait for the DOM to update. Or you could put it in the HTML, and then they just have to wait for the page to download.

For each new page on your site that your user loads, the benefit of single page apps becomes greater: Now she only has to load a bit of JSON, not a full page. So essentially single page apps are a tiny bit slower on the first page load, but much quicker on subsequent page loads.

Re: Googlebot's recent improvements might revolutionize web development

#53
This is a great improvement, but I'm struck by two things:

1. The state of JavaScript-only application development is still nascent. The number of JS-only sites I see that are buggy, don't use PushState correctly, or have other shortcomings is growing faster than the overall trend. Not that it can't be done well, but if your JavaScript-only "app" is really just a standard website, you might want to re-think your approach.

2. There has to be a better way. There are distinct benefits to approaches in caching content and providing feedback, but JavaScript seems to be a kluge-y approach. It reminds me of frames back in the day. Some of this is browser support; some of this is lack of standardization; some is perhaps a missing piece of the HTML spec; etc.

Re: Googlebot's recent improvements might revolutionize web development

#55
post #5

I wouldn't actually call this "recent" improvements. I mean, Google has been handling JavaScript for years now. And they're just now coming out and publicly saying it. Which is typical Google.

What they were saying before was that you always need a HTML fallback for JS-generated content. Now it seems they're saying you don't necessarily need to.

Re: Googlebot's recent improvements might revolutionize web development

#56

Earlier quoted context omitted.

Sure! And now your poor little mobile user has to wait for the page to download, then to execute javascript, then to wait for the API response, then wait for the DOM to update. Or you could put it in the HTML, and then they just have to wait for the page to download.

For each new page on your site that your user loads, the benefit of single page apps becomes greater: Now she only has to load a bit of JSON, not a full page. So essentially single page apps are a tiny bit slower on the first page load, but much quicker on subsequent page loads.

I'm suggesting the hybrid approach here, exemplified by the Rendr and React.

Give the full html of the first page, load the JS necessary. You get the best of both worlds.

On the first request, the client receives a rendered html page, gets to read the page, then JS functionality is attached.

On any subsequent pages, you do a simple JSON request and update the DOM.

The approaches come together without sacrificing anything. You get to write a single code base using JS (with a node.js backend doing string manipulation so it doesn't even need a DOM), the user gets a page without having to wait for external resource loads, and all page transitions past that are a simple data-fetch away.

Re: Googlebot's recent improvements might revolutionize web development

#57

Earlier quoted context omitted.

Roads are meant for horses, right? We shouldn't let one company, Ford, dictate how the roads work. My man, if we only used infrastructure and technology in the way it was originally intended and narrowly imagined, the world would be a dim place.

Ok, I see your point. But, have Yahoo or Bing or DuckDuckGo made the transition to be able to crawl the web with a full JS & DOM rendering engine? I doubt it. By eschewing that compatibility we're setting a very high bar for what any competitor to google would have to achieve. I like google. I just don't think it's good to have one company own a market so completely.

But, have Yahoo or Bing or DuckDuckGo made the transition to be able to crawl the web with a full JS & DOM rendering engine?

They can just use PhantomJS (http://phantomjs.org/), which is free and open source.

Re: Googlebot's recent improvements might revolutionize web development

#58
With the new improvements to Googlebot, single page apps will likely advance from being niche solutions for non-public websites to being the default way to build websites. A website will contain a single HTML page (typically heavily cached and served via a CDN). The JS on that page will then fetch content (as JSON) from the server and change the path as necessary using pushState.

I find the cheerleading for single page websites disconcerting and the proposed benefits unconvincing. Why should this be the default way to build websites? A few desultory upsides are presented without a full consideration of the multiple downsides to client-side development.

The biggest advantage of thick-client architecture is sending less data to the client and, if you like using javascript, writing everything in js, but there are multiple downsides compared to more traditional thin-client websites - load times which depend more on client capabilities (hugely variable and out of your control) than servers, dependence on js on the client, loading pages while your content is placed in the dom by js, forcing everyone to write in js instead of switching language on the server whenever they like, ignoring the simple document model of html served at predictable URIs, which has served the web so well and means you can use dynamic or static documents, full documents can be cached for very quick serving and by intermediaries, etc, etc. Of course some of these can be overcome, but there are serious obstacles, and the advantages are meagre to non-existent unless you enjoy javascript and feel its the only language you'll ever need.

For someone who doesn't like working in js, and/or doesn't have a huge amount of logic already in js (many websites work just fine with some limited ajax), trying to force every website into the procrustean bed of client-side development is not an appealing prospect. I can see why it appeals to those who have already invested in js frameworks, but predictions of its future dominance on the web, like predictions that eworld, activex or mobile would replace the web, are overblown.

I suspect the birth and death of Javascript will be a footnote in the history of the web, rather than taking it over as this article suggests. If anything we should be looking to replace our dependence on js, not making it mandatory.

Re: Googlebot's recent improvements might revolutionize web development

#59
post #7

One potential problem here is that google will use this to widen the gap between it and the 'one page apps' web and other search engines (such as duckduckgo) that can't match it in resources. How strong of an advantage that will be in the long run is uncertain, I would rather see a web that ships pages with actual content in them than empty containers for a variety of reasons (most of which have to do with accessibil…

Berners-Lee envisioned a decentralised peer-to-peer information sharing network where everyone was a server and a client.

Re: Googlebot's recent improvements might revolutionize web development

#60

Earlier quoted context omitted.

Roads are meant for horses, right? We shouldn't let one company, Ford, dictate how the roads work. My man, if we only used infrastructure and technology in the way it was originally intended and narrowly imagined, the world would be a dim place.

Ok, I see your point. But, have Yahoo or Bing or DuckDuckGo made the transition to be able to crawl the web with a full JS & DOM rendering engine? I doubt it. By eschewing that compatibility we're setting a very high bar for what any competitor to google would have to achieve. I like google. I just don't think it's good to have one company own a market so completely.

I agree that diversity is important, but if Google is owning the market via innovation, they deserve to own the market.

In this case, they're closing an ever widening rift between human consumable content and that which is targeted for search engines. The data that comes down the pipe can still be semantic, it's just glued together differently.

Web technology is presently outpacing the ability of indexing services. The onus is on Google's competitors now; they need to catch up. Luckily js rendering on the server isn't an academic problem, it's one of resource allocation.

Post reply on HN