Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Googlebot's recent improvements might revolutionize web development

blog.workhere.io

Post reply on HN
Print view View original

41–50 of 97 posts

Re: Googlebot's recent improvements might revolutionize web development

#41

Post by wutbrodo »

marknutter wrote: post #25
jacquesm wrote: post #7

One potential problem here is that google will use this to widen the gap between it and the 'one page apps' web and other search engines (such as duckduckgo) that can't match it in resources. How strong of an advantage that will be in the long run is uncertain, I would rather see a web that ships pages with actual content in them than empty containers for a variety of reasons (most of which have to do with accessibil…

I'm fairly sure the web has not been the web that Tim Berners-Lee envisioned for a long time now.

.... and that's a very good thing per se. The idea that the development of something as important and universal to the web should be limited to what one man (no matter how visionary) was able to envision a couple of decades ago is beyond bizarre.

Re: Googlebot's recent improvements might revolutionize web development

#42

Post by marknutter »

acqq wrote: post #35

So you want to have the URL for every content but you don't want to provide the content as HTML, instead expecting that once the page is by client, the client only then separately loads the content? Only because it's easier to you to program, you want to deliver me the content much slower than you can? There is some strange logic there.

Slower in some respects and faster in others. Suppose I cache the data I get back from the server for page 1 and page 2 of content. Now, if the user switches between page 1 and page 2 they don't needlessly ask the server for the HTML every time like they do when relying on the server to render templates.

And I'm not sure where you're getting that it's "easier to program" single page apps than it is to simply rely on the server to render html on the server. The fact that it's not easy is the very reason we have so many competing front-end frameworks to solve the problem elegantly.

Re: Googlebot's recent improvements might revolutionize web development

#43

Post by Flenser »

mixonic wrote: post #18
andrenotgiant wrote: post #8

Taking a step back: The "Page" paradigm is still very much alive, despite these recent javascript parsing advances. 1. Google still needs a URL-addressable "PAGE" to which it can send Users. 2. This "PAGE" needs to be find-able via LINKS (javascript or HTML) and it needs to exist within a sensible hierarchy of a SITE. 3. This "PAGE" needs to have unique and significant content visible immediately to the user, and on…

I'd debate the phrase "step back". If you replace all your references to PAGE with URL, you get closer to a real meaning. URLs for single-page applications are a serialization of application state. The fact that we now have an application platform (JavaScript/HTTP) providing sharable, mostly-human-readable state sharing (URLs) and is also indexed and searchable is nothing short of incredible. Yes, the basic abstracti…

> these are applications running on Google's own servers. Google is running my application (and hundreds of thousands more)

Which is also very beneficial for Google as they'll likely be the only company doing that for a while, and the one able to do it for the most sites for a long time to come, maintaining Google's search index lead.

Re: Googlebot's recent improvements might revolutionize web development

#44

Post by workhere-io »

timq wrote: post #28

I can't believe that single page webapp are easier to write than true old website. Maybe you can gain some performance improvement but if you use a framework you will loose this very little gain. Those who claims that developpement is easier with framework on a single page have too learn programming, because for most case, the "old" way works very well and is incredibly faster than a bloated javascript page. I really…

Those who claims that developpement is easier with framework on a single page have too learn programming, because for most case, the "old" way works very well and is incredibly faster than a bloated javascript page.

Who says the page will become bloated with JS just because you use clientside loading? The mechanism I'm talking about can be done with something like 10 lines of JS or less. No one's saying you have to use AngularJS with every web page you make.

When the article say "put the CSS inside a tag on the page - and the JS inside a tag", it's just horrible, fuck it.

First of all, this is not a requirement of single page apps, just an option. Secondly, when you're developing, you would still have your JS and CSS in separate files. Your compiler would then minify the whole thing and put it inside your minified HTML file.

Re: Googlebot's recent improvements might revolutionize web development

#45

Post by prophead »

workhere-io wrote: post #32
andrenotgiant wrote: post #8

Taking a step back: The "Page" paradigm is still very much alive, despite these recent javascript parsing advances. 1. Google still needs a URL-addressable "PAGE" to which it can send Users. 2. This "PAGE" needs to be find-able via LINKS (javascript or HTML) and it needs to exist within a sensible hierarchy of a SITE. 3. This "PAGE" needs to have unique and significant content visible immediately to the user, and on…

As I mentioned in the post, all these problems can be solved by using real paths/URLs and changing them dynamically using pushState.

But the onus is still on the developer to choose _what_ gets a unique URL and what does not.

It might be good for user deeplinking capabilities to change the URL every time any type of state change is made (for example sorting a list by date instead of name) - But exposing that many URLs to Google would be bad.

(This is the modern equivalent of the age-old "infinite calendar" problem that Googlebot had to deal with when dynamic calendar apps let you navigate to dates 2 millennia in the future.)

Re: Googlebot's recent improvements might revolutionize web development

#46

Post by workhere-io »

prophead wrote: post #45
workhere-io wrote: post #32

Earlier quoted context omitted.

As I mentioned in the post, all these problems can be solved by using real paths/URLs and changing them dynamically using pushState.

But the onus is still on the developer to choose _what_ gets a unique URL and what does not. It might be good for user deeplinking capabilities to change the URL every time any type of state change is made (for example sorting a list by date instead of name) - But exposing that many URLs to Google would be bad. (This is the modern equivalent of the age-old "infinite calendar" problem that Googlebot had to deal with w…

I agree with you; developers definitely have to think about the URLs they're exposing to Googlebot. But this is essentially no different from how things were before. Your example with sorting a list by date instead of name would be done with a query string (which Google does index to a point), e.g. /users?sortBy=date=&from=392. This can obviously create quite a lot of links to the same content, and developers should know how to handle this situation. Again, not different from before - single page apps don't change anything in this regard.

Re: Googlebot's recent improvements might revolutionize web development

#47

Post by workhere-io »

jacquesm wrote: post #7

One potential problem here is that google will use this to widen the gap between it and the 'one page apps' web and other search engines (such as duckduckgo) that can't match it in resources. How strong of an advantage that will be in the long run is uncertain, I would rather see a web that ships pages with actual content in them than empty containers for a variety of reasons (most of which have to do with accessibil…

One potential problem here is that google will use this to widen the gap between it and the 'one page apps' web and other search engines (such as duckduckgo) that can't match it in resources.

There are free and open source tools available that would help search engines parse pages containing JS (PhantomJS comes to mind).

Re: Googlebot's recent improvements might revolutionize web development

#48

Post by rcsorensen »

workhere-io wrote: post #36
rcsorensen wrote: post #23

Sending the framework of a page to your users and expecting them to do all the heavy lifting and slow loading of constructing the page and fetching the data is still rather unfriendly if you can afford a server to construct it. If you love your users, give them HTML and let the Javascript enhance it. Projects like Facebook's React ( http://facebook.github.io/react/docs/top-level-api.html#reac... ) and Rendr ( https:/…

Clientside rendering doesn't need to be heavy at all. In fact, you could do it with just $.getJSON('/api/users', function(data) { $('#users').text(data.content) }. Sure, that requires jQuery, but most "normal" sites require jQuery, too.

Sure!

And now your poor little mobile user has to wait for the page to download, then to execute javascript, then to wait for the API response, then wait for the DOM to update.

Or you could put it in the HTML, and then they just have to wait for the page to download.

Re: Googlebot's recent improvements might revolutionize web development

#49

Post by Touche »

iLoch wrote: post #19
jacquesm wrote: post #7

One potential problem here is that google will use this to widen the gap between it and the 'one page apps' web and other search engines (such as duckduckgo) that can't match it in resources. How strong of an advantage that will be in the long run is uncertain, I would rather see a web that ships pages with actual content in them than empty containers for a variety of reasons (most of which have to do with accessibil…

It's not difficult to set up middleware that'll render the page for any clients that require it. (For instance, we can assume any client that identifies as "bot" that's not Google probably wants a pre-rendered page, which we can do quite effortlessly. Here's one implementation for Nodejs: https://prerender.io , or you can always roll your own with something like Phantom.js.

Note that sending a different response to googlebot than what you send to normal users is a violation of Google's guidelines and can get your site penalized. Use at your own peril.

Re: Googlebot's recent improvements might revolutionize web development

#50

Post by workhere-io »

Author here. Some of you are saying that this will lead to bloated, JS-heavy websites. I disagree. The JS necessary for making a single page app can be done with something like 10 lines of JS (plus jQuery or something similar, but that is already included in most normal pages anyway).

A single page app isn't JS-heavy by definition, and a "normal" page (with HTML generated on the server) can easily be JS-heavy. It all depends on how you program it. Just keep in mind that single page apps don't necessarily need to use heavy frontend frameworks such as Knockout, Ember or AngularJS.

Post reply on HN