Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Incremental Builds in Gatsby Cloud

gatsbyjs.org

Post reply on HN
Print view View original

21–30 of 81 posts

Re: Incremental Builds in Gatsby Cloud

#21

Post by deltron3030 »

skrebbel wrote: post #14

Having never used a static site generator in anger, can someone explain to me like I'm five what's going on here? My understanding is that Gatsby is a tool that converts a bunch of markdown files into a static HTML website. Why is slow builds a problem for any static site generator? Why does it need a cloud? In other words, what problem am I supposed to be having that any of this solves? Note, I'm trying not to be sk…

> In other words, what problem am I supposed to be having that any of this solves?

It saves time, especially for larger pages, because instead of rebuilding the entire site with all its pages, you just rebuild those that change.

Re: Incremental Builds in Gatsby Cloud

#22

Post by denster »

skrebbel wrote: post #14

Having never used a static site generator in anger, can someone explain to me like I'm five what's going on here? My understanding is that Gatsby is a tool that converts a bunch of markdown files into a static HTML website. Why is slow builds a problem for any static site generator? Why does it need a cloud? In other words, what problem am I supposed to be having that any of this solves? Note, I'm trying not to be sk…

I'll take a stab at this, Kyle just shoot me if I get something wrong below :D

1) There's a server-centric approach and a client-centric approach:

--a) hand-maintained HTML + php falls into the first camp

--b) React (/Angular/Vue) fall into the second

2) If you go with the second camp (b), you end up having a higher initial page load time (due to pulling in the whole "single page app" experience), but a great time transitioning to "other pages" (really just showing different DIVs in the DOM)

3) Gatsby does some very clever things under the hood, to make it so that you get all the benefits of the second camp, without virtually any downsides.

4) There are of course all kinds of clever code-splitting, routing & pre-loading things Gatsby does, but I hope I got the general gist right.

If not, Kyle, get the nerf gun out! -- how would you describe the Gatsby (& static sitegen) benefits? :)

Re: Incremental Builds in Gatsby Cloud

#23

Post by elviswolcott »

skrebbel wrote: post #14

Having never used a static site generator in anger, can someone explain to me like I'm five what's going on here? My understanding is that Gatsby is a tool that converts a bunch of markdown files into a static HTML website. Why is slow builds a problem for any static site generator? Why does it need a cloud? In other words, what problem am I supposed to be having that any of this solves? Note, I'm trying not to be sk…

Gatsby is a fairly complex static site generator. At the highest level, it provides an ingest layer that can take any data sources (CMS, markdown, json, images, or anything that a plugin supports) and bring them into a single centralized GraphQL data source. Pages (which are built using React) can query this graph for the data they need to render. Gatsby then renders the React pages to static HTML and converts the queries to JSON (so there's no actual GraphQL in production).

This process is fairly fast on small/simple sites. Gatsby is overall very efficient and can render out thousands of pages drawing from large data sources rather quickly. The issue is that Gatsby isn't just used for personal blogs. As you can imagine, a site with thousands of pages of content that is processing thousands of images for optimization starts taking a long time to build (and a lot of resources). For example, I'm building a Gatsby site for a photographer than includes 16000+ photos totaling a few hundred GB. Without incremental builds, any change (e.g. fixing a typo) means every single page needs to be rebuilt.

Incremental builds means you don't have to rebuild everything. Because the data is all coming from the GraphQL (which Gatsby pre-processes and converts to static JSON), it is possible to diff the graphs (i.e. determine what data a commit has changed) and determine what pages it affects (i.e. which pages include queries that access that field). From there, Gatsby can only rebuild that changed pages.

This not only means faster build times, it also means that only the changed pages and assets have to be re-pushed to your CDN. This way, content that hasn't changed will remain cached and only modified pages will have to be sent down to your site's users.

Re: Incremental Builds in Gatsby Cloud

#24

Post by freedomben »

skrebbel wrote: post #14

Having never used a static site generator in anger, can someone explain to me like I'm five what's going on here? My understanding is that Gatsby is a tool that converts a bunch of markdown files into a static HTML website. Why is slow builds a problem for any static site generator? Why does it need a cloud? In other words, what problem am I supposed to be having that any of this solves? Note, I'm trying not to be sk…

I've only done simple stuff with Gatsby, but it fully supports generating static HTML from dynamic data sources. The difference between that and traditional JS frameworks is the generation for Gatsby happens at build time instead of runtime.

I love it because I vastly prefer serving static assets to server-side rendering because of the numerous simplicities it provides (aggressive caching, predictable latency, etc). In most cases you get to have the cake of complex sites generated from template and eat the cake of static asset serving.

Re: Incremental Builds in Gatsby Cloud

#25

Post by Touche »

denster wrote: post #22
skrebbel wrote: post #14

Having never used a static site generator in anger, can someone explain to me like I'm five what's going on here? My understanding is that Gatsby is a tool that converts a bunch of markdown files into a static HTML website. Why is slow builds a problem for any static site generator? Why does it need a cloud? In other words, what problem am I supposed to be having that any of this solves? Note, I'm trying not to be sk…

I'll take a stab at this, Kyle just shoot me if I get something wrong below :D 1) There's a server-centric approach and a client-centric approach: --a) hand-maintained HTML + php falls into the first camp --b) React (/Angular/Vue) fall into the second 2) If you go with the second camp (b), you end up having a higher initial page load time (due to pulling in the whole "single page app" experience), but a great time tr…

(3) is incorrect, Gatsby initial page load times are mostly really bad.

(2) is both overstated and overvalued. It's overstated because loading a static HTML page from a CDN is extremely fast. Too many people who point at this advantage for SPAs are thinking back to pre-CDN usage with slow origin servers. Of course there are still use-cases where going to network is not wanted, but these aren't the primary use-cases that Gatsby covers.

It's also overvalued in that most users are not getting to a page by navigating in a loaded site, they are coming from a social or search link (again, for the sort of use-cases that Gatsby pages are built for).

Re: Incremental Builds in Gatsby Cloud

#26

Post by wnoise »

deltron3030 wrote: post #21
skrebbel wrote: post #14

Having never used a static site generator in anger, can someone explain to me like I'm five what's going on here? My understanding is that Gatsby is a tool that converts a bunch of markdown files into a static HTML website. Why is slow builds a problem for any static site generator? Why does it need a cloud? In other words, what problem am I supposed to be having that any of this solves? Note, I'm trying not to be sk…

> In other words, what problem am I supposed to be having that any of this solves? It saves time, especially for larger pages, because instead of rebuilding the entire site with all its pages, you just rebuild those that change.

So does a Makefile.

Re: Incremental Builds in Gatsby Cloud

#27

Post by nogabebop23 »

akiselev wrote: post #17
skrebbel wrote: post #14

Having never used a static site generator in anger, can someone explain to me like I'm five what's going on here? My understanding is that Gatsby is a tool that converts a bunch of markdown files into a static HTML website. Why is slow builds a problem for any static site generator? Why does it need a cloud? In other words, what problem am I supposed to be having that any of this solves? Note, I'm trying not to be sk…

Static site generators output static HTML files instead of running a server that renders every request, it doesn't say anything about what language they consume. Gatsby is built around Javascript and React, not Markdown.

>> Static site generators output static HTML files

This is not really true; they often generate a static client-side web application vs. a dynamic first-time (or every time) app based on server-side processing. This provides a highly optimized, largely self-contained application that avoids a lot of the runtime dependencies and complexity we typically get (ex: web servers and databases). They are still highly dynamic through the use of APIs and such.

Gatsby has an extensive build pipeline and can query almost any data source during the build, but the original base source is markdown, and react is the Javascript.

Re: Incremental Builds in Gatsby Cloud

#28

Post by tvanantwerp »

skrebbel wrote: post #14

Having never used a static site generator in anger, can someone explain to me like I'm five what's going on here? My understanding is that Gatsby is a tool that converts a bunch of markdown files into a static HTML website. Why is slow builds a problem for any static site generator? Why does it need a cloud? In other words, what problem am I supposed to be having that any of this solves? Note, I'm trying not to be sk…

For a hundred markdown files, no big deal. But if your site has tens of thousands of pages, those build times become a real pain point. Why should every single page rebuild if you only changed one of them?

Re: Incremental Builds in Gatsby Cloud

#29

Post by denster »

Touche wrote: post #25
denster wrote: post #22

Earlier quoted context omitted.

I'll take a stab at this, Kyle just shoot me if I get something wrong below :D 1) There's a server-centric approach and a client-centric approach: --a) hand-maintained HTML + php falls into the first camp --b) React (/Angular/Vue) fall into the second 2) If you go with the second camp (b), you end up having a higher initial page load time (due to pulling in the whole "single page app" experience), but a great time tr…

(3) is incorrect, Gatsby initial page load times are mostly really bad. (2) is both overstated and overvalued. It's overstated because loading a static HTML page from a CDN is extremely fast. Too many people who point at this advantage for SPAs are thinking back to pre-CDN usage with slow origin servers. Of course there are still use-cases where going to network is not wanted, but these aren't the primary use-cases t…

So, for (2) above, not sure I understand:

camp 1) at best, a TCP connection is re-used, and the HTML for "page 2" is fetched over the network, parsed, the CSS OM is applied, and then the whole caboodle* is "painted on screen".

camp 2) the CSS OM is applied and "page 2" is painted on-screen (possibly even faster if the browser cached "page 2" in a texture on the GPU, so the CSS OM application step may be optimized away)

So I genuinely don't understand how fetching a "page 2" from a CDN

(we use Cloudfront & GCP's CDN at https://mintdata.com, so I'm basing my experience on this)

is faster than the SPA approach?

I am genuinely curious on the above -- not trying to start a flame war :D

* Yes, apparently caboodle is a word?! I had to Google it just like you to make sure :)

Post reply on HN