Live data from Hacker News

Incremental Builds in Gatsby Cloud

gatsbyjs.org

51–60 of 81 posts

Re: Incremental Builds in Gatsby Cloud

#51

Javascript re-invents "code typing" (Typescript) Javascript re-invents "Promises" because callback hell Javascript re-invents "compilers" (babel) Javascript re-invents "build systems" (webpack, etc) Javascript re-invents "caching" (incremental builds) - but paid, and in the cloud Because why not.

/s/re-invents/implements/g and suddenly JavaScript is running a successful dev cycle.

Re: Incremental Builds in Gatsby Cloud

#52
post #26

Earlier quoted context omitted.

> In other words, what problem am I supposed to be having that any of this solves? It saves time, especially for larger pages, because instead of rebuilding the entire site with all its pages, you just rebuild those that change.

So does a Makefile.

A makefile works great when you have a pile of source files and you want to make a parallel pile of output files, and each source file is compiled individually. It's not so great when you have a compilation process where you have a folder of entrypoint source files that each need their own output artifacts produced but they happen to share many dependencies, and you want to automatically create common output chunks if there's enough overlap between them, etc. I'm sure you could find a way to involve Make by automatically generating makefiles, but at that point, Make is only handling the really easy part and isn't worth it.

Think of how many makefiles just end with one big linker call. Most web toolchains (which crunch a lot of source files into a few artifacts) have more in common with that linker call than the rest of the stuff that happens in a makefile. You have to have a system that's more integrated with what's being built to make that step meaningfully incremental.

Re: Incremental Builds in Gatsby Cloud

#53
How much is the speed issue related to the language used? I know Hugo is an order of magnitude faster than most static site generators for example - it's written in Go with e.g. 2 seconds to generate about 10K pages https://forestry.io/blog/hugo-vs-jekyll-benchmark/.

I would have thought the generation process could be massively parallelised and a typical blog page would only need a modest amount of computation e.g. concat header, footer, pull in body text, resolve a few URLs. I can't help but think about how much work a typical computer game is doing in comparison 60 times per second even without a GPU.

Re: Incremental Builds in Gatsby Cloud

#54

How much is the speed issue related to the language used? I know Hugo is an order of magnitude faster than most static site generators for example - it's written in Go with e.g. 2 seconds to generate about 10K pages https://forestry.io/blog/hugo-vs-jekyll-benchmark/ . I would have thought the generation process could be massively parallelised and a typical blog page would only need a modest amount of computation e.g.…

I don’t think it’s a language issue. Even for JavaScript bundlers you have the slow extensible bundle and the “new super fast bundler” that dies in a month because it only fits one use case.

How flexible is Hugo? And how many plugins does someone generally use?

Re: Incremental Builds in Gatsby Cloud

#55

How much is the speed issue related to the language used? I know Hugo is an order of magnitude faster than most static site generators for example - it's written in Go with e.g. 2 seconds to generate about 10K pages https://forestry.io/blog/hugo-vs-jekyll-benchmark/ . I would have thought the generation process could be massively parallelised and a typical blog page would only need a modest amount of computation e.g.…

Some of it is due to the language and the general JS tooling being bloated and slow.

However, in many cases build time is slow because you're doing something that's slow, like calling a REST API. You are not going to generate 10k pages in 2sec if you need to make 10k REST requests, each taking 100ms, to a remote API to fetch the data for your pages. This kind of "data integration" from various sources is a standard use cases for site generators like gatsby and next.js. It seems like what this is targeting is smarter caching to avoid such expensive calls when possible.

Hugo is different in that it basically just transforms local HTML/Templates/Markdown. That's always fast. Even JS can handle that.

Re: Incremental Builds in Gatsby Cloud

#56

How much is the speed issue related to the language used? I know Hugo is an order of magnitude faster than most static site generators for example - it's written in Go with e.g. 2 seconds to generate about 10K pages https://forestry.io/blog/hugo-vs-jekyll-benchmark/ . I would have thought the generation process could be massively parallelised and a typical blog page would only need a modest amount of computation e.g.…

Some of it is due to the language and the general JS tooling being bloated and slow. However, in many cases build time is slow because you're doing something that's slow, like calling a REST API. You are not going to generate 10k pages in 2sec if you need to make 10k REST requests, each taking 100ms, to a remote API to fetch the data for your pages. This kind of "data integration" from various sources is a standard u…

> Hugo is different in that it basically just transforms local HTML/Templates/Markdown. That's always fast. Even JS can handle that.

Do you know of any benchmarks that show that? As far as I know, most static site generators can take minutes to process a few thousand Markdown files with Hugo being the exception.

Re: Incremental Builds in Gatsby Cloud

#57

How much is the speed issue related to the language used? I know Hugo is an order of magnitude faster than most static site generators for example - it's written in Go with e.g. 2 seconds to generate about 10K pages https://forestry.io/blog/hugo-vs-jekyll-benchmark/ . I would have thought the generation process could be massively parallelised and a typical blog page would only need a modest amount of computation e.g.…

I don’t think it’s a language issue. Even for JavaScript bundlers you have the slow extensible bundle and the “new super fast bundler” that dies in a month because it only fits one use case. How flexible is Hugo? And how many plugins does someone generally use?

> How flexible is Hugo? And how many plugins does someone generally use?

It processes Markdown, JSON, YAML and SASS, can pull in data files from URLs, and has custom templates/themes, custom macros/shortcuts, image processing and live reload. It doesn't have a plugin system as far as I know but nothing stops you combining Hugo with other tools e.g. run a JS script to pull in and transform a JSON file before Hugo runs.

Re: Incremental Builds in Gatsby Cloud

#58

Earlier quoted context omitted.

Gatsby is a fairly complex static site generator. At the highest level, it provides an ingest layer that can take any data sources (CMS, markdown, json, images, or anything that a plugin supports) and bring them into a single centralized GraphQL data source. Pages (which are built using React) can query this graph for the data they need to render. Gatsby then renders the React pages to static HTML and converts the qu…

Hi, have you thought about hosting for this yet? I've got a similar site which I originally tried on AWS Amplify but it got too big for the artifact size limit so I opted for S3/Cloudflare instead however build times are slow and more of a manual process currently.

I’m yet to figure that out! I’m procrastinating on that but until I have everything else figured out (it’s for a family member so no strict timeline). I’m thinking in the end the setup will be something with a CMS for editing the photo metadata, a file storage system for the images, and everything else in git. The build would pull from all 3, run and then the processed images will be pulled out and hosted on their own. I’m planning that it’ll just run and take a long time on a droplet unless I figure something better out.

Re: Incremental Builds in Gatsby Cloud

#59

Having never used a static site generator in anger, can someone explain to me like I'm five what's going on here? My understanding is that Gatsby is a tool that converts a bunch of markdown files into a static HTML website. Why is slow builds a problem for any static site generator? Why does it need a cloud? In other words, what problem am I supposed to be having that any of this solves? Note, I'm trying not to be sk…

Gatsby is a fairly complex static site generator. At the highest level, it provides an ingest layer that can take any data sources (CMS, markdown, json, images, or anything that a plugin supports) and bring them into a single centralized GraphQL data source. Pages (which are built using React) can query this graph for the data they need to render. Gatsby then renders the React pages to static HTML and converts the qu…

But if you have 16,000 of anything, why are you using a static site? Surely the access patterns are long tail and you need to build more often than most pages are even accessed.

Re: Incremental Builds in Gatsby Cloud

#60

Having never used a static site generator in anger, can someone explain to me like I'm five what's going on here? My understanding is that Gatsby is a tool that converts a bunch of markdown files into a static HTML website. Why is slow builds a problem for any static site generator? Why does it need a cloud? In other words, what problem am I supposed to be having that any of this solves? Note, I'm trying not to be sk…

Server-side rendering (like Wordpress) generates HTML in response to a URL. Static site generators just visit every possible URL at build time and save the final HTML as files. This makes it easy to deploy and scale when your site is static and doesn't need any features of dynamic server-side rendering. Gatsby (and other frameworks) automate this process by going through whatever data sources you have (directory of m…

Wordpress in particular is not “just fine”. During this crisis, every government site based on WordPress ends up crashing under the load. Yes, you can avoid this with a good cache plugin and a CDN. But you can also avoid it by using a tool that is designed to not crash under load in he first place.
Post reply on HN