Are static site generators only for small sites? E.g. if there is a site with 100,000 pages and the site owner wants to change something in the template (adding something to the sidebar, or footer, etc.) then does he have to regenerate and redeploy all the 100,000 pages every time when he makes a global change?
It may require regenerating all pages, but depending on the generator that may not be particularly expensive - they may cache partial assets, so you replace part of the build asset and then the build is basically a bunch of cat-like calls, which tend to be extremely cheap.
The largest static site I've managed was only a couple thousand pages (2138 pages from the last build log), but the build time was around 8 seconds (8.1 from the last build log) with cached build assets.
I'd be happy using a static generator for most sites.
> It may require regenerating all pages, but depending on the generator that may not be particularly expensive - they may cache partial assets, so you replace part of the build asset and then the build is basically a bunch of cat-like calls, which tend to be extremely cheap.
I've been thinking about this, and you can only do quick dependecy checks if the input is in files, right? So if the actual data is in a database then you do have to have some process to dump the changed data into files from the database, so the build system can pick them up.
Otherwise, if the build system takes data directly from the db then it doesn't know if the data is changed, so it has to retrieve everything from database which won't be efficient.
> I've been thinking about this, and you can only do quick dependecy checks if the input is in files, right? So if the actual data is in a database then you do have to have some process to dump the changed data into files from the database, so the build system can pick them up.
You can run diffs across in-database data as well, so that you only need to make the exact changes you want. (You could use something like SQLite's sqldiff, or Python's difflib, etc.)
You may only have to upload a _diff_ of the changes. And as global changes tend to be similar on every page, that diff may be extremely small for quite a lot of difference.
Yes changing the global template would involve regenerating all 100,000 pages in most static site generators. The only exception being those rare ones which offload template rendering to client side (there's a couple that I've seen). Still, some static site generators like Hugo, are built for bigger sites specifically by being able to generate massive amounts of pages very quickly. Even in your 100,000 page example a site probably would be completely rebuilt within a second or two.
Depends how it's setup. I remember reading something one time about how you can set it up where each individual build step is cached which would cut down on a lot of build time. So for instance, in this case it would only remind the sidebar.
I can not answer for large sites (nor for makesite.py specifically), but in general to your scenario, yes, all pages would need to be regenerated. I guess you could think of it as a build server. But instead of building code for an app that you would be deploying, you are generating your content for your static site/blog. That being said, you could then create a build server optimized to generate your content, and/or try a static site generator whose forte is the build/generation process (I think hugo fans tend to highlight this feature for that platform?). Maybe instead of generating all content on your local machine, you could keep all your content on a beefier machine, and generate it from there - which could process stuff much faster, etc. Even still, it is probably unlikely that one would have to alter things like sidebars or footers that often, right? Also, worse case scenario, something tells me that even if needing to regenerate that many pages, i mean, would it be ok if it takes just a few minutes - probably not the end of the world, right? (I guess it depends on the timeliness of the single post or footer change that needs to wait for 100,000 pages to get regenerated, eh?) I hope my comments helped a little.
I would guess that depends on whether the change affects the HTML. If it's only a CSS change, you could probably get away with redeploying the CSS file.
Based on our experience (n=1) I would say, yes, it works best for small sites. We have a medium site with on the order of 1000s of pages. Jekyll has become a bit unwieldy to rebuild a full static version of the site (takes about 5 mins for a full rebuild). I would certainly say an order of magnitude higher we’d definitely want server side Wordpress or similar.
5 minute+ rebuilds are enough to move you to WordPress? 5 minute rebuilds does seem high but the benefits you get with static site generators (security, push “anywhere”, performance, etc) far outweigh having to manage everything else coming with a wordpress site. I suppose if you and your team are pushing new articles multiple times a day the builds could become annoying but even then I’d build/deploy and go grab a cup of joe; come back all done
There’s more to it than that. It’s also that our marketing team can understand and work with Wordpress. Static site generators are still the domain of “developer websites”
Taking 5 minutes to build frankly is more a reason our developers give in and say “meh whatever the marketing team wants, just point me at a place to blog”.
Hm, I switched from Jekyll to Hugo years ago and don't remember having to re-work the templates all that much but you're right, that may have been only easy because I never deviated much from the default. If you've done a lot of customization then you may have to do that all over again.
It depends on how you do it but generally speaking, the break even point is “will I be generating pages faster than they are accessed?” Eg if I have a site like Wired.com with a long tail of old articles, it might be that by generating it, I refresh old articles 500x daily for the 500 new story creation and edits, even though the average old article gets less than one view per day.
