Cool, but I'm so tired of Python that I've started to avoid any projects written with it. Dealing with dependencies, 2 vs. 3, pip, virtualenv, etc is such a pain.
I think the Java/Go/Rust model where all dependencies are included in the binary is the way to go.
You raise a fair general point about Python dependencies, but that issue is usually related to poorly considered use of third-party dependencies or the need to distribute binaries and leverage C bindings.
But here we have exactly the kind of project that should be written in Python, forked and cloned by you, then tweaked and maintained as if you wrote it yourself. From a brief glance through the code, and taking the submission title at face value, this project is a refreshingly appropriate use of the right language. I offer this as someone who writes Python and Go, generally preferring Go anywhere possible.
I plan to poke around some more in this repo to see if I can easily replace my use of Hugo with this, since I use only a tiny fraction of the features in Hugo and have found it challenging to hack on it as a low-priority weekend project.
For turning personal blog posts from markdown to HTML with some general convenience functions around maintaining the front-matter and templates, I can’t think of something more appropriate than a Python script sitting inside your blog directory with all of its dependencies except the interpreter itself checked in with your templates and content.
I won't bother with Python again because these conflicts happen on your first time even trying out the language. (running linux, any better on Windows?)
It was such a mess that I can't even describe the frustration. And I even got Apache working with SVN! (40 pages of docs and no testing points from start till it's fully working, yuck)
EDIT: Since I got downvotes so quick, tell me why anyone should choose Python over the many alternative out now?
Sorry you went through the trouble of what is called "setting up a development environment" and yes, I wish you could simply start coding and distributing programs as soon as you boot up a computer, but that is simply not the case anymore. Not with python, not with ruby and not with go.
There’s too much to untangle for a brief explanation, but if this really happened when you tried writing Python on Linux for the first time, or using a program written in Python you installed through your distribution’s package manager, file a bug report with your distribution.
It wasn't "bugs" it was all the hoops to jump through, conflicting hoops, multiple installed versions, pip vs all the other package installers, that also caused conflicts, etc...
Since I am not a Python dev, I would be unable to provide a useful report even if it were bugs.
In my opinion, the worst problem is the lack of a standard way to handle dependencies. I recently had a hard time deploying a small Python project over a few heterogeneous Linux machines.
For example, this "simple, lightweight" program has dependencies, as one can see in its .travis.yaml file. That's why the readme has the command `pip install commonmark` (plus more dependencies for dev). But, wait, doesn't `pip` install as root in /usr/local/ by default? (BTW, the documentation of `pip install` is mute on the subject.) So I should use pipenv, but last time I tried it was broken and the ticket had low activity. So virtualenv + pip, but it's a mess since I like using many terminals. I've heard about poetry, but that's not standard at all...
Why is virtualenv a mess with multiple terminals ? pip install for root unless you activate a virtualenv in which case it installs in it. If you make an alias of . env/bin/activate it is very simple, no ?
Python deployment is a shitshow. Unfortunately mission critical stuff (looking at you letsencrypt) is also written in it which means that your systems are built on very shaky foundations.
Well in ten years I have never had a nuked server by pip or virtualenv. I am sure there are edge cases but that must be true for all the ways of deploying stuff. I guess the ones you know well appear simple to you !
I kind of agree with both of you. I don't think Python deployment is that bad, as most modern managers get it right (I prefer Poetry nowadays). However something like letsencrypt-auto, which AFAIK installs its own dependencies, is not something I'd run. I've actually never used it precisely because it's too magical. Programs shouldn't install their own dependencies anywhere outside of a virtualenv.
Does certbot do the same ? I don’t use either so I can’t tell.
I strongly agree that any python program must install its dependencies in a virtualenv. One could even argue that this principle could be extended to any software !
People who know what they're doing can build a lot of things with python. It just so happens that I am not one of those people so I try to avoid it with a ten foot stick. I still use things built with python everyday.
I heard about dehydrated from PythonAnywhere [1]. I haven't ever been burned by LE from a Python library, but I have had issues in the past with the kube-lego Helm chart not renewing as it should (easily fixed by deleting the pod though). [I know kube-lego is in maintenance mode now in favor of cert-manager.]
It’s actually not as bad as I thought it would be. I modified one of the twenty-x themes that come with it and installed a few plugins to make a decent personal site. It took less time that it would have to create anything with remotely similar functionality.
I run it off a single vps that has no other purpose. If it gets hacked, nothing of value is lost. I can revoke the api keys for the external services I use and restore to a new server from my backups.
WordPress used to be quite vulnerable a few years ago, but nowadays not so much, just avoid less popular plugins and you should be ok. Of course since it's the most widely used cms's out there, its a prime target for hackers. But the "WordPress is vulnerable" thing is much of a meme in the current day.
For example, making sure that software cannot modify itself will ensure that even if server is vulnerable, it won’t get permanently compromised. Having a server that does not execute any file with right extension makes sure that sanitization errors do not lead to code execution. Having admin system be separate from main site makes sure XSS cannot cause compromises.
with a default wp install - you are right,
however it's trivial to add a couple of security plugins and turn on auto-update.
adding the "ipgeoblock plugin" wipes out most attacks straight away.
with some of my wp sites that got attacked a lot on a regular basis, I use a 'static html generator' plugin - and delete all the wp php files -
no way to login, add comments or hack the wp core or plugins or themes, since they are not in use when you convert it all the static html.
On wp sites where I actually add content with regularity, I don't delete the wp files, and just use shield, sucuri, ipgeoblock, plugin things like that depending on threat.
> I think the Java/Go/Rust model where all dependencies are included in the binary is the way to go.
This is how triple scripts are designed[1] to work.
One proof-of-concept just so happens to be a minimalist static site generator called triickl[2]. You can drop it onto (virtually) any machine, double click, and point it at a source directory for your site—subject to the expectation that it follows a certain jekyll-like structure—and then you're off to the races. I'm using it as the build step for the triplescripts.org home page itself and my personal site.
Are static site generators only for small sites? E.g. if there is a site with 100,000 pages and the site owner wants to change something in the template (adding something to the sidebar, or footer, etc.) then does he have to regenerate and redeploy all the 100,000 pages every time when he makes a global change?
It may require regenerating all pages, but depending on the generator that may not be particularly expensive - they may cache partial assets, so you replace part of the build asset and then the build is basically a bunch of cat-like calls, which tend to be extremely cheap.
The largest static site I've managed was only a couple thousand pages (2138 pages from the last build log), but the build time was around 8 seconds (8.1 from the last build log) with cached build assets.
I'd be happy using a static generator for most sites.
> It may require regenerating all pages, but depending on the generator that may not be particularly expensive - they may cache partial assets, so you replace part of the build asset and then the build is basically a bunch of cat-like calls, which tend to be extremely cheap.
I've been thinking about this, and you can only do quick dependecy checks if the input is in files, right? So if the actual data is in a database then you do have to have some process to dump the changed data into files from the database, so the build system can pick them up.
Otherwise, if the build system takes data directly from the db then it doesn't know if the data is changed, so it has to retrieve everything from database which won't be efficient.
> I've been thinking about this, and you can only do quick dependecy checks if the input is in files, right? So if the actual data is in a database then you do have to have some process to dump the changed data into files from the database, so the build system can pick them up.
You can run diffs across in-database data as well, so that you only need to make the exact changes you want. (You could use something like SQLite's sqldiff, or Python's difflib, etc.)
You may only have to upload a _diff_ of the changes. And as global changes tend to be similar on every page, that diff may be extremely small for quite a lot of difference.
Yes changing the global template would involve regenerating all 100,000 pages in most static site generators. The only exception being those rare ones which offload template rendering to client side (there's a couple that I've seen). Still, some static site generators like Hugo, are built for bigger sites specifically by being able to generate massive amounts of pages very quickly. Even in your 100,000 page example a site probably would be completely rebuilt within a second or two.
Depends how it's setup. I remember reading something one time about how you can set it up where each individual build step is cached which would cut down on a lot of build time. So for instance, in this case it would only remind the sidebar.
I can not answer for large sites (nor for makesite.py specifically), but in general to your scenario, yes, all pages would need to be regenerated. I guess you could think of it as a build server. But instead of building code for an app that you would be deploying, you are generating your content for your static site/blog. That being said, you could then create a build server optimized to generate your content, and/or try a static site generator whose forte is the build/generation process (I think hugo fans tend to highlight this feature for that platform?). Maybe instead of generating all content on your local machine, you could keep all your content on a beefier machine, and generate it from there - which could process stuff much faster, etc. Even still, it is probably unlikely that one would have to alter things like sidebars or footers that often, right? Also, worse case scenario, something tells me that even if needing to regenerate that many pages, i mean, would it be ok if it takes just a few minutes - probably not the end of the world, right? (I guess it depends on the timeliness of the single post or footer change that needs to wait for 100,000 pages to get regenerated, eh?) I hope my comments helped a little.
I would guess that depends on whether the change affects the HTML. If it's only a CSS change, you could probably get away with redeploying the CSS file.
Based on our experience (n=1) I would say, yes, it works best for small sites. We have a medium site with on the order of 1000s of pages. Jekyll has become a bit unwieldy to rebuild a full static version of the site (takes about 5 mins for a full rebuild). I would certainly say an order of magnitude higher we’d definitely want server side Wordpress or similar.
5 minute+ rebuilds are enough to move you to WordPress? 5 minute rebuilds does seem high but the benefits you get with static site generators (security, push “anywhere”, performance, etc) far outweigh having to manage everything else coming with a wordpress site. I suppose if you and your team are pushing new articles multiple times a day the builds could become annoying but even then I’d build/deploy and go grab a cup of joe; come back all done
There’s more to it than that. It’s also that our marketing team can understand and work with Wordpress. Static site generators are still the domain of “developer websites”
Taking 5 minutes to build frankly is more a reason our developers give in and say “meh whatever the marketing team wants, just point me at a place to blog”.
Hm, I switched from Jekyll to Hugo years ago and don't remember having to re-work the templates all that much but you're right, that may have been only easy because I never deviated much from the default. If you've done a lot of customization then you may have to do that all over again.
It depends on how you do it but generally speaking, the break even point is “will I be generating pages faster than they are accessed?” Eg if I have a site like Wired.com with a long tail of old articles, it might be that by generating it, I refresh old articles 500x daily for the 500 new story creation and edits, even though the average old article gets less than one view per day.
Good job, but why would I use this when it takes less code and less time than it takes to read Makesite.py source code, to build my own with Jinja2 [1], PyQuery [2], markdown2 [3]?
Python is so easy and powerful there's no need for a static site generator.
I don't want to bash or put down anyone, but frankly I don't see any point with Makesite.py. But, why do we have a toy project that openly admits that it reinvents the wheel on the first page of HN?
Makesite.py is "sold" as an alternative to Jekyll for people who prefer Python. Yet, it completely fails to mention Pelican [1]. Why should I bother with this instead of going with the featureful and well-tested Pelican?
I don't use Makesite.py but if you see the network graph [1], obviously there is an audience for this kind of tool. I see many more results of people using and modifying makesite.py here [2].
I think the appeal of a tool like Makesite.py is that you can make it what you want [3]. It is about a 100 lines of code and written very neatly. Instead of wading through documentation and configuration to figure how to get Jekyll or Pelican running, you can make your own Jekyll or Pelican. I can see why many individualist programmers would like such a thing.
> Makesite.py is "sold" as an alternative to Jekyll for people who prefer Python.
That's a really strange argument to make. If you see the README, it mentions, "But then did you yearn to use something even simpler to generate your blog? Do you like Python? Perhaps the thought of writing your own static site generator crossed your mind but you thought it would be too much work? If you answered "yes" to these questions, then this project is for you."
Makesite.py is being presented as a do-it-yourself blogging solution with makesite.py serving as a good starting point to start hacking. It is very different from Jekyll. How does it make sense to mention Pelican which is, if anything, similar to Jekyll and not similar to Makesite.py?
In fact I have written more than one DIY static site generators for small sites in the distant past. The bottom line of my experience was: they don't worth the effort. You will start missing features sooner than you think, and it will be entirely up to you to implement them from scratch. Instead of actually adding content to your site, you'll end up adding features to your feature-strapped hack.
In the presence of excellent tools like Jekyll or Pelican that are also fairly easy to use and extend, I'd need a concrete use-case before going DIY. If there is no concrete use-case, the DIY static generators is just another version of the "not invented here" syndrome. Just my 2¢.
There is no one-size-fits-all solution. Some people prefer full-blown feature-rich static site generators like Jekyll or Pelican or Hugo. Some people like to write their own minimal static site generator. To each their own.
I think the Java/Go/Rust model where all dependencies are included in the binary is the way to go.