Over-And-Under Static WordPress

Proponents of GraphQL interfaces are quick to point out that this approach allows you to easily use WordPress as a headless Content Management System (CMS) with a static site serving as your front end.

They’re not wrong. And as a developer, using the WPGraphQL interface is a lovely experience. With this approach, you create something like a Gatsbyjs site and the generator for that site sucks up all your blog entries and pages from the WordPress instance and spews out HTML pages, as all Static Site Generators do. All these blog pages display lightning fast and give you a blazing static WordPress site, but right out of the gate you’ve lost at least two functions from your old WordPress: comments and search.

In many cases—particularly pages that serve marketing goals—you don’t want the comment system turned on anyway. That said, there still may be situations (actual blog entries, possibly) where comments are a good idea. And while search is less important on, say, a landing page, in general it’s hard to see why you’d willingly give up support for search for a site as a whole.

The “standard” way to return comment, search, and other dynamic functionality to a static site is to use third-party services that offer APIs. So, maybe Algolia for search or Disqus for comments. There are upsides and downsides to this approach but I won’t belabor them in this piece.

WordPress Under, Static Over (WuSo?)

With all this in mind, I set out to create a site that’s static on the front end, but that understands when something is supposed to be dynamic and links back into the WordPress site in order to get the dynamic services WordPress normally handles. During the handling of the request (in real time for the user), a new static version of the page is updated to the static version of the site as needed (if, for instance, a new comment has been added—the static page will now show that new comment).

Initial Caveats

When I first started experimenting with this, I was impressed by how simple it seemed. If you have a static page and you leave the standard WordPress comment section on it, but simply change the link that the form sends its GET request to back to the original WordPress site, it’s almost frighteningly seamless (at least it is if the static page looks exactly like the WordPress page—more on this later).

It wasn’t actually quite as seamless as I initially thought. Or at least not as easy to make really work.

So, yeah: getting the new static page updated with a new comment requires sorting out a lot more details than you might initially expect. Additionally, while the update of the comment itself isn’t susceptible to race conditions caused by other comments arriving at the same time, some approaches to updating the static page open a small window of chance that one update will get temporarily overwritten by another one (the way I’m currently going about this, though, seems to avoid this problem).

One more important caveat: a significant security boost can be had by taking your WordPress site offline whenever you’re not updating the content. In this scenario, which is the one that current plugins like WP2Static and SimplyStatic provide, only the static site is available to would-be attackers, and it’s hard (though not entirely impossible) to successfully compromise a static site.

There are a couple of service providers—Strattic and HardyPress—that run your WordPress instance in a virtual container (I think they both use Docker containers) that is spun up only when administrators log in through the service’s front end. Otherwise, the dynamic WordPress site is spun down and simply isn’t there to attack. It’s ingenious, from a security point of view, and both vendors have added functionality to support search and comments. In other words, these guys already do the basics of what I set out to do, and fairly affordably too.

I have some further destinations in mind, though, about which more in subsequent posts. So I needed control of the underlying code and the ability to release plugins for some of the functionality.

WordPress Under

In an approach where dynamic components are being handled by a live WordPress site, that site obviously has to be up and running somewhere. Exactly where that is can vary, to be sure, and putting it somewhere hard for random scans run by hackers to find could theoretically lower your attack profile, but of course the rewritten links in your static code, pointing back to the WordPress instance, make it readily findable to anyone who understands even basic HTML. (Could this be made more bulletproof? Maybe. Possibly one has an edge handler of the sort that Netlify offers to static site creators and it serves as an intermediary between static user and WordPress instance… Something to think about.)

All that said, if you’ve been running your site on WordPress up to this point and been basically OK with your security posture, you’re no worse off for having that same site still running behind the static site. And you’re arguably a little better off, because attackers who are using scanners to look at your domain for signs that you are running WordPress may not find them, since your installation is running a bit behind the scenes and doesn’t have quite the directory tree that a WordPress-only site would have.

The Over/Under Setup

As I noted, you could put the WordPress installation pretty much anywhere, but to keep myself sane while running a bunch of these setups, I decided to have the static site running at the “top” of a domain (where you’d normally find your WordPress entry point, index.php) and then have the entire WordPress site moved to a subdirectory within that top level. I tend to think of this as having the WordPress site in a directory that’s “under” the static site. At first, I literally called this folder “under,” and inside of it you’d find the usual WordPress directories: wp-admin, wp-contents, and wp-includes. More recently, I’ve changed the “under” directory to names that are randomly generated, password-like strings, things like “df2CvIE9etuVJX4snt7N” so that automated hacking tools would be less likely to stumble into it (though, again, all the links from static to dynamic pages include this subdirectory name and someone expressly looking for it can readily find it).

Creating the Static Site

As I’ve already hinted at, there are a couple of easy ways to create a static version of your WordPress site if you don’t want to jump to a full-blown static site generator (SSG) like Gatsby or Next.js. These come in the form of WordPress plugins: SimplyStatic, WP2Static and Export WP Page to Static HTML/CSS.

I’m going to write in more detail about these in another article, but here are a few salient facts.

SimplyStatic doesn’t work on Windows platforms right now, which is a problem more for local development than for most production sites, which tend to be running on a Linux variant. It works very cleanly in my experience and you can point the output directly at the “over” root directory, which saves some extra fiddling.

It’s pretty fast at the task, even though it (as well as WP2Static) only does full builds (almost certainly a deal killer for really large sites).

It creates a new directory for each file it encounters as it spiders through your site. Inside that folder is an index.html and a subfolder with all the assets required to display the file.

If you’re looking to display a few pages, this is perfectly fine. For a whole site, you wind up with all sorts of duplicated files, all of which are also sitting in subfolders within the underlying WordPress site. So while I would generally give SimplyStatic high marks, the asset folder-palooza isn’t ideal for our purposes.

Having a separate folder for each page is the easiest way to have URLs match what you’ve got in your WordPress site, so it makes sense that plugins would go this route and, indeed, it’s what I currently do. This is another thing I’d like to look at further, however, when other more pressing bugs and issues are ironed out.

Then, there’s WP2Static, brainchild of Leon Stafford, who seems to just live and breathe the open-source approach. I don’t have a lot to say about it, because it does exactly what you think it does. It’s the incumbent option for this sort of plugin.

There’s also Export WP Page to Static HTML/CSS, by ReCorp, which has a free version that only lets you process a list of files, and there are limits on even this. If you buy a license, you can export as long a list as you like, plus you can instead export the whole site. I tried both versions and both work, even on Windows, but the pro version is licensed via Freemius, which I find a pretty unpalatable approach.

And there’s a new kid on the block that I just ran across and haven’t looked at yet, called WP2HTML. The static world is a busy place these days, I guess.

With any of these plugins, you have two problems left unaddressed. First, you have copies of all the image and other assets. This makes sense for their main use case (that is, making a static site that is fully independent of the WordPress site), but is total overkill for us, because we could just as easily bring in our assets from WordPress.

Second, neither approach saves you from the sometimes breathtaking bloat in CSS and JS files within WordPress themes. The static version will be static, but it will still load in all the junk it loaded in with your big, fancy Divi-based WordPress theme. It’s possible that your Google Pagespeed scores won’t improve a whit (though, for the most part, you’ll still see improvements, especially on your mobile scores).

It appears to me, from my preliminary soundings of the tangle of code that is Export WP Page to Static HTML/CSS, that it does make some effort to cull out unused CSS code, but honestly it looks like it doesn’t actually cull out much. This is reasonable insofar as it takes a fair bit of logic to figure out, site-wide, what can actually be cut. Nothing I’ve looked at so far looks at the site in toto—they work entirely on a page-by-page basis.

What one really wants is a plugin that doesn’t change URLs that don’t need to be changed. If you think about it, actually, most URLs could continue to point wherever they already pointed. In fact, they all could, with the exception of links to other static pages.

And all we really need for the static version of the site is the HTML pages. So what we want is sort of like what SimplyStatic and WP2Static spit out, but just HTML files. For each of the HTML files, the links that are “staying the same” actually need to be adjusted to point down to the “under” directory.

All right, I concede that what I just described in no way fixes the style and JavaScript file bloat issue. But there are ways to tackle this and I’ll talk about my approach in upcoming posts.

How to Get a Static Page

Both WP2Static and SimplyStatic do what you might expect in order to get all the pages in a site: they start at the index page, look through all the links on the page, determine which ones are in the same domain, then recursively look at the pages on the ends of each of those links.

Both of them then cleverly use Ajax requests to have the pages at URLs returned to them. It’s smart and I may well use the approach in a later iteration of this over/under thing. But for now I went a different way, an inherently incremental one.

The Output Buffer

PHP has this wonderful and weird couple of functions that access a built-in buffer construct. It’s exactly the sort of thing you might imagine would get created in a language designed almost exclusively to create web pages on the fly.

With the output buffer, you can turn it on before WordPress would normally have written anything, then capture everything that WordPress writes into the buffer. Before you let go of the captured page, you can make whatever substitutions you’d like.

This is where things get interesting. You’re going to return the “native WordPress” version of this file, except that all the links to HTML files within the domain are going to be rewritten to point up to the top level where the static site lives. But we’re also going to write a version of the file up to the static level, overwriting the one that’s already up there with a more current version, and in that version all the links that aren’t links to HTML pages will be pointed “down” into the “under” directory.

Search requests are handled differently, in that no permanent static page is created. Instead, the regular WordPress page is created, but all links to HTML pages on the site are rerouted up to the static version of the site.

It probably goes without saying, but every URL that contains “wp-admin” (that is, every page in the admin console) is left to run dynamically on the site.

Still Clumsy

It’s a pretty simple setup and, if you choose a “sane” theme for your WordPress site, it’ll be fast, and in fact your Pagespeed scores will probably improve on the static site. I’ve run against three popular free themes as I write this and they all come out blinding fast and kind of “just worked.”

Crazily enough, with this approach many of the plugins you might be using will still work (though of course plenty of them won’t). Plugins you use to add features like sliders—things that are inserted into the page as it is created—will still be inserted and will wind up in the static versions of the pages. If they rely on JavaScript functions to, say, make the slider work, they’ll include .JS files that will be included (with a rerouted URL) in the static version.

For the most part, things like form builder plugins still work. You build the forms within the admin console as always, then the forms are inserted into the pages as they are being dished out for conversion to static versions.

One regrettable thing that does get broken in an important way is user management. You can login through the back end, as per usual, and any pages you access on the dynamic WordPress site will work as always, but if you have front-end controls on what users or non-users can’t see, the static versions of these pages will be flummoxed. The best approach to fixing this is something I’m still looking into, but I suspect it boils down to writing static-page calls into the WordPress REST API.

The Current State of Play

I’ve got running code at this point, but there are lots of gotchas that require tweaks and rethinking of algorithms. But all the same I have a site running (well, most of the time) at PeakZebra.link (not .com, which is where you are right now). It gets blown away and reinstalled, functions added and deleted, and themes swapped out pretty frequently, so I’m by no means suggestion you run over and see what you think of it, because it’s entirely possible is that you’ll think it’s broken.

On the other hand, it wouldn’t surprise me if I got to a point fairly soon where it becomes usably stable. I’ll come back and revise this entry at that point.

The end game, I should mention, is to produce “better-than-native” static pages, by which I mean pages that look like the WordPress site they come from and that still have WordPress dynamic functions, but that have optimized those pages in various ways.

Readers of PeakZebra already know that I’m pretty impressed with the way Gatsby uses React such that React components can be swapped in and rerendered within static pages at run time, so it’s probably no surprise to learn that I either want to convert the output either to straight-up Gatsby source code or else output static files that use some of the same React tricks.

It’ll be interesting, I promise. More to come.