For the last 2+ years, this blog has been parasitically run as an “app” on a ServerPilot managed Digital Ocean droplet. I had/have plenty of disk and compute bandwidth, and honestly, a “good” day finds maybe 10 page views, and occasionally, I will serve up to 50 people in a day.

Yawn.

But I like having this outlet, where I can let my hair down, and act naturally (that is cynically, and abrasively).

About a month ago, give or take, the platform this is built on, Ghost released a major update from version 3 to version 4, with a lot (and I mean a WHOLE FUCK-TON of content goodness) of improvements.

I waited a few weeks, and tried to do an in place upgrade.

That failed. Badly. Well, not actually. It completed. The system would run, but something happened and Nginx wasn’t happy, and I got a 502 error.

Shit.

Of course, I was flying by the seat of my pants, and hadn’t done a full backup prior to the attempt (my bad) so to get back to a known state, I had to restore from a droplet backup. Fortunately, the hosting solution I use, Digital Ocean, made is stupid easy to restore. I just rolled back to the last backup, 4 days earlier, and no content was lost.

Fast forward a few weeks. The template I use for this site was upgraded to be compliant with Ghost v4.x, and I decided to be smarter, and back up the whole magilla.

But first, I wanted to play with the version 4, so I spun up a new droplet, assigned one of my “spare” domains to it, and experimented. I got the tweaks to the theme where I wanted them, and got familiar with the changes. By and large, I LOVE the changes. All cool. Not sure I will turn it into a subscription newsletter thing, but I love the fact that I can possibly dump the Mailchimp free account I use, and gain some control. I am 99.99999999999993% certain that I will never turn on the paid/public option and build a community here, but it is intriguing (note: if I wanted to go that route, I would move to Substack, and leverage their community building vehicles).

I got it all working the way I like, so it was time to be more serious about upgrading.

I bet you can already guess how that went.

Badly. Really badly.

Decision Time

The fact that a clean upgrade from 3 to 4 failed was an eye opener. I did some searching, and found some kindred spirits. But none of the remedies I found and tried actually made any difference. Upgraded Node.JS, no dice. Manually adjusted the Systemd settings. Strike 2.

So I was faced with leaving the blog on version 3, which is still being maintained - for now - or deleting and recreating from scratch. An added calculus was that it was not a great fit to run under ServerPilot, as that was optimized for PHP systems, and bolting on node.js was a bit clunky (it worked, and it had good performance, but I hardly stress it with the traffic from this site), and that weighed in favor of moving to a fresh droplet. A little more money a month (really, less than I spend on coffee in a week) and a full stack tuned for Ghost makes sense.

Given the circumstances, and the fact that I had a clean export of data and content, I built from scratch.

Troubles in the install

I did what I usually do. Pick a droplet (one CPU, 2G ram, 50GB disk), and follow the recommended server setup. One of the benefits of Digital Ocean is the massive amount of online tutorials, how-to’s and walkthroughs. If you want to build a scaled, distributed system, they got ya covered. Want to harden a system? Easy Peasy. In this case, I picked the latest LTS version of Ubuntu, 20.04, and away I went.

After the install, I log in as root, create a local account, assign it sudo privileges, configure UFW to open ports for OpenSSH, SFTP, and then copy the public key file from the root to the new local account.

All good. That, coupled with doing the first update, which captured 133 packages that needed to be updated, and I was primed to start the install process.

Ghost is trickier to install than Wordpress. Wordpress is famous for their 5 minute install, and that makes it almost idiot proof. Almost all hosting solutions will have a generic LAMP (Linux, Apache, MySQL, PHP) stack pre installed and configured, so it is bonehead simple to get up and running.

Ghost doesn’t use Apache, or PHP, so you have to start by installing and configuring Nginx, which sounds scary, but isn’t. You do need to remember to configure the firewall for it, but that is no big deal, because there are App level config options, so you don’t have to individually open/close ports.

Then you have to install MySQL. Again, this is pretty trivial, but you do have to manually log in as root, and set a password.

Put a pin in this, as I will be returning to this.

Once you have that done, you instal Node.JS, and then from the NPM, you install the Ghost CLI tool, that then does the install with the command:

ghost install

Once that is done, the app starts up automatically, and you can do your migration of data, and other setup items.

Or so you should be able to.

The Problem

Of course, nothing goes as easy as planned.

One of the steps that is required was to manually add a password to the root account on MySQL. It requires you to log in, to type a prescripted command to set a password (remembering what that password is) and then quitting.

When I need secure passwords for applications like this, I go to a site I use very often, and have it generate a long, secure password. The one I use is Secure Password Generator.

n.b.: If I was running a commercial, high stakes site, where I collected PII, or financial information, I would be a lot more careful. But for personal blogs, this is an acceptable level of risk

I created the password, dropped it into a text file to store so that I could log in again if need be, and then moved on to the rest of the installation process.

Skipping ahead, I go through the installation process and all seems good, but I get a red asterisk at the creation of a local ghost user for MySQL. It is an automated process, and in the install, you provide the root password (aka the one generated with the password generator) and it creates the database, and then it is scripted to create a dedicated password for the blog to access.

Simple. Safe. Especially if you are going to host multiple apps on the same SQL server.

But upon completing the install, the instance wouldn’t start. It gave some lame error about not being able to access something.

I was pulling my hair out. Destroyed and recreated the droplet multiple times. No luck.

Then I looked at the config file config.prodution.json and I noticed something odd.

The password I used had a ‘\’ in it, along with numbers, letters, and other symbols.

It appears that the engine that constructs that config json file escapes the backslash with a second backslash. So during the installation, when it goes to create the new MySQL user for the instance, it uses the wrong password (remember second backslash) and fails, and it falls apart.

Fuck.

I must have gone through the install/reinstall/reinstall process 7 times before figuring this out. Removed the “ambiguous” characters from the password generator, and BAM all works.

The Actual Migration of Data

That was bonehead simple. SFTP in, and drop all the image files into the Content directory, then change permissions/owners. Then take the json export, and import it, delete the sample posts from a new Ghost install, reset the password from my old account, and the system is back up.

It even remembered my Google analytics tracking bits, and I was able to move the mail configurations from the old blog (it was configured with Mailgun as a making service) and it is like it never happened. Well, I have yet to do the subsequent subscription setup. I have a few pages to create for that, but otherwise, all is well.

Next steps

Get the subscription thing set up.

Move my subscribers from Mailchimp to Ghost (there are like 20 of them)

Delete the Disqus integration. Seriously, nobody actually comments anymore. Not one comment in more than 2 years...

Get back to publishing new, product management focused posts weekly at a minimum.