Upgrading major versions of a multi instance Ghost setup

I run a couple of blogs using Ghost and running on a Digital Ocean droplet and today I had to go through the arduous task of upgrading it. I am not exaggerating when I say arduous as it was anything but trivial. Initially, I started by upgrading the Ghost instances in place but soon I realised that it was going to be more of a pain than I wanted. You see, I had set up my blog more than 5 years back using the formerly 1-click installation of Ghost (now called marketplace image installation) and in my infinite wisdom neglected to upgrade the installation. And thus I found that it was still running on Ubuntu 18.04 and Node 14. So, I decided that instead of spending hours trying to upgrade from the latest v4 to v5 of Ghost and potentially ruining the installation and crying in a corner in misery, I could just spin up a droplet, install Ghost afresh, and restore a backup. A genius idea I must say!

But first some system information:

Source system:

1. Ubuntu 18.04.6 with 1 vCPU and 1GB RAM
2. Ghost CLI 1.25.3 and Ghost 4.48.9
3. Themed with Casper 4.7.4

Destination system:

1. Ubuntu 22.04.3 with 1 vCPU and 2GB RAM
2. Ghost CLI 1.25.3 and Ghost 5.79.3
3. Themed with Casper 5.7.0

I expected things to be an easy affair but soon I was proved wrong when I realised that the Digital Ocean marketplace image itself doesn't support multi instance Ghost installations.

After spending over 24 hours researching and bringing up and destroying droplets over and over again to get the results I wanted, I finally found out the perfect steps to create a multi instance setup when starting from a marketplace image and how to migrate an existing setup to the new one. Hopefully, this will save some time for someone.

Backup Source Blogs

Perform the following steps in the source droplet.

1. Switch to the ghost-mgr user.

   sudo -i -u ghost-mgr
   

2. Go to each directory where Ghost is installed. For our purposes, we will assume /var/www/sayak. One can go to the directories by the following command.

   cd /var/www/sayak
   

3. Run the backup command using the ghost CLI in each directory.

   ghost backup
   

   This will generate a .zip file for each backed up directory.
4. Download the backups to a local system. We will need them shortly.
5. Turn off the droplet from the Digital Ocean control panel or run the following in the terminal to shut down the droplet

   poweroff
   

6. Take a snapshot of the droplet from Digital Ocean. Taking a snapshot means that in case something horrible happens, we can create a fresh droplet from the snapshot.
7. Turn on the droplet from the Digital Ocean control panel once the snapshot is done.

Change the URL of the source blogs

Once we start setting up the destination droplet, we will need to give the URLs of the blogs. This means we will need to change the DNS of our domains to the destination droplet's IP address which means we will no longer be able to access the source blogs.

However, it is helpful to have access to the source blogs even after the destination blogs are up and running. One can use both of them to compare differences and make small changes in the destination blogs as necessary. This is the case when the themes have breaking changes.

1. Create a new DNS entry for the source blogs. In our case, we will make it old.sayakm.me.
2. SSH into the source droplet and switch to the ghost-mgr user.

   sudo -i -u ghost-mgr
   

3. Go to each directory where Ghost is installed. For our purposes, we will assume /var/www/sayak. One can go to the directories by the following command.

   cd /var/www/sayak
   

4. Run the ghost config command in each directory to update the URL in the configuration.

   ghost config url https://old.sayakm.me
   

   This will generate an output confirming the success.

   Successfully set 'url' to 'https://old.sayakm.me'
   

5. Next, run the setup command to update the Nginx and SSL configuration in each directory

   ghost setup nginx
   

   This will give an output as such

   ✔ Setting up Nginx
   

   And

   ghost setup ssl
   

   This will ask for your email which will be used to generate the SSL certificate only. Provide it which will give this output

   ? Enter your email (For SSL Certificate) youremail@gmail.com
   ✔ Setting up SSL
   

6. Finally, restart each Ghost installation using the following command

   ghost restart
   

   On success, we will get

   ✔ Restarting Ghost
   

Setup the destination blog

Now, it's time to setup the destination droplet and blogs.

1. Create the destination droplet in Digital Ocean with the Ghost Marketplace image.
2. Once the droplet gets created, update the IP of the existing blog domains to the IPs of the new droplet.
3. Now access the droplet using SSH. The Ghost CLI will start initialising the droplet with the default Ghost installation. We don't want this installation but we do want some of the bootstrapping it does. So, we will provide our options carefully.
4. The configuration will pause to ask for the blog URL and SSL certificate email. Provide a generic URL that is not controlled as we don't need an SSL certificate to be generated. Here's what the output will look like. Of course, the SSL step will fail. That's expected and fine.

   ✔ Checking system Node.js version - found v18.17.1
   ✔ Checking current folder permissions
   ✔ Checking memory availability
   ✔ Checking free space
   ✔ Checking for latest Ghost version
   ✔ Setting up install directory
   ✔ Downloading and installing Ghost v5.79.3
   ✔ Finishing install process
   ? Enter your blog URL: https://example.com
   ✔ Configuring Ghost
   ✔ Setting up instance
   + sudo useradd --system --user-group ghost
   + sudo chown -R ghost:ghost /var/www/ghost/content
   ✔ Setting up "ghost" system user
   ✔ Setting up "ghost" mysql user
   + sudo mv /tmp/example-com/example.com.conf /etc/nginx/sites-available/example.com.conf
   + sudo ln -sf /etc/nginx/sites-available/example.com.conf /etc/nginx/sites-enabled/example.com.conf
   + sudo nginx -s reload
   ✔ Setting up Nginx
   ? Enter your email (For SSL Certificate) youremail@gmail.com
   + sudo mkdir -p /etc/letsencrypt
   + sudo ./acme.sh --install --home /etc/letsencrypt
   + sudo /etc/letsencrypt/acme.sh --issue --home /etc/letsencrypt --server letsencrypt --domain example.com --webroot /var/www/ghost/system/nginx-root --reloadcmd "nginx -s reload" --accountemail youremail@gmail.com --keylength 2048
   ✖ Setting up SSL
   + sudo mv /tmp/example-com/ghost_example-com.service /lib/systemd/system/ghost_example-com.service
   + sudo systemctl daemon-reload
   ✔ Setting up Systemd
   + sudo systemctl is-active ghost_example-com
   + sudo systemctl start ghost_example-com
   + sudo systemctl is-enabled ghost_example-com
   + sudo systemctl enable ghost_example-com --quiet
   ✔ Starting Ghost
   One or more errors occurred.
   

5. A directory /var/www/ghost will be created. Get into the directory using

   cd /var/www/ghost
   

6. We get the configuration details of this installation using

   cat config.production.json
   

   The output will look like the following

   {
     "url": "https://example.com",
     "server": {
       "port": 2368,
       "host": "127.0.0.1"
     },
     "database": {
       "client": "mysql",
       "connection": {
         "host": "127.0.0.1",
         "user": "ghost-69",
         "password": "thepassword",
         "port": 3306,
         "database": "ghost_production"
       }
     },
     "mail": {
       "transport": "Direct"
     },
     "logging": {
       "transports": [
         "file",
         "stdout"
       ]
     },
     "process": "systemd",
     "paths": {
       "contentPath": "/var/www/ghost/content"
     }
   }
   

   Note the property of database.connection.host, database.connection.user and database.connection.password.
7. Switch to the ghost-mgr user.

   sudo -i -u ghost-mgr
   

8. Go to each directory where Ghost is installed using the following command.

   cd /var/www/ghost
   

9. Next, uninstall this Ghost instance using

   ghost uninstall
   

   This will result in the following output

   ? Are you sure you want to do this? Yes
   + sudo systemctl is-active ghost_example-com
   + sudo systemctl stop ghost_example-com
   + sudo systemctl is-enabled ghost_example-com
   + sudo systemctl disable ghost_example-com --quiet
   ✔ Stopping Ghost
   + sudo rm -rf /var/www/ghost/content
   ✔ Removing content folder
   + sudo rm -f /etc/nginx/sites-available/example.com.conf
   + sudo rm -f /etc/nginx/sites-enabled/example.com.conf
   + sudo nginx -s reload
   + sudo rm /lib/systemd/system/ghost_example-com.service
   ✔ Removing related configuration
   ✔ Removing Ghost installation
   

   This will also mean that all contents of the /var/www/ghost will be removed

Next, we will need to create directories for each of our blogs. The following steps will need to be followed for each of the blog instances. I will note only the first one.

1. Ensure you are in root user.

2. Create a directory for the Ghost instance that's going to be created.

   cd /var/www
   mkdir sayak
   

3. Modify the permissions of this directory to ensure that the ghost-mgr user can access it.

   chown ghost-mgr:ghost-mgr /var/www/sayak
   chmod 775 /var/www/sayak
   

4. Before we install Ghost, we will need to ensure that the database it needs is already present. The Ghost startup creates a database called ghost_production for the pre-created instance. For new instances, this is not automatically created but we will use that database as a template for our new database. We will need to login to MySQL as the root user and for that, we need that password.

   Thankfully, it's already present in the file /root/.digitialocean_password. Once we get the password from the file, we will login to MySQL as the root user.

   cat /root/.digitialocean_password
   

   This will display the password. Copy it to the clipboard.

   mysql -p
   

   This will prompt for the password so paste it from the clipboard.
   This should get you logged into MySQL's console.

5. First, we will check what databases are present

   SHOW DATABASES;
   

   We should get an output like

   +--------------------+
   | Database           |
   +--------------------+
   | ghost_production   |
   | information_schema |
   | mysql              |
   | performance_schema |
   | sys                |
   +--------------------+
   

6. We will also check what users are present and what kind of grants are provided to the ghost-69 user that we found from the database.connection.user property of the config.production.json file.

   SELECT user FROM mysql.user;
   

   We should get an output like

   +------------------+
   | user             |
   +------------------+
   | ghost-69         |
   | root             |
   | debian-sys-maint |
   | mysql.infoschema |
   | mysql.session    |
   | mysql.sys        |
   | root             |
   +------------------+
   

   Run the following command to check the grants provided to the user ghost-69 using the host found in the database.connection.host property of the config.production.json file.

   SHOW GRANTS FOR 'ghost-69'@'127.0.0.1';
   

   which will result in an output of

   +------------------------------------------------------------------------+
   | Grants for ghost-69@127.0.0.1                                          |
   +------------------------------------------------------------------------+
   | GRANT USAGE ON *.* TO `ghost-69`@`127.0.0.1`                           |
   | GRANT ALL PRIVILEGES ON `ghost_production`.* TO `ghost-69`@`127.0.0.1` |
   +------------------------------------------------------------------------+
   

7. Since the directory we created in /var/www was sayak, we will create a database named sayak_prod as that is the default naming convention of Ghost. We will also grant privileges to the user ghost-69.

   CREATE DATABASE sayak_prod;
   GRANT ALL PRIVILEGES ON sayak_prod.* TO 'ghost-69'@'127.0.0.1';
   

   We will verify by running

   SHOW GRANTS FOR 'ghost-69'@'127.0.0.1';
   

   which will result in an output of

   +------------------------------------------------------------------------+
   | Grants for ghost-69@127.0.0.1                                          |
   +------------------------------------------------------------------------+
   | GRANT USAGE ON *.* TO `ghost-69`@`127.0.0.1`                           |
   | GRANT ALL PRIVILEGES ON `ghost_production`.* TO `ghost-69`@`127.0.0.1` |
   | GRANT ALL PRIVILEGES ON `sayak_prod`.* TO `ghost-69`@`127.0.0.1`       |
   +------------------------------------------------------------------------+
   

8. Finally, we can exit MySQL

   exit
   

Finally, we can get around to installing Ghost.

1. Switch to the ghost-mgr user.

   sudo -i -u ghost-mgr
   

2. Go to the directory where Ghost is to be installed.

   cd /var/www/sayak
   

3. Finally, install Ghost

   ghost install
   

   There will be prompts for multiple questions

   ✔ Checking system Node.js version - found v18.17.1
   ✔ Checking current folder permissions
   ✔ Checking memory availability
   ✔ Checking free space
   ✔ Checking for latest Ghost version
   ✔ Setting up install directory
   ✔ Downloading and installing Ghost v5.79.3
   ✔ Finishing install process
   ? Enter your blog URL: https://sayakm.me
   ? Enter your MySQL hostname: 127.0.0.1
   ? Enter your MySQL username: sayak-69
   ? Enter your MySQL password: [hidden]
   ? Enter your Ghost database name: sayak_prod
   ✔ Configuring Ghost
   ✔ Setting up instance
   + sudo chown -R ghost:ghost /var/www/sayak/content
   ✔ Setting up "ghost" system user
   ℹ Setting up "ghost" mysql user [skipped]
   ? Do you wish to set up Nginx? Yes
   + sudo mv /tmp/sayakm-me/sayakm.me.conf /etc/nginx/sites-available/sayakm.me.conf
   + sudo ln -sf /etc/nginx/sites-available/sayakm.me.conf /etc/nginx/sites-enabled/sayakm.me.conf
   + sudo nginx -s reload
   ✔ Setting up Nginx
   ? Do you wish to set up SSL? Yes
   ? Enter your email (For SSL Certificate) mukhopadhyaysayak@gmail.com
   + sudo /etc/letsencrypt/acme.sh --upgrade --home /etc/letsencrypt
   + sudo /etc/letsencrypt/acme.sh --issue --home /etc/letsencrypt --server letsencrypt --domain sayakm.me --webroot /var/www/sayak/system/nginx-root --reloadcmd "nginx -s reload" --accountemail mukhopadhyaysayak@gmail.com --keylength 2048
   + sudo mv /tmp/sayakm-me/sayakm.me-ssl.conf /etc/nginx/sites-available/sayakm.me-ssl.conf
   + sudo ln -sf /etc/nginx/sites-available/sayakm.me-ssl.conf /etc/nginx/sites-enabled/sayakm.me-ssl.conf
   + sudo nginx -s reload
   ✔ Setting up SSL
   ? Do you wish to set up Systemd? Yes
   + sudo mv /tmp/sayakm-me/ghost_sayakm-me.service /lib/systemd/system/ghost_sayakm-me.service
   + sudo systemctl daemon-reload
   ✔ Setting up Systemd
   + sudo systemctl is-active ghost_sayakm-me
   ? Do you want to start Ghost? Yes
   + sudo systemctl start ghost_sayakm-me
   + sudo systemctl is-enabled ghost_cmdrgarud-blog
   + sudo systemctl enable ghost_cmdrgarud-blog --quiet
   ✔ Starting Ghost
   
   Ghost was installed successfully! To complete setup of your publication, visit:
   
       https://sayakm.me/ghost/
   

4. Finally, verify that the blog is running

   ghost ls
   

   which should give an output of

   ┌────────────────┬────────────────────┬─────────┬──────────────────────┬────────────────────────┬──────┬─────────────────┐
   │ Name           │ Location           │ Version │ Status               │ URL                    │ Port │ Process Manager │
   ├────────────────┼────────────────────┼─────────┼──────────────────────┼────────────────────────┼──────┼─────────────────┤
   │ sayakm-me      │ /var/www/sayak     │ 5.79.3  │ running (production) │ https://sayakm.me      │ 2369 │ systemd         │
   └────────────────┴────────────────────┴─────────┴──────────────────────┴────────────────────────┴──────┴─────────────────┘
   

And with that, the destination Ghost blog is up and running. Now to follow the above steps for each blog you need to migrate!

Importing from backup

Now wait! We have the destination blog running but we are yet to transfer the data. That's a simpler part and is already documented by the folks at Ghost but for the sake of completeness, I will reiterate them.

1. Copy the backups present in the local system to a location in the destination droplet.
2. Ensure you are in the Ghost installation directory, for eg. /var/www/sayak. If not, head to it using

   cd /var/www/sayak
   

3. Unzip the backups to the content directory and provide permissions to the ghost user to access all the contents within it.

   sudo unzip /location-of-backup-zip-file.zip -d content
   sudo chown -R ghost:ghost content
   

4. Switch to the ghost-mgr user

   sudo -i -u ghost-mgr
   

5. Go to the directory where Ghost is installed.

   cd /var/www/ghost
   

6. Restart the Ghost instance to load all the changes

   ghost restart
   

7. Now, we will need to restore the data to the database. We can find a json file in /var/www/sayak/content/data. This will be the source of the restore. To restore run

   ghost import content/data/the-json-file.json
   

   This will ask for an admin password. Provide the password that was used for the admin account in the source blog.
8. Restart the Ghost instance again to ensure database changes are loaded.

   ghost restart
   

9. Open the Ghost web admin portal and one should see all their content and settings.

Updating profile images

This step should not be needed but looks like for some reason, the admin profile image and the cover image aren't getting restored. The trivial way to fix it is to upload the images again in the web admin portal but I am nitpicky and don't want to re-upload images already present in the server. And yes, I checked, the files are there but the database entries are empty. So, we are going to fix that!

For this, we need a few info.

1. The MySQL ghost username. We have been using ghost-69 so that's what it will be.
2. The Password for the above username. We have used thepassword as an example.
3. The MySQL hostname. It's 127.0.0.1 as noted earlier.
4. In the json file present in /var/www/sayak/content/data, need to get the values of profile_image and cover_image from db.data.users key. The values should start with https://sayakm.me/content/images. Either or both can even be null. If both are null then the following steps don't need to be followed and you are done!

Once we have them, we can go ahead with updating the database.

1. Login to MySQL as the ghost-69 user.

   mysql -u 'ghost-69' -h '127.0.0.1' -p
   

   When prompted provide the password.
2. Activate the sayak_prod database.

   USE sayak_prod;
   

3. We will check the users table to verify the problem

   SELECT id, name, profile_image, cover_image FROM users;
   

   This should give an output like

   +----+-------+---------------+-------------+
   | id | name  | profile_image | cover_image |
   +----+-------+---------------+-------------+
   | 1  | Sayak | NULL          | NULL        |
   +----+-------+---------------+-------------+
   

   Those 2 fields should not be NULL if you had profile and cover images earlier.
4. To fix it, we will run the following command

   UPDATE users SET profile_image = 'value-from-the-profile_image-key', cover_image = 'value-from-the-cover_image-key' WHERE id=1;
   

   If one of profile_image or cover_image in the JSON file is null, don't include that in the above SQL. For eg. if profile_image is not null but cover_image is null in the JSON file, the SQL command should instead be

   UPDATE users SET profile_image = 'value-from-the-profile_image-key' WHERE id=1;
   

And with that, we are finally done! Your blog should be ready to use with everything as it was before, with a few caveats and oddities.

Caveats and oddities

1. Your theme might reset to whatever Ghost considers default. Just changing back to the earlier theme should be enough to fix that.
2. The admin panel might go back to light mode if you were a dark mode user.
3. If the theme has undergone a major change, prepare for things to break, especially if you have injected scripts and styles.

And well, that's all folks! I hope this saves some time for future folks. Remember, this is what my experience has been and Ghost moves fast and things mentioned here might not work in the future. Still, I would be happy to improve, just tweet me, or was it X me🤔.