185 lines
4.6 KiB
Markdown
185 lines
4.6 KiB
Markdown
# Rabbi Gerzi State and Database Recovery
|
|
|
|
This procedure restores Rabbi Gerzi from its BorgBase backup. It replaces the
|
|
current application state directory and PostgreSQL database with the selected
|
|
archive.
|
|
|
|
The backup contains:
|
|
|
|
- `/var/lib/rabbi-gerzi`
|
|
- `/var/backup/rabbi-gerzi/rabbi-gerzi.dump`
|
|
|
|
## Requirements
|
|
|
|
- BorgBase repository URL:
|
|
`ssh://mgjjruz9@mgjjruz9.repo.borgbase.com/./repo`
|
|
- Decrypted Borg SSH private key and repository passphrase
|
|
- The original Rabbi Gerzi production environment, including `APP_KEY`
|
|
- A target server already switched to the Rabbi Gerzi NixOS module
|
|
- A shell with `borg`, `openssh`, `postgresql`, and `rsync`
|
|
|
|
On NixOS or another machine with Nix:
|
|
|
|
```sh
|
|
nix-shell -p borgbackup openssh postgresql rsync
|
|
```
|
|
|
|
The commands below assume the default state directory, database, user, and
|
|
PHP-FPM pool names. Substitute the configured values if the NixOS module uses
|
|
non-default names.
|
|
|
|
## Prepare Borg Access
|
|
|
|
Copy the Borg SSH key into a local recovery directory:
|
|
|
|
```sh
|
|
mkdir -p ~/borg-recovery/rabbi-gerzi
|
|
cp /path/to/borg-private-key \
|
|
~/borg-recovery/rabbi-gerzi/borg-private-key
|
|
chmod 600 ~/borg-recovery/rabbi-gerzi/borg-private-key
|
|
```
|
|
|
|
Set the Borg connection environment:
|
|
|
|
```sh
|
|
export BORG_REPO='ssh://mgjjruz9@mgjjruz9.repo.borgbase.com/./repo'
|
|
export BORG_RSH='ssh'\
|
|
' -i ~/borg-recovery/rabbi-gerzi/borg-private-key'\
|
|
' -o IdentitiesOnly=yes'\
|
|
' -o StrictHostKeyChecking=accept-new'
|
|
```
|
|
|
|
Read the Borg passphrase without displaying it:
|
|
|
|
```sh
|
|
read -rsp 'Borg passphrase: ' BORG_PASSPHRASE
|
|
export BORG_PASSPHRASE
|
|
echo
|
|
```
|
|
|
|
## Extract an Archive
|
|
|
|
List the available archives:
|
|
|
|
```sh
|
|
borg list
|
|
```
|
|
|
|
Choose an archive, then extract it into a temporary working directory. Do not
|
|
extract directly into `/`.
|
|
|
|
```sh
|
|
export ARCHIVE='ARCHIVE_NAME_FROM_BORG_LIST'
|
|
|
|
mkdir -p ~/borg-recovery/rabbi-gerzi/extract
|
|
cd ~/borg-recovery/rabbi-gerzi/extract
|
|
|
|
borg extract ::"$ARCHIVE" var/lib/rabbi-gerzi \
|
|
var/backup/rabbi-gerzi/rabbi-gerzi.dump
|
|
```
|
|
|
|
## Verify Without Replacing Production
|
|
|
|
Restore the extracted dump into a temporary database and inspect its core
|
|
table counts:
|
|
|
|
```sh
|
|
sudo -u postgres dropdb --if-exists rabbi-gerzi-backup-verify
|
|
sudo -u postgres createdb \
|
|
--owner=rabbi-gerzi rabbi-gerzi-backup-verify
|
|
sudo -u postgres pg_restore \
|
|
--exit-on-error \
|
|
--no-owner \
|
|
--role=rabbi-gerzi \
|
|
--dbname=rabbi-gerzi-backup-verify \
|
|
./var/backup/rabbi-gerzi/rabbi-gerzi.dump
|
|
sudo -u postgres psql -d rabbi-gerzi-backup-verify <<'SQL'
|
|
SELECT 'users' AS table_name, count(*) FROM users
|
|
UNION ALL
|
|
SELECT 'sets', count(*) FROM sets
|
|
UNION ALL
|
|
SELECT 'elements', count(*) FROM elements;
|
|
SQL
|
|
```
|
|
|
|
List the extracted managed files and confirm expected uploads are present:
|
|
|
|
```sh
|
|
find ./var/lib/rabbi-gerzi/storage/app/public -type f -print
|
|
```
|
|
|
|
Remove only the temporary verification database when the checks pass:
|
|
|
|
```sh
|
|
sudo -u postgres dropdb rabbi-gerzi-backup-verify
|
|
```
|
|
|
|
## Restore Production
|
|
|
|
Run these commands from `~/borg-recovery/rabbi-gerzi/extract` on the target
|
|
server. Capture absolute tool paths so they remain available through `sudo`:
|
|
|
|
```sh
|
|
RSYNC="$(command -v rsync)"
|
|
PSQL="$(command -v psql)"
|
|
DROPDB="$(command -v dropdb)"
|
|
CREATEDB="$(command -v createdb)"
|
|
PG_RESTORE="$(command -v pg_restore)"
|
|
```
|
|
|
|
Stop the API and ensure PostgreSQL is running:
|
|
|
|
```sh
|
|
sudo systemctl stop phpfpm-rabbi-gerzi.service
|
|
sudo systemctl start postgresql.service
|
|
```
|
|
|
|
Replace the application state and repair ownership:
|
|
|
|
```sh
|
|
sudo "$RSYNC" -a --delete \
|
|
./var/lib/rabbi-gerzi/ /var/lib/rabbi-gerzi/
|
|
sudo chown -R rabbi-gerzi:rabbi-gerzi /var/lib/rabbi-gerzi
|
|
```
|
|
|
|
Replace the PostgreSQL database:
|
|
|
|
```sh
|
|
sudo -u postgres "$PSQL" -d postgres -v ON_ERROR_STOP=1 <<'SQL'
|
|
SELECT pg_terminate_backend(pid)
|
|
FROM pg_stat_activity
|
|
WHERE datname = 'rabbi-gerzi'
|
|
AND pid <> pg_backend_pid();
|
|
SQL
|
|
|
|
sudo -u postgres "$DROPDB" --if-exists rabbi-gerzi
|
|
sudo -u postgres "$CREATEDB" --owner=rabbi-gerzi rabbi-gerzi
|
|
sudo -u postgres "$PG_RESTORE" \
|
|
--exit-on-error \
|
|
--no-owner \
|
|
--role=rabbi-gerzi \
|
|
--dbname=rabbi-gerzi \
|
|
./var/backup/rabbi-gerzi/rabbi-gerzi.dump
|
|
```
|
|
|
|
Run current migrations and restart the API:
|
|
|
|
```sh
|
|
set -e
|
|
sudo systemctl restart rabbi-gerzi-setup.service
|
|
sudo systemctl start phpfpm-rabbi-gerzi.service
|
|
```
|
|
|
|
## Verify Production
|
|
|
|
Check service state and recent logs:
|
|
|
|
```sh
|
|
sudo systemctl status rabbi-gerzi-setup.service --no-pager
|
|
sudo systemctl status phpfpm-rabbi-gerzi.service --no-pager
|
|
sudo journalctl -u rabbi-gerzi-setup.service -b --no-pager -n 100
|
|
sudo journalctl -u phpfpm-rabbi-gerzi.service -b --no-pager -n 100
|
|
```
|
|
|
|
Open the public site and confirm that sets, elements, uploaded images, PDFs,
|
|
and administrator login all work.
|