Dimitris Ganotis
dganotis.dev
Published on

Syncing Strapi Instances When the Official CLI Falls Short

Authors
  • avatar
    Name
    Dimitris Ganotis
    Twitter

Strapi has strapi export and strapi import commands for moving data between instances. I spent a while trying to get them to work reliably before giving up. On a deployment with a large uploads directory they just don't hold up — timeouts, memory issues, corrupted archives. The larger the app, the worse it gets.

My setup is Strapi on a VPS with Docker Compose, MySQL, and uploads stored as local files. I ended up writing a bash script that bypasses the CLI entirely and handles the transfer directly. I have some devops experience but I'm not a bash expert — I built the final version iteratively with the help of an LLM, which made it a lot more practical to get into the edge cases I'd have otherwise skipped. This post is mostly notes on what I built and the problems I ran into. Not all of it will be relevant to your setup.

Skip the CLI, use the primitives

At the end of the day you're moving a database and a folder of files. mysqldump and tar over SSH are boring tools that actually work. The script is a wrapper around those two with some coordination logic on top.

Order of operations

You can't just dump the database and sync the files as two separate steps. If Strapi writes a new upload between them you'll end up with either a file that has no database record, or a record pointing to something that never made it across.

What I settled on:

  1. Backup the destination DB and uploads
  2. Stop the destination services
  3. Dump and transfer the source database
  4. Sync the uploads
  5. Import the database
  6. Rewrite URLs in the database
  7. Start everything back up

Stopping destination services before the transfer means nothing can write during the window. The source keeps running the whole time — mysqldump --single-transaction is enough for consistency there, no table locks were needed on my case.

The database dump streams directly from the source MySQL container to the destination over a pipe — no intermediate file on your local machine:

ssh user@source "docker compose exec -T mysql sh -lc '
  MYSQL_PWD="$MYSQL_ROOT_PASSWORD" mysqldump \
    --single-transaction --quick --no-tablespaces \
    -uroot mydb
' | gzip -1" | ssh user@destination "cat > /tmp/dump.sql.gz"

Same idea for uploads — tar on the source piped into tar on the destination:

ssh user@source "tar -C /path/to/uploads -cf - ." \
  | ssh user@destination "tar -C /path/to/uploads -xf -"

Disk space

My uploads were around 16GB and this is where most of the friction was. An atomic swap — unpack into a temp directory, then rename into place — needs roughly double that free at the same time. On a VPS that's often not realistic.

The script checks available disk before doing anything destructive:

free_bytes=$(ssh user@destination "df -PB1 /path/to/uploads | awk 'NR==2 {print \$4}'")
required_bytes=$((uploads_size + db_size + safety_margin))

if [[ "$free_bytes" -lt "$required_bytes" ]]; then
  echo "Not enough disk space. Available: $free_bytes, required: $required_bytes"
  exit 1
fi

A couple of things that helped reduce the required space: skipping the pre-sync uploads backup (the destination is getting overwritten anyway), and excluding large ZIPs from the transfer. I had a handful of big archive files that weren't needed on the other instance, so I skip them during the tar stream and write empty placeholder files at the same paths on the destination instead. The database records stay intact, the files just aren't downloadable. Cut the effective transfer size significantly.

If there's still not enough room for an atomic swap, you can stream directly into the destination directory after clearing it first. There's a window where uploads are gone entirely, which isn't ideal, but it works.

File ownership

This one took me a while to figure out. Strapi runs inside Docker as a non-root user — uid 1000 in my case. The uploads folder is bind-mounted from the host, owned by that uid.

When you extract a tar archive over SSH the files come out owned by your SSH user, not the container user. Strapi then can't write new uploads because it doesn't own the directory.

The fix is running chown as root inside a throwaway container using the same image as your backend service:

docker run --rm -u 0 \
  -v /host/uploads:/app/uploads \
  your-backend-image \
  chown -R 1000:1000 /app/uploads

No sudo required on the host. I use the same approach for clearing the uploads directory before a direct sync — find -mindepth 1 -delete inside the container, because doing it from the SSH session fails with permission denied on files owned by the container user.

The script also resolves the uid automatically from the running container rather than hardcoding it:

uid=$(ssh user@destination "docker compose exec -T backend sh -lc 'id -u'")
gid=$(ssh user@destination "docker compose exec -T backend sh -lc 'id -g'")

URL rewrites

After importing the database, all the URLs in text columns still point to the source domain. Strapi stores asset URLs and rich text content across a lot of tables.

My first attempt was building a single UPDATE dynamically with GROUP_CONCAT and running it through PREPARE. That doesn't work when more than one table matches — PREPARE only accepts a single statement. A stored procedure with a cursor that executes each UPDATE separately does the job:

CREATE PROCEDURE rewrite_urls()
BEGIN
  DECLARE done INT DEFAULT 0;
  DECLARE tbl VARCHAR(64);
  DECLARE col VARCHAR(64);
  DECLARE cur CURSOR FOR
    SELECT TABLE_NAME, COLUMN_NAME
    FROM information_schema.COLUMNS
    WHERE TABLE_SCHEMA = DATABASE()
      AND DATA_TYPE IN ('char','varchar','tinytext','text','mediumtext','longtext');
  DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = 1;
  OPEN cur;
  loop: LOOP
    FETCH cur INTO tbl, col;
    IF done THEN LEAVE loop; END IF;
    SET @sql = CONCAT(
      'UPDATE `', tbl, '` SET `', col, '` = REPLACE(`', col, '`, ''https://source.example.com'', ''https://destination.example.com'') ',
      'WHERE `', col, '` LIKE ''%source.example.com%'''
    );
    PREPARE stmt FROM @sql;
    EXECUTE stmt;
    DEALLOCATE PREPARE stmt;
  END LOOP;
  CLOSE cur;
END;

Run it once per URL pair — frontend, API, www variant.

This is essentially the same idea as the "search and replace" functionality you find in WordPress migration plugins — replace all occurrences of the old domain across the entire database.

SSH connection reuse

The script opens a lot of SSH connections across a single run. Without doing anything special that means a lot of password prompts. SSH ControlMaster keeps one session open and routes everything else through it:

control_path="/tmp/ssh-control-$$.sock"

# Open the master connection once
ssh -o ControlMaster=yes \
    -o ControlPersist=600 \
    -o ControlPath="$control_path" \
    -Nf user@host

# All subsequent calls reuse it silently
ssh -o ControlMaster=auto \
    -o ControlPath="$control_path" \
    user@host "some-command"

Two password prompts total for the whole run, one per host.

Cleanup on failure

One thing worth getting right from the start is a cleanup trap. If the script fails halfway through, you don't want to leave the destination services stopped or temp files sitting around:

cleanup() {
  # Remove temp files on destination
  ssh user@destination "rm -f /tmp/dump.sql.gz" || true

  # Bring destination services back up if they were stopped
  if [[ "$services_stopped" == "1" ]]; then
    ssh user@destination "docker compose up -d backend frontend" || true
  fi
}
trap cleanup EXIT

The full script ties all of this together with disk space estimation, preflight validation, backup retention, and a few other things. It's fairly specific to my setup so I haven't published it yet, but I might clean it up and put it out at some point.