Efficiently Publish to a Website

Summary: alias publish="rsync -rtvzP --filter=':- .gitignore' --exclude='.git*' ~/website_path/ user@hostname.com:/home/public"

Abbreviated SCP

I used to run the following commands to upload files to this website:

      scp file user@hostname.com:/home/public
      scp articles/article-name user@hostname.com:/home/public/articles

Typing out the path of the host is tedious, so I wrote a shell script to do it.

      #!/bin/sh

      [ "$1" = "-a" ] && shift && scp $@ user@hostname.com:/home/public/articles && exit
      scp $@ user@hostname.com:/home/public

[ "$1" = "-a" ] checks if the first argument is -a. && causes the following command to run only if the previous statement succeeded. shift removes the first argument. Then, the files ($@) are uploaded to the articles' folder. If the -a option is not present, the files will be uploaded to the public folder.

Now, the process of uploading files looks like this:

      upload file
      upload -a articles/article-name

Rsync

The problem with that method is that the files need to be stated. To avoid this, rsync can be used.

Those options mean the following:

• -r: recursively
• -t: transfer modification times, allowing unmodified files to be skipped
• -v: increase verbosity, showing the files uploaded
• -z: compress files for uploading
• -P: if a large file fails to be uploaded, continue from where it was instead of uploading its entirety
• --filter':- .gitignore': do not upload files that are listed in .gitignore
• --exclude='.git*': do not upload the .gitignore file nor the .git folder

The slash after the folder name (riazj.com/) is there for uploading the contents of the folder and not the folder itself to the location. Without the slash, it would result in /home/public/riazj.com.

Git and SCP

I do not recommend using this method unless there is a reason to not use rsync. rsync is faster than this script. I wrote this before I decided to use rsync for my website.

Post-receive hooks can upload files to website using git push. However, I do not want to have a .git folder (even if the size is small) on the server.

An SSH key to automatically sign in to the host is required.

      #!/bin/sh

      changed_files="$(git diff-tree --no-commit-id --name-only origin/main..HEAD -r)"

      for file in $changed_files; do
        [ -e $file ] || ssh user@hostname.com "rm -f $file" && continue
        [ "${file#*"articles/"}" != "$file" ] && scp $file user@hostname.com:/home/public/articles && continue
        scp $file user@hostname.com:/home/public
      done

      git push

This script is meant to be run before git push so that it can see the files that were changed in the local commits.

[ -e $file ] checks if the file exists. If it does not, it is removed. continue skips to the next iteration of the loop.

[ "${file#*"articles/"}" != "$file" ] checks if the file's path is not the same after trying to remove the string "articles/" from the start of it. If it is not the same as before, it must mean that the file path starts with "articles/". The file is then uploaded to the articles' folder.

If the previous commands did not run, the file is uploaded to the public folder.

This script does not account for removing an old file that has been renamed. It can be expanded on by using git diff-tree --no-commit-id --name-status origin/main..HEAD -r and grep to find the renamed files.