2mo ago

Looking for help: Running the same commands for multiple, similar variables

For fun, convenience and a chance to learn shell scripting for myself, I've been working on a collection of scripts to semi-automate the backup of my computer, two network connected Raspberry Pi's and my Android phone with Termux.

The main script basically runs a bunch of remote backup scripts, then copies those remote backups to a dedicated partition on my computer and finally creates a copy of that partition to an external drive connected to my computer. I use rsync and it's dry run feature so I am able to examine any file changes which has been super useful to me for catching mistakes and issues as I've been learning how to self-host over the past half year.

I have a simplified version of those backup scripts that makes a copy of my /home directory:

    
#!/bin/sh



# VARIABLES
## ENABLE FILE TRANSFER: Change variable `DRY_RUN` to `#DRY_RUN` to enable file tranfers
DRY_RUN="--dry-run" # Disables all file transfers

## PATHS (USER DEFINED)
SOURCE_DIRECTORY_PATH="/home"
SOURCE_DIRECTORY_PATH_EXCLUSIONS="--exclude=lost+found --exclude=.cache/*"
BACKUP_NAME="home"
BACKUP_BASE_PATH="/backup"

## PATHS (SCRIPT DEFINED/DO NOT TOUCH)
SOURCE_DIR="${SOURCE_DIRECTORY_PATH}/"
DESTINATION_DIR="${BACKUP_BASE_PATH}/${BACKUP_NAME}/"

## EXCLUSIONS (SCRIPT DEFINED/DO NOT TOUCH)
EXCLUDE_DIR="${SOURCE_DIRECTORY_PATH_EXCLUSIONS}"

## OPTIONS (SCRIPT DEFINED/DO NOT TOUCH)
OPTIONS="--archive --acls --one-file-system --xattrs --hard-links --sparse --verbose --human-readable --partial --progress --compress"
OPTIONS_EXTRA="--delete --numeric-ids"



# FUNCTIONS
## SPACER
SPACER() {
    printf "\n\n\n\n\n"
}

## RSYNC ERROR WARNINGS
ERROR_WARNINGS() {
    if [ "$RSYNC_STATUS" -eq 0 ]; then

        # SUCCESSFUL
        printf "\nSync successful"
        printf "\nExit status(0): %s\n" "$RSYNC_STATUS"

    else
        # ERRORS OCCURED
        printf "\nSome error occurred"
        printf "\nExit status(0): %s\n" "$RSYNC_STATUS"
    fi
}

## CONFIRMATION (YES/NO)
CONFIRM_YESNO() {
    while true; do
        prompt="${1}"
        printf "%s (Yes/No): " "${prompt}" >&2 # FUNCTION CALL REQUIRES TEXT PROMPT ARGUMENT
        read -r reply
        case $reply in
            [Yy]* ) return 0;; # YES
            [Nn]* ) return 1;; # NO
            * ) printf "Options: y / n\n";;
        esac
    done
}



##### START

# CHECK FOR ROOT
if ! [ "$(id -u)" = 0 ]; then

    # EXIT WITH NO ACTIONS TAKEN
    printf "\nRoot access required\n\n"
    return

else
    printf "\nStarting backup process..."

    # ${SOURCE_DIR} TO ${DESTINATION_DIR} DRY RUN
    SPACER
    printf "\nStarting %s dry run\n" "${SOURCE_DIR}"
    rsync --dry-run ${OPTIONS} ${OPTIONS_EXTRA} ${EXCLUDE_DIR} "${SOURCE_DIR}" "${DESTINATION_DIR}"
    RSYNC_STATUS=$?
    ERROR_WARNINGS

    # CONFIRM ${SOURCE_DIR} TO ${DESTINATION_DIR} BACKUP
    SPACER
    if CONFIRM_YESNO "Proceed with ${SOURCE_DIR} backup?"; then

        # CONTINUE ${SOURCE_DIR} TO ${DESTINATION_DIR} BACKUP & EXIT
        printf "\nContinuing %s backup\n" "${SOURCE_DIR}"
        rsync ${DRY_RUN} ${OPTIONS} ${OPTIONS_EXTRA} ${EXCLUDE_DIR} "${SOURCE_DIR}" "${DESTINATION_DIR}"
        RSYNC_STATUS=$?
        ERROR_WARNINGS

        printf "\n%s backup completed\n\n" "${SOURCE_DIR}"
        return

    else
        # SKIP ${SOURCE_DIR} TO ${DESTINATION_DIR} BACKUP & EXIT
        printf "\n%s backup skipped\n\n" "${SOURCE_DIR}"
        return
    fi
fi

##### FINISH

I would like to adapt this script so that I can add multiple copies of the following variables:

    
## PATHS (USER DEFINED)
SOURCE_DIRECTORY_PATH="/home"
SOURCE_DIRECTORY_PATH_EXCLUSIONS="--exclude=lost+found --exclude=.cache/*"
BACKUP_NAME="home"
BACKUP_BASE_PATH="/backup"

without having to make multiple copies of the following commands within the running script:

    
    # ${SOURCE_DIR} TO ${DESTINATION_DIR} DRY RUN
    SPACER
    printf "\nStarting %s dry run\n" "${SOURCE_DIR}"
    rsync --dry-run ${OPTIONS} ${OPTIONS_EXTRA} ${EXCLUDE_DIR} "${SOURCE_DIR}" "${DESTINATION_DIR}"
    RSYNC_STATUS=$?
    ERROR_WARNINGS

I'm mainly just looking for a way to avoid touching the script commands itself so I don't have to change the variable names for each additional directory I want to add. I'm not sure what that would be called or where to look. Any help would be greatly appreciated.

7 comments

You probably want something like the array variables from bash. You can find out more about them in the manpage, in the Arrays section.
Since you are using sh, you don't have direct access to them. If you want to use them, change the shebang to #!/usr/bin/env bash.
- I was hoping to stay using shell since I am using these scripts on both Alpine Linux and Debian. I have a preference for learning how to keep the script portable which is why I tried my best to follow POSIX standards. Shellcheck was quite useful for that when I rewrote all the scripts I wrote.
  If I must, it wouldn't be hard add bash to Alpine although I prefer to keep my web facing server as minimal as possible.
  Although I can look into arrays, that seems like a good spot to start looking.
  
  you will still be invoking this from the shell using the solution that person described. the line of code that they wrote for you starting with #! is called a shebang, and it's job is to tell the computer what interpreter to use to run your script if you execute the script, rather than executing bash and passing the script as an argument. that is, ./script.sh rather than bash my_script.sh. note, you will need to run chmod +x my_script.sh first in order to flag the script file as okay to execute.
try storing the paths to backup in a dot-file; one path per line. then, in the script, iterate that file line by line backing up the paths.
- I ended up using dot-files like you suggested. Instead of multiple source/destination locations within one dot-file, I've set it up to read multiple dot-files that contains one source/destination location each.
  Now I can use the command dir-sync /path/to/directory/.sync1 /path/to/directory/.sync2 or dir-sync /path/to/directory/* which is close enough to how I imagined it working.
  The challenge is that POSIX shell only has one array to work with. I may be able to inefficiently use more arrays, and I might try just for fun in the future but for now I have something that functions the way I want it to. I would have to use something like Bash if I intend to work more in depth with arrays though.
  
  I now have a bunch of other ideas on how I want to improve the script now. Adding ssh/network options, a check to make sure there are no empty command arguments, an option to create a blank template .sync- file in a chosen directory and a reverse direction transfer option are a few things I can think of off the top off my head. Hopefully this array was the most difficult part to learn and understand.