User Tools

Site Tools


linux:parallel_rsync

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision
Previous revision
Last revisionBoth sides next revision
parallel_rsync [2012/12/06 18:06] dodgerlinux:parallel_rsync [2019/07/29 12:11] – [Code] dodger
Line 1: Line 1:
 +====== [SCRIPT] psync (parallel rsync) ======
 +
 ====== Description ====== ====== Description ======
 This set of scripts will parallelize the transfer of a huge directory tree keeping in mind a maximum number of simultaneous transfers. This set of scripts will parallelize the transfer of a huge directory tree keeping in mind a maximum number of simultaneous transfers.
  
 ====== Instructions ====== ====== Instructions ======
 +I suggest you tu launch psync with the following line:
 +<code bash>
 +./psync.sh /path/to/folder
 +</code>
 +Don't launch it with the FINAL SLASH:
 +  * NOP: <del>./psync.sh /path/to/folder/</del>
 +  * YES: ./psync.sh /path/to/folder
 +
 +
 ===== Pre-Reqs ===== ===== Pre-Reqs =====
   * gnu screen   * gnu screen
Line 20: Line 31:
 ==== Code ==== ==== Code ====
  
-<code>+<file bash psync.sh>
 #!/bin/bash #!/bin/bash
 [ ! $1 ] && echo "Usage: $0 /path/to/run" && exit 1 [ ! $1 ] && echo "Usage: $0 /path/to/run" && exit 1
Line 47: Line 58:
     local let MAXPARALEL=20     local let MAXPARALEL=20
  
-    LAUCHRSYNC="/root/autosync/launch_rsync.sh"+    LAUCHRSYNC="$(dirname $0)/launch_rsync.sh"
     local let y=0     local let y=0
     for FOLDER in $(find ${TARGET} -mindepth ${MAXDEPTH} -maxdepth ${MAXDEPTH} -type d) ; do     for FOLDER in $(find ${TARGET} -mindepth ${MAXDEPTH} -maxdepth ${MAXDEPTH} -type d) ; do
Line 75: Line 86:
  
 sync_this ${TARGET} sync_this ${TARGET}
-</code>+</file>
  
 ==== Script Variables ==== ==== Script Variables ====
Line 93: Line 104:
  
 ==== Code ==== ==== Code ====
-<code>+<file bash launch_rsync.sh>
 #!/bin/bash #!/bin/bash
 # launch_rsync.sh # launch_rsync.sh
Line 120: Line 131:
  echo "$RES : ${TARGET}" >> ${LOGDIR}/${RECURSIVE//-/}_TRANSFERS.FAIL  echo "$RES : ${TARGET}" >> ${LOGDIR}/${RECURSIVE//-/}_TRANSFERS.FAIL
 fi fi
-</code>+</file>
 ==== Variables ==== ==== Variables ====
 ^ Variable ^ Description ^ ^ Variable ^ Description ^
linux/parallel_rsync.txt · Last modified: 2022/02/11 11:36 by 127.0.0.1