linux:parallel_rsync
Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revision | ||
parallel_rsync [2012/12/06 17:55] – dodger | linux:parallel_rsync [2022/02/11 11:36] (current) – external edit 127.0.0.1 | ||
---|---|---|---|
Line 1: | Line 1: | ||
+ | ====== [SCRIPT] psync (parallel rsync) ====== | ||
+ | |||
====== Description ====== | ====== Description ====== | ||
This set of scripts will parallelize the transfer of a huge directory tree keeping in mind a maximum number of simultaneous transfers. | This set of scripts will parallelize the transfer of a huge directory tree keeping in mind a maximum number of simultaneous transfers. | ||
====== Instructions ====== | ====== Instructions ====== | ||
+ | I suggest you tu launch psync with the following line: | ||
+ | <code bash> | ||
+ | ./psync.sh / | ||
+ | </ | ||
+ | Don't launch it with the FINAL SLASH: | ||
+ | * NOP: < | ||
+ | * YES: ./psync.sh / | ||
+ | |||
+ | |||
===== Pre-Reqs ===== | ===== Pre-Reqs ===== | ||
* gnu screen | * gnu screen | ||
Line 20: | Line 31: | ||
==== Code ==== | ==== Code ==== | ||
- | <code> | + | <file bash psync.sh> |
#!/bin/bash | #!/bin/bash | ||
[ ! $1 ] && echo " | [ ! $1 ] && echo " | ||
Line 47: | Line 58: | ||
local let MAXPARALEL=20 | local let MAXPARALEL=20 | ||
- | LAUCHRSYNC=" | + | LAUCHRSYNC=" |
local let y=0 | local let y=0 | ||
for FOLDER in $(find ${TARGET} -mindepth ${MAXDEPTH} -maxdepth ${MAXDEPTH} -type d) ; do | for FOLDER in $(find ${TARGET} -mindepth ${MAXDEPTH} -maxdepth ${MAXDEPTH} -type d) ; do | ||
Line 75: | Line 86: | ||
sync_this ${TARGET} | sync_this ${TARGET} | ||
- | </code> | + | </file> |
==== Script Variables ==== | ==== Script Variables ==== | ||
Line 86: | Line 97: | ||
===== launch_rsync.sh ===== | ===== launch_rsync.sh ===== | ||
+ | ==== Description ==== | ||
+ | This script will: | ||
+ | * Launch rsync non-parallel or parallel | ||
+ | * Log the exit code of rsync to know if everything gones fine or not | ||
+ | |||
+ | |||
==== Code ==== | ==== Code ==== | ||
- | <code> | + | <file bash launch_rsync.sh> |
#!/bin/bash | #!/bin/bash | ||
# launch_rsync.sh | # launch_rsync.sh | ||
Line 114: | Line 131: | ||
echo "$RES : ${TARGET}" | echo "$RES : ${TARGET}" | ||
fi | fi | ||
- | </ | + | </file> |
+ | ==== Variables ==== | ||
+ | ^ Variable ^ Description ^ | ||
+ | |<code>RECURSIVE=$(echo $1 | tr ' | ||
+ | |< | ||
+ | |< | ||
+ | |< | ||
+ | |< | ||
+ | |< | ||
linux/parallel_rsync.1354816509.txt.gz · Last modified: 2012/12/06 17:55 by dodger