PROGRESS SCRIPT 4 CYGWIN

PROGRESS EVEN BETTER THEN ABOVE DOES THE MATH FOR YOU INSTEAD OF WOLFRAMALPHA
################################################################################


EXAMPLE OUTPUT:
===============

Sat, Jul 27, 2013 11:18:12 PM

===DF===
Filesystem      Size  Used Avail Use% Mounted on
E:              3.7T  2.9T  809G  79% /cygdrive/e
F:              3.7T  726G  3.0T  20% /cygdrive/f
Filesystem      1K-blocks       Used  Available Use% Mounted on
E:             3906983932 3059552052  847431880  79% /cygdrive/e
F:             3907010372  761199672 3145810700  20% /cygdrive/f

===LAST 6 LINES OF THE JOB===
...some personal docs, you dont need to see what files I am copying...

===JOB INFO:===
TARGET: 743360 MB, 725.938 GB
SOURCE: 2987844 MB, 2917.82 GB
REMAINING 2244484 MB, 2191.88 GB - PERCENT DONE: 24.8795 % <- complete if close to 100%

===SPEED AND TIME ANALYSIS ESTIMATIONS BETWEEN LAST 2 RUNS===
IN THE LAST [2 SECS] [28672 KBYTES] WERE TRANSFERED @ A RATE OF [14336 KB/s = 14 MB/s = 114688 Kb/s = 112 Mb/s]

--TIME REPORTS IN SECONDS--
ETA IS [160320 s - calc from (KB)/(KBPS)] = [160320 s - calc from (MB)/(MBPS)]
RUNNING TIME IS [53097.1 s - calc from (KB)/(KBPS)] = [53097.1 s - calc from (MB)/(MBPS)]

--TIME REPORTS IN HOURS--
ETA IS [44.5333 h - calc from (KB)/(KBPS)] = [44.5333 h - calc from (MB)/(MBPS)]
RUNNING TIME IS [14.7492 h - calc from (KB)/(KBPS)] = [14.7492 h - calc from (MB)/(MBPS)]

===CPU LOAD AVERAGES:===
0.00 0.00 0.00 1/8

===MEMORY USAGE===
MemTotal:       16717684 kB
MemFree:        13040436 kB
SwapTotal:       3145728 kB
SwapFree:        3040080 kB

===WHATS STILL RUNNING===
IS c[p] -Rnv STILL RUNNING: NO
IS rsyn[c] STILL RUNNING: YES
IS ta[r] STILL RUNNING: NO


HOW I DID THAT:
===============

The script full path: /cygdrive/e/1DailyBackupOfDocuments/Simba2UpBack/e2f-OPTIONAL-progress.sh

Whats this? Show the best type of progress, it has to be run with a watch or a while loop to get best results make the time difference about 2 to 3 seconds minimum, 10 seconds for better estimates.

How I run this with the script above, Note this works best on the first clone when the TARGET starts off completely empty.

You can manually run it like this:
cd /cygdrive/e/1DailyBackupOfDocuments/Simba2UpBack/
./e2f-OPTIONAL-progress.sh
then wait 2 seconds
./e2f-OPTIONAL-progress.sh
then wait 2 seconds
./e2f-OPTIONAL-progress.sh

Or:
while true; do echo =============; ./e2f-OPTIONAL-progress.sh; sleep 2; done

TO GET THIS: Make script below
Make sure this file also ends with LF and not CR+LF
TO DO THAT: With notepad++ go to Edit->EOL Conversion->UNIX

e2f-OPTIONAL-progress.sh
========================

#!/bin/bash
# this script measures progress just modify a couple of variables to match your backup job
# Also it generates to small temp files /tmp/time1 and /tmp/target1
# this works best when the target starts off completely empty - its okay if you started this late thru the job
######### JUST MODIFY THIS SECTION  ############
SOURCE_PHRASE_FROM_DF="drive/e" # MUST CHANGE THIS FOR YOUR JOB
TARGET_PHRASE_FROM_DF="drive/f" # MUST CHANGE THIS FOR YOUR JOB
SCRIPT1="c[p] -Rnv" # modify with the output of "ps" or "ps aux", usually leave it alone 
SCRIPT2="rsyn[c]" # modify with the output of "ps" or "ps aux", usually leave it alone 
SCRIPT3="ta[r]" # modify with the output of "ps" or "ps aux", usually leave it alone 
FILE_TO_TAIL="nohup.out" # MUST CHANGE THIS FOR YOUR JOB, USE FULL PATH TO FILE, I GOT AWAY WITH RELATIVE PATH BECAUSE THE NOHUP.OUT IS IN THE SAME DIRECTORY AS I RAN THIS FILE FROM
###### THE PROGRAM SECTION USUALLY DOESNT NEED MODIFICATION #########
TIME1=`cat /tmp/time1`
TARGET1=`cat /tmp/target1`
date
echo
echo "===DF==="
(df -h && df) | egrep -i "$SOURCE_PHRASE_FROM_DF|$TARGET_PHRASE_FROM_DF|file"
echo
echo "===LAST 6 LINES OF THE JOB==="
tail -n6 $FILE_TO_TAIL; 
echo
echo
echo "===JOB INFO:==="
TARGET_KB=`df -P | egrep -i "$TARGET_PHRASE_FROM_DF" | awk '{print $3}'`
TARGET=`df -Pm | egrep -i "$TARGET_PHRASE_FROM_DF" | awk '{print $3}'`
TARGET_GB=`echo $TARGET | awk '{x=$0; y=x/1024; print y;}'`
echo "TARGET: $TARGET MB, $TARGET_GB GB"
SOURCE_KB=`df -P | egrep -i "$SOURCE_PHRASE_FROM_DF" | awk '{print $3}'`
SOURCE=`df -Pm | egrep -i "$SOURCE_PHRASE_FROM_DF" | awk '{print $3}'`
SOURCE_GB=`echo $SOURCE | awk '{x=$0; y=x/1024; print y;}'`
echo "SOURCE: $SOURCE MB, $SOURCE_GB GB"
DIFF_KB=`echo $SOURCE_KB $TARGET_KB | awk '{x=$1; y=$2; print x-y;}'`
DIFF=`echo $SOURCE $TARGET | awk '{x=$1; y=$2; print x-y;}'`
DIFF_GB=`echo $DIFF | awk '{x=$0; y=x/1024; print y;}'`
PERC=`echo $SOURCE $TARGET | awk '{x=$1; y=$2; print y/x*100;}'`
echo "REMAINING $DIFF MB, $DIFF_GB GB - PERCENT DONE: $PERC % <- complete if close to 100%"
echo
echo "===SPEED AND TIME ANALYSIS ESTIMATIONS BETWEEN LAST 2 RUNS==="
TARGET2=$TARGET_KB
SIZE_DIFF=`echo $TARGET2 $TARGET1 | awk '{x=$1; y=$2; print x-y;}'`
TARGET1=$TARGET2
TIME2=`date +%s` # CURRENT TIME IN SECONDS SINCE 1970
TIME_DIFF=`echo $TIME2 $TIME1 | awk '{x=$1; y=$2; print x-y;}'`
TIME1=$TIME2 # CURRENT TIME BECOMES OLD TIME
SPEED_KBYTES=`echo $SIZE_DIFF $TIME_DIFF | awk '{x=$1; y=$2; print x/y;}'` 2> /dev/null
SPEED_KBITS=`echo $SPEED_KBYTES | awk '{x=$1; print x*8;}'`
SPEED_MBYTES=`echo $SPEED_KBYTES | awk '{x=$1; print x/1024;}'`
SPEED_MBITS=`echo $SPEED_MBYTES | awk '{x=$1; print x*8;}'`
ETA_SECONDS=`echo $DIFF_KB $SPEED_KBYTES | awk '{x=$1; y=$2; print x/y;}'` 2> /dev/null
TOTAL_SECONDS=`echo $TARGET_KB $SPEED_KBYTES | awk '{x=$1; y=$2; print x/y;}'` 2> /dev/null
ETA_SECONDS_M=`echo $DIFF $SPEED_MBYTES | awk '{x=$1; y=$2; print x/y;}'` 2> /dev/null
TOTAL_SECONDS_M=`echo $TARGET $SPEED_MBYTES | awk '{x=$1; y=$2; print x/y;}'` 2> /dev/null
ETA_SECONDS_H=`echo $ETA_SECONDS | awk '{x=$1; print x/3600;}'` 2> /dev/null
TOTAL_SECONDS_H=`echo $TOTAL_SECONDS | awk '{x=$1; print x/3600;}'` 2> /dev/null
ETA_SECONDS_M_H=`echo $ETA_SECONDS_M | awk '{x=$1; print x/3600;}'` 2> /dev/null
TOTAL_SECONDS_M_H=`echo $TOTAL_SECONDS_M | awk '{x=$1; print x/3600;}'` 2> /dev/null
echo "IN THE LAST [$TIME_DIFF SECS] [$SIZE_DIFF KBYTES] WERE TRANSFERED @ A RATE OF [$SPEED_KBYTES KB/s = $SPEED_MBYTES MB/s = $SPEED_KBITS Kb/s = $SPEED_MBITS Mb/s]"
echo
echo "--TIME REPORTS IN SECONDS--"
echo "ETA IS [$ETA_SECONDS s - calc from (KB)/(KBPS)] = [$ETA_SECONDS_M s - calc from (MB)/(MBPS)]"
echo "RUNNING TIME IS [$TOTAL_SECONDS s - calc from (KB)/(KBPS)] = [$TOTAL_SECONDS_M s - calc from (MB)/(MBPS)]"
echo
echo "--TIME REPORTS IN HOURS--"
echo "ETA IS [$ETA_SECONDS_H h - calc from (KB)/(KBPS)] = [$ETA_SECONDS_M_H h - calc from (MB)/(MBPS)]"
echo "RUNNING TIME IS [$TOTAL_SECONDS_H h - calc from (KB)/(KBPS)] = [$TOTAL_SECONDS_M_H h - calc from (MB)/(MBPS)]"
echo
echo "===CPU LOAD AVERAGES:==="
cat /proc/loadavg;
echo
echo "===MEMORY USAGE==="
cat /proc/meminfo | egrep -i "mem|swap"
echo
echo "===WHATS STILL RUNNING===" 
echo -e "IS $SCRIPT1 STILL RUNNING:" `if ps aux | egrep -qi "$SCRIPT1" ; then echo "YES"; else echo "NO"; fi`
echo -e "IS $SCRIPT2 STILL RUNNING:" `if ps aux | egrep -qi "$SCRIPT2" ; then echo "YES"; else echo "NO"; fi`
echo -e "IS $SCRIPT3 STILL RUNNING:" `if ps aux | egrep -qi "$SCRIPT3" ; then echo "YES"; else echo "NO"; fi`
echo $TIME1 > "/tmp/time1"
echo $TARGET1 > "/tmp/target1"
Comments