LINUX - Bechmark Storage with IOZONE

Download the article below "iozone-kossboss.txt" for easy view with the NOTEPAD++, where you can zoom out and zoom in with the CONTROL+MOUSE WHEEL, or turn ON and OFF text wrapping with View menu.

TESTING WITH IOZONE
########################

* Note about article: In the first secion I prefix every command with # so they stand out, in the second section (!!!-----Megatest-----!!!) I dont prefix commands with #, only the comments

Iozone is another good tool to test storage performance with. I tested it from a linux system.

Testing storage1 from a machine called linux1. IP ADDRESS OF storage1 IS 10.1.20.64 

ON linux1 DO THESE COMMANDS TO INSTALL

# cd /

confirm all the available shares, i see backup is one
# smbclient -L 10.1.20.64

# mount //10.1.20.64/Backup /mnt
Password:
just hit enter, blank pw because we didnt set any on wills nas, its all access for everyone

confirm mount
# mount
//10.1.20.64/Backup on /mnt type cifs (rw,relatime,sec=ntlm,unc=\\10.1.20.64\Backup,username=root,uid=0,noforceuid
,gid=0,noforcegid,addr=10.1.20.64,file_mode=0755,dir_mode=0755,nounix,serverino,rsize=16384,wsize=131007,actimeo=1)

Get iozone

# cd /tmp
# wget http://www.iozone.org/src/current/iozone3_414.tar
# tar xvf iozone*
# cd iozone*
# cd src/current
# make
# make linux

It made the iozone file right in /tmp/iozone3_414/src/current
So running "/tmp/iozone3_414/src/current/iozone -h" will show the help

# cd /tmp/iozone3_414/src/current/

To get help
# ./iozone -h

Look at previous output or manpage for meaning of below examples: -a is for all the tests (it doesnt do some tests though as they are useless, you can see it in the excel spreadsheet results you get a square area of 0s where it didnt do the tests, as its a useless area to test) -g 1G sets the max file size, set -g to be bigger then how much ram you have. I have 512MB ram in the machine so I set two times as much 1 G. In machine with 4 gb or ram set it to 8G. -U is for a remount between tests, its useful but it didnt work for me because I dont have the mount specified in /etc/fstab. -f is how you specify what to test. I mounted the remote nas to /mnt/ and i specified a none existant file iozone3 which it will make on the spot. os note that -f /mnt/iozone3 is a file that it will make and this file shouldnt exist but the directory path should exist and it should be pointed at your test device -b is where you want the excel resutls to save. -R or -c also shows the results in text format. To get graphs in the end you need to use your excel skills.

# ./iozone -Rac -g 1G -U /mnt/ -f /mnt/iozone3 -b /tmp/iozone-storage1-result.xls
The -U Remounts from fstab so need to remount it between every test with -U it because our fstab doesnt have any entry for mnt:

I got problems with -U because I dont have fstab with an entry for storage1's share Backup to be mounted to /mnt:
# ./iozone -Rac -g 1G -f /mnt/iozone3 -b /tmp/iozone-storage1-result.xls

Anyways My favorite way to run it is (and I will show this as the first command in the megatest below this section in the article):
# time ./iozone -Rac -g 4G -f /mnt/iozone/iozone3all -b /tests/iozone-results/iozone-storage1a-4Gig-all.xls | tee /tests/iozone-results/iozone-storage1a-4Gig-all.txt

ABOUT RESULTS:
Iozone manual states results are in kilobytes/second but i seriously question that:
When its done all results are in kilobits / second not kilobytes / second... Im still debating that though... Its important to know, it just doesnt make sense if the results were in kilobytes/second all of my speeds are faster then line rate then... I think it actually might be kilobits and its super big number because of the cache...

So to get megabytes/second from the numbers you see on the results: Multiply each value by 0.00012207

In Excel you can convert everything to MB/Sec from the Kilobits/SEcond like this:
HOW TO MUTIPLE CERTAIN CELLS ALL AT ONCE BY ONE NUMBER - this is useful to convert your results to megabytes/second:
2207 to one cell, then copy it, select all the cells you want to convert to megabytes/second then right click and select Paste Special -> Paste Special again and do a Multiply

Actually I ended up constructing a megatest with lots of tests, and refresh command run in between each test. the refresh command is a script that just syncs.

#########################
#########################
#########################
#########################
!!!!-----MEGATEST-----!!!
#########################
#########################
#########################
#########################

* PREQUISITE: have a tests folder for logs thats not part of the destination, and mount the test subject to mnt. Note dont call it tests in your case lol.
* In this section I dont put # as it means comment in bash, and these are scripts, so I put # where its actually a comment, the commands that need to be run are sitting without a # in front

######################################
COPY PASTE FOR linux1 -> storage1 RUN:
######################################

PRE TEST:
=========
mount | egrep cifs
cd /tests/iozone/iozone3_414/src/current
rm -rf /tests/iozone-results/* /mnt/iozone/*
mkdir -p /tests/iozone-results/
mkdir -p /mnt/iozone/

BETWEEN TEST (# touch between.sh; chmod +x between.sh; vim between.sh;): between.sh:::
======================================================================================
#!/bin/bash
sync
rm -rf /mnt/iozone/*
sync
mkdir -p /mnt/iozone/
sync

TEST (# touch megatest.sh; chmod +x megatest.sh; vim megatest.sh;): megatest.sh:::
===================================================================================
#!/bin/bash
THETIME1=`date`
echo "====MEGATEST START: $THETIME1===="
time ./between.sh
# every test
time ./iozone -Rac -g 4G -f /mnt/iozone/iozone3all -b /tests/iozone-results/iozone-storage1a-4Gig-all.xls | tee /tests/iozone-results/iozone-storage1a-4Gig-all.txt
# smallnet builder test
time ./iozone -Rab /tests/iozone-results/iozone-storage1a-smallnet.xls -i 0 -i 1 -+u -f /mnt/iozone/iozone3smallnet -y 64k -q 64k -n 64k -g 4G -z | tee /tests/iozone-results/iozone-storage1a-smallnet.txt; ./between.sh
# google test
time ./iozone -a -B -n 256M -g 1G -R -b /tests/iozone-results/iozone-storage1a-goog.xls -i 0 -i 1 -M -+u -+A 4 -f /mnt/iozone/iozone3goog | tee /tests/iozone-results/iozone-storage1a-goog.txt; ./between.sh
# dbis test
time ./iozone -R -b /tests/iozone-results/iozone-storage1a-dbi1.xls -f /mnt/iozone/iozone3dbi1 | tee /tests/iozone-results/iozone-storage1a-dbi1.txt; ./between.sh
# incremental
time ./iozone -Rb /tests/iozone-results/iozone-storage1a-16g-1M.xls -s 16G -r 1M -f /mnt/iozone/inc16g-1m | tee /tests/iozone-results/iozone-storage1a-16g-1M.txt; ./between.sh
time ./iozone -Rb /tests/iozone-results/iozone-storage1a-16g-1M.xls -s 16G -r 4M -f /mnt/iozone/inc16g-4m | tee /tests/iozone-results/iozone-storage1a-16g-4M.txt; ./between.sh
time ./iozone -Rb /tests/iozone-results/iozone-storage1a-16g-1M.xls -s 16G -r 8M -f /mnt/iozone/inc16g-8m | tee /tests/iozone-results/iozone-storage1a-16g-8M.txt; ./between.sh
time ./iozone -Rb /tests/iozone-results/iozone-storage1a-16g-1M.xls -s 16G -r 16M -f /mnt/iozone/inc16g-16m | tee /tests/iozone-results/iozone-storage1a-16g-16M.txt; ./between.sh
# dbis test mod 2 to 10 cores/threads 10 threads max 1k to 16m record size on 4gb file
time ./iozone -R -b /tests/iozone-results/iozone-storage1a-multi2to10-4g-1k.xls -t 10 -F /mnt/iozone/1 /mnt/iozone/2 /mnt/iozone/3 /mnt/iozone/4 /mnt/iozone/5 /mnt/iozone/6 /mnt/iozone/7 /mnt/iozone/8 /mnt/iozone/9 /mnt/iozone/10  -r 1k -s 4g -l 2 -u 10 | tee /tests/iozone-results/iozone-storage1a-multi2to10-4g-1k.txt; ./between.sh
time ./iozone -R -b /tests/iozone-results/iozone-storage1a-multi2to10-4g-2k.xls -t 10 -F /mnt/iozone/1 /mnt/iozone/2 /mnt/iozone/3 /mnt/iozone/4 /mnt/iozone/5 /mnt/iozone/6 /mnt/iozone/7 /mnt/iozone/8 /mnt/iozone/9 /mnt/iozone/10  -r 2k -s 4g -l 2 -u 10 | tee /tests/iozone-results/iozone-storage1a-multi2to10-4g-2k.txt; ./between.sh
time ./iozone -R -b /tests/iozone-results/iozone-storage1a-multi2to10-4g-4k.xls -t 10 -F /mnt/iozone/1 /mnt/iozone/2 /mnt/iozone/3 /mnt/iozone/4 /mnt/iozone/5 /mnt/iozone/6 /mnt/iozone/7 /mnt/iozone/8 /mnt/iozone/9 /mnt/iozone/10  -r 4k -s 4g -l 2 -u 10 | tee /tests/iozone-results/iozone-storage1a-multi2to10-4g-4k.txt; ./between.sh
time ./iozone -R -b /tests/iozone-results/iozone-storage1a-multi2to10-4g-8k.xls -t 10 -F /mnt/iozone/1 /mnt/iozone/2 /mnt/iozone/3 /mnt/iozone/4 /mnt/iozone/5 /mnt/iozone/6 /mnt/iozone/7 /mnt/iozone/8 /mnt/iozone/9 /mnt/iozone/10  -r 8k -s 4g -l 2 -u 10 | tee /tests/iozone-results/iozone-storage1a-multi2to10-4g-8k.txt; ./between.sh
time ./iozone -R -b /tests/iozone-results/iozone-storage1a-multi2to10-4g-16k.xls -t 10 -F /mnt/iozone/1 /mnt/iozone/2 /mnt/iozone/3 /mnt/iozone/4 /mnt/iozone/5 /mnt/iozone/6 /mnt/iozone/7 /mnt/iozone/8 /mnt/iozone/9 /mnt/iozone/10  -r 16k -s 4g -l 2 -u 10 | tee /tests/iozone-results/iozone-storage1a-multi2to10-4g-16k.txt; ./between.sh
time ./iozone -R -b /tests/iozone-results/iozone-storage1a-multi2to10-4g-32k.xls -t 10 -F /mnt/iozone/1 /mnt/iozone/2 /mnt/iozone/3 /mnt/iozone/4 /mnt/iozone/5 /mnt/iozone/6 /mnt/iozone/7 /mnt/iozone/8 /mnt/iozone/9 /mnt/iozone/10  -r 32k -s 4g -l 2 -u 10 | tee /tests/iozone-results/iozone-storage1a-multi2to10-4g-32k.txt; ./between.sh
time ./iozone -R -b /tests/iozone-results/iozone-storage1a-multi2to10-4g-64k.xls -t 10 -F /mnt/iozone/1 /mnt/iozone/2 /mnt/iozone/3 /mnt/iozone/4 /mnt/iozone/5 /mnt/iozone/6 /mnt/iozone/7 /mnt/iozone/8 /mnt/iozone/9 /mnt/iozone/10  -r 64k -s 4g -l 2 -u 10 | tee /tests/iozone-results/iozone-storage1a-multi2to10-4g-64k.txt; ./between.sh
time ./iozone -R -b /tests/iozone-results/iozone-storage1a-multi2to10-4g-128k.xls -t 10 -F /mnt/iozone/1 /mnt/iozone/2 /mnt/iozone/3 /mnt/iozone/4 /mnt/iozone/5 /mnt/iozone/6 /mnt/iozone/7 /mnt/iozone/8 /mnt/iozone/9 /mnt/iozone/10  -r 128k -s 4g -l 2 -u 10 | tee /tests/iozone-results/iozone-storage1a-multi2to10-4g-128k.txt; ./between.sh
time ./iozone -R -b /tests/iozone-results/iozone-storage1a-multi2to10-4g-256k.xls -t 10 -F /mnt/iozone/1 /mnt/iozone/2 /mnt/iozone/3 /mnt/iozone/4 /mnt/iozone/5 /mnt/iozone/6 /mnt/iozone/7 /mnt/iozone/8 /mnt/iozone/9 /mnt/iozone/10  -r 256k -s 4g -l 2 -u 10 | tee /tests/iozone-results/iozone-storage1a-multi2to10-4g-256k.txt; ./between.sh
time ./iozone -R -b /tests/iozone-results/iozone-storage1a-multi2to10-4g-512k.xls -t 10 -F /mnt/iozone/1 /mnt/iozone/2 /mnt/iozone/3 /mnt/iozone/4 /mnt/iozone/5 /mnt/iozone/6 /mnt/iozone/7 /mnt/iozone/8 /mnt/iozone/9 /mnt/iozone/10  -r 512k -s 4g -l 2 -u 10 | tee /tests/iozone-results/iozone-storage1a-multi2to10-4g-512k.txt; ./between.sh
time ./iozone -R -b /tests/iozone-results/iozone-storage1a-multi2to10-4g-1m.xls -t 10 -F /mnt/iozone/1 /mnt/iozone/2 /mnt/iozone/3 /mnt/iozone/4 /mnt/iozone/5 /mnt/iozone/6 /mnt/iozone/7 /mnt/iozone/8 /mnt/iozone/9 /mnt/iozone/10  -r 1m -s 4g -l 2 -u 10 | tee /tests/iozone-results/iozone-storage1a-multi2to10-4g-1m.txt; ./between.sh
time ./iozone -R -b /tests/iozone-results/iozone-storage1a-multi2to10-4g-2m.xls -t 10 -F /mnt/iozone/1 /mnt/iozone/2 /mnt/iozone/3 /mnt/iozone/4 /mnt/iozone/5 /mnt/iozone/6 /mnt/iozone/7 /mnt/iozone/8 /mnt/iozone/9 /mnt/iozone/10  -r 2m -s 4g -l 2 -u 10 | tee /tests/iozone-results/iozone-storage1a-multi2to10-4g-2m.txt; ./between.sh
time ./iozone -R -b /tests/iozone-results/iozone-storage1a-multi2to10-4g-4m.xls -t 10 -F /mnt/iozone/1 /mnt/iozone/2 /mnt/iozone/3 /mnt/iozone/4 /mnt/iozone/5 /mnt/iozone/6 /mnt/iozone/7 /mnt/iozone/8 /mnt/iozone/9 /mnt/iozone/10  -r 4m -s 4g -l 2 -u 10 | tee /tests/iozone-results/iozone-storage1a-multi2to10-4g-4m.txt; ./between.sh
time ./iozone -R -b /tests/iozone-results/iozone-storage1a-multi2to10-4g-8m.xls -t 10 -F /mnt/iozone/1 /mnt/iozone/2 /mnt/iozone/3 /mnt/iozone/4 /mnt/iozone/5 /mnt/iozone/6 /mnt/iozone/7 /mnt/iozone/8 /mnt/iozone/9 /mnt/iozone/10  -r 8m -s 4g -l 2 -u 10 | tee /tests/iozone-results/iozone-storage1a-multi2to10-4g-8m.txt; ./between.sh
time ./iozone -R -b /tests/iozone-results/iozone-storage1a-multi2to10-4g-16m.xls -t 10 -F /mnt/iozone/1 /mnt/iozone/2 /mnt/iozone/3 /mnt/iozone/4 /mnt/iozone/5 /mnt/iozone/6 /mnt/iozone/7 /mnt/iozone/8 /mnt/iozone/9 /mnt/iozone/10  -r 16m -s 4g -l 2 -u 10 | tee /tests/iozone-results/iozone-storage1a-multi2to10-4g-16m.txt; ./between.sh
# dbis test mod 2 to 10 cores/threads 10 threads max 1k to 16m record size on 16gb file
time ./iozone -R -b /tests/iozone-results/iozone-storage1a-multi2to10-16g-1k.xls -t 10 -F /mnt/iozone/1 /mnt/iozone/2 /mnt/iozone/3 /mnt/iozone/4 /mnt/iozone/5 /mnt/iozone/6 /mnt/iozone/7 /mnt/iozone/8 /mnt/iozone/9 /mnt/iozone/10  -r 1k -s 16g -l 2 -u 10 | tee /tests/iozone-results/iozone-storage1a-multi2to10-16g-1k.txt; ./between.sh
time ./iozone -R -b /tests/iozone-results/iozone-storage1a-multi2to10-16g-2k.xls -t 10 -F /mnt/iozone/1 /mnt/iozone/2 /mnt/iozone/3 /mnt/iozone/4 /mnt/iozone/5 /mnt/iozone/6 /mnt/iozone/7 /mnt/iozone/8 /mnt/iozone/9 /mnt/iozone/10  -r 2k -s 16g -l 2 -u 10 | tee /tests/iozone-results/iozone-storage1a-multi2to10-16g-2k.txt; ./between.sh
time ./iozone -R -b /tests/iozone-results/iozone-storage1a-multi2to10-16g-4k.xls -t 10 -F /mnt/iozone/1 /mnt/iozone/2 /mnt/iozone/3 /mnt/iozone/4 /mnt/iozone/5 /mnt/iozone/6 /mnt/iozone/7 /mnt/iozone/8 /mnt/iozone/9 /mnt/iozone/10  -r 4k -s 16g -l 2 -u 10 | tee /tests/iozone-results/iozone-storage1a-multi2to10-16g-4k.txt; ./between.sh
time ./iozone -R -b /tests/iozone-results/iozone-storage1a-multi2to10-16g-8k.xls -t 10 -F /mnt/iozone/1 /mnt/iozone/2 /mnt/iozone/3 /mnt/iozone/4 /mnt/iozone/5 /mnt/iozone/6 /mnt/iozone/7 /mnt/iozone/8 /mnt/iozone/9 /mnt/iozone/10  -r 8k -s 16g -l 2 -u 10 | tee /tests/iozone-results/iozone-storage1a-multi2to10-16g-8k.txt; ./between.sh
time ./iozone -R -b /tests/iozone-results/iozone-storage1a-multi2to10-16g-16k.xls -t 10 -F /mnt/iozone/1 /mnt/iozone/2 /mnt/iozone/3 /mnt/iozone/4 /mnt/iozone/5 /mnt/iozone/6 /mnt/iozone/7 /mnt/iozone/8 /mnt/iozone/9 /mnt/iozone/10  -r 16k -s 16g -l 2 -u 10 | tee /tests/iozone-results/iozone-storage1a-multi2to10-16g-16k.txt; ./between.sh
time ./iozone -R -b /tests/iozone-results/iozone-storage1a-multi2to10-16g-32k.xls -t 10 -F /mnt/iozone/1 /mnt/iozone/2 /mnt/iozone/3 /mnt/iozone/4 /mnt/iozone/5 /mnt/iozone/6 /mnt/iozone/7 /mnt/iozone/8 /mnt/iozone/9 /mnt/iozone/10  -r 32k -s 16g -l 2 -u 10 | tee /tests/iozone-results/iozone-storage1a-multi2to10-16g-32k.txt; ./between.sh
time ./iozone -R -b /tests/iozone-results/iozone-storage1a-multi2to10-16g-64k.xls -t 10 -F /mnt/iozone/1 /mnt/iozone/2 /mnt/iozone/3 /mnt/iozone/4 /mnt/iozone/5 /mnt/iozone/6 /mnt/iozone/7 /mnt/iozone/8 /mnt/iozone/9 /mnt/iozone/10  -r 64k -s 16g -l 2 -u 10 | tee /tests/iozone-results/iozone-storage1a-multi2to10-16g-64k.txt; ./between.sh
time ./iozone -R -b /tests/iozone-results/iozone-storage1a-multi2to10-16g-128k.xls -t 10 -F /mnt/iozone/1 /mnt/iozone/2 /mnt/iozone/3 /mnt/iozone/4 /mnt/iozone/5 /mnt/iozone/6 /mnt/iozone/7 /mnt/iozone/8 /mnt/iozone/9 /mnt/iozone/10  -r 128k -s 16g -l 2 -u 10 | tee /tests/iozone-results/iozone-storage1a-multi2to10-16g-128k.txt; ./between.sh
time ./iozone -R -b /tests/iozone-results/iozone-storage1a-multi2to10-16g-256k.xls -t 10 -F /mnt/iozone/1 /mnt/iozone/2 /mnt/iozone/3 /mnt/iozone/4 /mnt/iozone/5 /mnt/iozone/6 /mnt/iozone/7 /mnt/iozone/8 /mnt/iozone/9 /mnt/iozone/10  -r 256k -s 16g -l 2 -u 10 | tee /tests/iozone-results/iozone-storage1a-multi2to10-16g-256k.txt; ./between.sh
time ./iozone -R -b /tests/iozone-results/iozone-storage1a-multi2to10-16g-512k.xls -t 10 -F /mnt/iozone/1 /mnt/iozone/2 /mnt/iozone/3 /mnt/iozone/4 /mnt/iozone/5 /mnt/iozone/6 /mnt/iozone/7 /mnt/iozone/8 /mnt/iozone/9 /mnt/iozone/10  -r 512k -s 16g -l 2 -u 10 | tee /tests/iozone-results/iozone-storage1a-multi2to10-16g-512k.txt; ./between.sh
time ./iozone -R -b /tests/iozone-results/iozone-storage1a-multi2to10-16g-1m.xls -t 10 -F /mnt/iozone/1 /mnt/iozone/2 /mnt/iozone/3 /mnt/iozone/4 /mnt/iozone/5 /mnt/iozone/6 /mnt/iozone/7 /mnt/iozone/8 /mnt/iozone/9 /mnt/iozone/10  -r 1m -s 16g -l 2 -u 10 | tee /tests/iozone-results/iozone-storage1a-multi2to10-16g-1m.txt; ./between.sh
time ./iozone -R -b /tests/iozone-results/iozone-storage1a-multi2to10-16g-2m.xls -t 10 -F /mnt/iozone/1 /mnt/iozone/2 /mnt/iozone/3 /mnt/iozone/4 /mnt/iozone/5 /mnt/iozone/6 /mnt/iozone/7 /mnt/iozone/8 /mnt/iozone/9 /mnt/iozone/10  -r 2m -s 16g -l 2 -u 10 | tee /tests/iozone-results/iozone-storage1a-multi2to10-16g-2m.txt; ./between.sh
time ./iozone -R -b /tests/iozone-results/iozone-storage1a-multi2to10-16g-4m.xls -t 10 -F /mnt/iozone/1 /mnt/iozone/2 /mnt/iozone/3 /mnt/iozone/4 /mnt/iozone/5 /mnt/iozone/6 /mnt/iozone/7 /mnt/iozone/8 /mnt/iozone/9 /mnt/iozone/10  -r 4m -s 16g -l 2 -u 10 | tee /tests/iozone-results/iozone-storage1a-multi2to10-16g-4m.txt; ./between.sh
time ./iozone -R -b /tests/iozone-results/iozone-storage1a-multi2to10-16g-8m.xls -t 10 -F /mnt/iozone/1 /mnt/iozone/2 /mnt/iozone/3 /mnt/iozone/4 /mnt/iozone/5 /mnt/iozone/6 /mnt/iozone/7 /mnt/iozone/8 /mnt/iozone/9 /mnt/iozone/10  -r 8m -s 16g -l 2 -u 10 | tee /tests/iozone-results/iozone-storage1a-multi2to10-16g-8m.txt; ./between.sh
time ./iozone -R -b /tests/iozone-results/iozone-storage1a-multi2to10-16g-16m.xls -t 10 -F /mnt/iozone/1 /mnt/iozone/2 /mnt/iozone/3 /mnt/iozone/4 /mnt/iozone/5 /mnt/iozone/6 /mnt/iozone/7 /mnt/iozone/8 /mnt/iozone/9 /mnt/iozone/10  -r 16m -s 16g -l 2 -u 10 | tee /tests/iozone-results/iozone-storage1a-multi2to10-16g-16m.txt; ./between.sh
THETIME2=`date`
echo "====MEGATEST END: $THETIME2===="
echo "Remember it started @ $THETIME1"


#########################
#########################
#########################
#########################
#########################
#########################
#########################
#########################

* Im not the author the article below, I didnt write anything below the ==== divider, its just a copy of a good article, and this is me being legally safe and being honest
EXERPT OF THIS SITE: GOOD READ: http://www.schneideradvconcepts.com/services/document1.html#
* I am not claiming ownership to this site, just making a copy of it, what if there servers went down, and you needed to read this golden information? I have a copy and im not claiming im the auther

=========================
=========================

Benchmarking With IOZone

Contents

[hide]
1 IOZONE
1.1 Preference
1.2 Installation
1.3 Usage
1.4 Testing
1.4.1 Test Setup
1.4.2 Test #1: General Performance
1.4.3 Test #2: NFS Specific Performance
2 Testing with Bonnie
2.1 Testing
3 Mount Options
4 Results
IOZONE

We will get into detail on how to measure I/O Filesystem performance using iozone, a free benchmarking tool for Linux/BSD.

Preference

To test the speed of both the existing EMC and Equalogix SANs, as well as the latency of service from the NFS servers 1,2 and 11,12, I have created two servers (VM) with this software installed:

CENTOS:
192.168.99.174*
192.168.55.65
192.168.44.128
FreeBSD:
192.168.99.173
192.168.55.64*
192.168.44.129

= Network affinity. I have reserved the IP address in IPPlan.
Installation

CentOS: You have to download the .RPM and install it manually using RPM:

cd /tmp
wget http://www.iozone.org/src/current/iozone-3-303.i386.rpm
rpm -ivh iozone-3-303.i386.rpm
FreeBSD: This is included into ports:

cd /usr/ports/benchmarks/iozone
make config
make install
...It's just that easy folks...

Usage

The iozone.org webiste kinda sucks, but there are many other webistes out there that make good use of communication of how to use IOZone. Tests can take up to 4 hours or longer. Run this command with "time" if possible to yield the time it took to perform the test. Add the "&" to the end of the statement to run in the backgroud.

Linux.com
Western Michigan University
Cyberciti
IO Zone Org
Here is a breakdown of the man page:

[root@NFSTestCentos-1 tmp]# /opt/iozone/bin/iozone -h iozone: help mode

   Usage: iozone [-s filesize_Kb] [-r record_size_Kb] [-f [path]filename] [-h]
                 [-i test] [-E] [-p] [-a] [-A] [-z] [-Z] [-m] [-M] [-t children]
                 [-l min_number_procs] [-u max_number_procs] [-v] [-R] [-x] [-o]
                 [-d microseconds] [-F path1 path2...] [-V pattern] [-j stride]
                 [-T] [-C] [-B] [-D] [-G] [-I] [-H depth] [-k depth] [-U mount_point]
                 [-S cache_size] [-O] [-L cacheline_size] [-K] [-g maxfilesize_Kb]
                 [-n minfilesize_Kb] [-N] [-Q] [-P start_cpu] [-e] [-c] [-b Excel.xls]
                 [-J milliseconds] [-X write_telemetry_filename] [-w] [-W]
                 [-Y read_telemetry_filename] [-y minrecsize_Kb] [-q maxrecsize_Kb]
                 [-+u] [-+m cluster_filename] [-+d] [-+x multiplier] [-+p # ]
                 [-+r] [-+t] [-+X] [-+Z] [-+w percent dedupable] [-+y percent_interior_dedup]
                 [-+C percent_dedup_within]

          -a  Auto mode
          -A  Auto2 mode
          -b Filename  Create Excel worksheet file
          -B  Use mmap() files
          -c  Include close in the timing calculations
          -C  Show bytes transferred by each child in throughput testing
          -d #  Microsecond delay out of barrier
          -D  Use msync(MS_ASYNC) on mmap files
          -e  Include flush (fsync,fflush) in the timing calculations
          -E  Run extension tests
          -f filename  to use
          -F filenames  for each process/thread in throughput test
          -g #  Set maximum file size (in Kbytes) for auto mode (or #m or #g)
          -G  Use msync(MS_SYNC) on mmap files
          -h  help
          -H #  Use POSIX async I/O with # async operations
          -i #  Test to run (0=write/rewrite, 1=read/re-read, 2=random-read/write
                3=Read-backwards, 4=Re-write-record, 5=stride-read, 6=fwrite/re-fwrite
                7=fread/Re-fread, 8=random_mix, 9=pwrite/Re-pwrite, 10=pread/Re-pread
                11=pwritev/Re-pwritev, 12=preadv/Re-preadv)
          -I  Use VxFS VX_DIRECT, O_DIRECT,or O_DIRECTIO for all file operations
          -j #  Set stride of file accesses to (# * record size)
          -J #  milliseconds of compute cycle before each I/O operation
          -k #  Use POSIX async I/O (no bcopy) with # async operations
          -K  Create jitter in the access pattern for readers
          -l #  Lower limit on number of processes to run
          -L #  Set processor cache line size to value (in bytes)
          -m  Use multiple buffers
          -M  Report uname -a output
          -n #  Set minimum file size (in Kbytes) for auto mode (or #m or #g)
          -N  Report results in microseconds per operation
          -o  Writes are synch (O_SYNC)
          -O  Give results in ops/sec.
          -p  Purge on
          -P #  Bind processes/threads to processors, starting with this cpu
          -q #  Set maximum record size (in Kbytes) for auto mode (or #m or #g)
          -Q  Create offset/latency files
          -r #  record size in Kb
             or -r #k .. size in Kb
             or -r #m .. size in Mb
             or -r #g .. size in Gb
          -R  Generate Excel report
          -s #  file size in Kb
             or -s #k .. size in Kb
             or -s #m .. size in Mb
             or -s #g .. size in Gb
          -S #  Set processor cache size to value (in Kbytes)
          -t #  Number of threads or processes to use in throughput test
          -T  Use POSIX pthreads for throughput tests
          -u #  Upper limit on number of processes to run
          -U  Mount point to remount between tests
          -v  version information
          -V #  Verify data pattern write/read
          -w  Do not unlink temporary file
          -W  Lock file when reading or writing
          -x  Turn off stone-walling
          -X filename  Write telemetry file. Contains lines with (offset reclen compute_time) in ascii
          -y #  Set minimum record size (in Kbytes) for auto mode (or #m or #g)
          -Y filename  Read  telemetry file. Contains lines with (offset reclen compute_time) in ascii
          -z  Used in conjunction with -a to test all possible record sizes
          -Z  Enable mixing of mmap I/O and file I/O
          -+K Sony special. Manual control of test 8.
          -+m  Cluster_filename   Enable Cluster testing
          -+d  File I/O diagnostic mode. (To troubleshoot a broken file I/O subsystem)
          -+u  Enable CPU utilization output (Experimental)
          -+x # Multiplier to use for incrementing file and record sizes
          -+p # Percentage of mix to be reads
          -+r Enable O_RSYNC|O_SYNC for all testing.
          -+t Enable network performance test. Requires -+m 
          -+n No retests selected.
          -+k Use constant aggregate data set size.
          -+q Delay in seconds between tests.
          -+l Enable record locking mode.
          -+L Enable record locking mode, with shared file.
          -+B Sequential mixed workload.
          -+D Enable O_DSYNC mode.
          -+A #  Enable madvise. 0 = normal, 1=random, 2=sequential
                                 3=dontneed, 4=willneed
          -+V Enable shared file. No locking.
          -+X Enable short circuit mode for filesystem testing ONLY
              ALL Results are NOT valid in this mode.
          -+Z Enable old data set compatibility mode. WARNING.. Published
              hacks may invalidate these results and generate bogus, high
              values for results.
          -+w ## Percent of dedup-able data in buffers.
          -+y ## Percent of dedup-able within & across files in buffers.
          -+C ## Percent of dedup-able within & not across files in buffers.

YUCK!
Testing

Test Setup

To test the filesystem performance of the EMC, old NFS and new NFS, I went ahead and made the following test mount directories:

mkdir /test
mkdir /test/emc
mkdir /test/nfs11
mkdir /test/nfs1 
mkdir /test/nfs12
mkdir /test/nfs2
This should give us at least three specific mount directories:

1. EMC - Test the EMC speed as a Benchmark for the other two. Mount that share up on /test/emc
mount <emcstuff> /test/emc
2. NFS1 - Test the "old" SAN speed and compare it to the "new" SAN. Mount that share up on /test/nfs1. Optional: Mount a share from NFS2 up in /test/nfs2.
mount 192.168.44.230:/export/es-test /test/test2
3. NFS11 - Test the "new" SAN speed and compare it to the other statistics. Mount that share up on /test/nfs11. This is a cluster, so you should only mount the VIP between the two cluster hosts.
On VLAN55

mount 192.168.55.56:/zfstest /test/nfs11
On VLAN99

mount 192.168.99.83:/zfstest /test/nfs11
NOTE

On BSD servers, use this command to mount up NFS shares from the Solaris Cluster, or RPC Mounter will fail:

mount_nfs -o rw,tcp,intr,soft,bg 192.168.55.56:/zfstest /test/nfs11
Test #1: General Performance

Test #1

time /opt/iozone/bin/iozone -R -a -g 1G | tee -a /tmp/iozone-nfs1-1.txt &
The line above will do a test that is run in throughput mode with a single thread, read and write with a 4 Gigabyte file size in the /test/nfs1 directory, dumping results into an Excel Compatible spreadsheet table. Then we "pipe" this to a file onto the guest (server you are running iozone from) to a text file that we will later generate some graphs with.

Flags -R = Excel Spreadsheet
-a = Automatic mode
-g = Extend the maximum file size to be twice your system's main memory size, that size defined as 4G.

Test #2: NFS Specific Performance

Test #2


/opt/iozone/bin/iozone -R -a -c

------------------From Another Site--------------------------
/opt/iozone/bin/iozone -R -r 4 -s 60m -l 1 -u 50 -i 0 -i 1 -i 8 -+p 60 -+m nodelist -b excel_noflush.wks -C -O | tee -a /tmp/iozone-nfs1-2.txt &
--------------------------------------------------------------

The One I am using:

iozone -Rac -g 1G -U /test/nfs11 -f /test/nfs11/iozone3 -b /tmp/iozone-nfs11-wload-1.xls
Copy the .XLS spreadsheet onto your workstation and make some heads or tails from the data.

Testing with Bonnie

I ran some tests using another app called Bonnie, which provides simpler stats on the disks IO performance.

Testing

ON each server:

bonnie -d <directory> -s <size> -m <machine name>
Mount Options

I used five difference mount options to determine wether or not the performance gain was worth the effort:

BSD:

Default = “-o -c”
EO = rw,async,-d,-U,-3,-s,-i,noatime,-c
EO4 = rw,async,-r=4098,-w=4098,-d,-U,-3,-s,-i,noatime,-c
EO8 = rw,async,-r=8196,-w=8196,-d,-U,-3,-s,-i,noatime,-c
EO16 = rw,async,-r=16392,-w=16392,-d,-U,-3,-s,-i,noatime,-c
Cent:

Default = "rw"
EO = rw,async,noatime,nfsvers=3,udp,soft,intr
EO4 = rw,rsize=4098,wsize=4098,async,noatime,nfsvers=3,udp,soft,intr
EO8 = rw,rsize=8196,wsize=8196,async,noatime,nfsvers=3,udp,soft,intr 
EO16 = rw,rsize=16392,wsize=16392,async,noatime,nfsvers=3,udp,soft,intr
Results

I have attached a spreadsheet to this WIKI as to the results of my findings:

Media:NFS_Performance_Testing.xls

Personal tools

 
This page was last modified on 21 October 2009, at 16:38.
This page has been accessed 35 times.
Content and Images Property Of Schneider Advanced Concepts, LLC.

© 2011 Schneider Advanced Concepts, LLC.
ċ
iozone-kossboss.txt
(29k)
koss boss,
Jun 16, 2013, 6:29 AM
Comments