LINUX - Guide: how to make raid 55 with encryption + progress-bars, luks encryption + more

MAKE RAID 55 OUT OF 12 DISKS + luks encryptions + progress bars + troubleshooting Raid + more
#############################################################################################

ABSTRACT CONCEPT: Going to make four raid 5s and those will act as "disks"/members to a big raid 5. For one test I will lay ext3 on it, and then for another I will make an encrypted system. I will run some disk speeds etc. All this on my Virtual Machine. NOTE THIS WILL VERY VERY SLOW DUE TO THE LAYERS AND LAYERS OF ENCRYPTION. RAID 55 IN REALWORLD IS PROBABLY VERY UNFEASABLE AND NOT USEABLE - HOWEVER I WROTE THIS TO SHOW YOU IT EXISTS AND HOW TO MAKE IT.

==TOPICS YOU WILL LEARN:==
* make more loopback devices then are given by the system
* Making raids
* Making hybrid raids
* make a member of raid not a spare and making auto-read-only go away - without hurting data - if there was any on it at least, in this example there was no data but if there was i show you how to make a spare active without hurting data and with that randomly the auto-read-only status went away
* progress bars with dd by using "pv" and "bar"
* filesystem checks with ext2,3,4
* doing read and write speed test
* Encrypting a device block level
* Changing password/passphrase on encrypted device

==MAP OF THE 2 PATHS ILL TAKE:==
* You can look at these as layers too from physical on the left to logical on the right
Disks->Loopback Image->4 Raids5 ->Hybrid Raid5->ext3
Disks->Loopback Image->4 Raids5 ->Hybrid Raid5->Encryption Block Level->ext4

==SIDE NOTE:==
All of the commands that you should type are prefixed with # Thats if you want to follow this guide. I dont know why you would its just a hypothetical turned to a possibility turned to a means turned to an I did it. I guess you can get the I did it as well by following this. Also I hope I didnt leave out any of the little commands - like "cd" ing into the right directory - If I did forgive me and let your logic figure it out.

################
===THE SYSTEM===
################

Here are the stats of my machine.

# cat /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 42
model name      : Intel(R) Core(TM) i5-2400 CPU @ 3.10GHz
stepping        : 7
cpu MHz         : 2975.309
cache size      : 6144 KB
fdiv_bug        : no
hlt_bug         : no
f00f_bug        : no
coma_bug        : no
fpu             : yes
fpu_exception   : yes
cpuid level     : 5
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx rdtscp lm constant_tsc up pni monitor ssse3 lahf_lm
bogomips        : 5950.61
clflush size    : 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:

Not much memory but no worries

# cat /proc/meminfo
MemTotal:         384332 kB

I have some Swap Space

# swapon -s
Filename                                Type            Size    Used    Priority
/dev/sda5                               partition       392188  48792   -1
/dev/sdb1                               partition       979928  0       -2

I have 2 disk sda and sdb. SDA is the root its small, its partitioned into I cant remember how many partitions. But here is the output - Its 3 partitions okay.. :-)

cat /proc/partitions
major minor  #blocks  name

   8        0    8388608 sda
   8        1    7993344 sda1
   8        2          1 sda2
   8        5     392192 sda5
   8       16    8388608 sdb
   8       17     979933 sdb1
   8       18    7405965 sdb2
   
# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda1             7.6G  4.7G  2.5G  66% /
tmpfs                 188M     0  188M   0% /lib/init/rw
udev                  183M  216K  183M   1% /dev
tmpfs                 188M     0  188M   0% /dev/shm
/dev/sdb2             7.1G  4.1G  2.4G  64% /vol

I will make the raid 5s out of images that will lay in disk 

#########################
===BUILDING THE RAID55===
#########################

===WHAT IS THIS?===
- we are going to make it out of disk images
- make 12 image files filled with zero bits for 300 MB - that will emulate a 300 MB disk
- make a loopback device out of them with losetup
- since linux only comes with 8 loop backs we need a few more
- Then we make the mdadm commands from loop devices to make 4 raid 5s
- then we make a raid 5 out of raids (That are also raid 5). These raid out of raids are called hybrid raids.

===INTERPRETATION:===
RAID 55
RAID xy
Y is the first raid
X is the raid made out of the raids

===THE STEPS===

We are going to make 12 mini disk emulations with empty image files - like isos and use loopbacks to make them  considered devices - (I heard though mdadm doesnt need them to be loops - I could of just fed the image files to mdadm - oh well - good thought Ill try that another time but wont write an article on it). Loopbacks are used to mount cdrom images etc. And also my favorite use is testing filesystems, big volumes with lvm, raid with mdadm, encryption with cryptosetup, etc - so basically testing block layer stuff and above.

We need 5 more loopback devices but linux comes with only 8 (0 thru 7) i need to have loopback devices to go up to 12

NOTE we are not using loop0 because I wanted the count to start at 1.

First see what loopbacks you need to create in my case it only went up to 7 so i needed to make 8,9,10,11, and 12:
# ls -l /dev/loop*

MAKE THEM:
# mknod -m660 /dev/loop8 b 7 8
# mknod -m660 /dev/loop9 b 7 9
# mknod -m660 /dev/loop10 b 7 10
# mknod -m660 /dev/loop11 b 7 11
# mknod -m660 /dev/loop12 b 7 12

THE HAVE THE WRONG OWNERSHIP OF root.root (user.group) SO CHANGE IT TO MATCH THE ORIGINAL LOOPBACK WHICH IS root.disk:
# chown root.disk /dev/loop8
# chown root.disk /dev/loop9
# chown root.disk /dev/loop10
# chown root.disk /dev/loop11


===MAKE LOOPBACK DEVICES===
First I will make the loopback devices with dd its nice to get a progress bar - the program for that is bar. Have to download the .deb for it:

# cd /tmp 
# apt-get update
# apt-get wget
# wget http://downloads.sourceforge.net/clpbar/bar_1.10.9_i386.deb?modtime=1183036896&big_mirror=0

Get the file name with "ls" note my filename literally was like this:

# dpkg -i bar_1.10.9_i386.deb?modtime=1183036896&big_mirror=0
# mkdir -p /vol/raid
# cd /vol/raid

Im going to make a for loop that makes the image files and makes them into loopback devices with progress bars so we can see how far we are with the process:

# for i in 1 2 3 4 5 6 7 8 9 10 11 12; do echo "===ab$i===" && dd if=/dev/zero bs=1M count=300 | bar -s 300m -of ab$i && losetup /dev/loop$i ab$i;done;

TRANSLATION OF THAT INTO ENGLISH:
-NOTE DONT DO THESE COMMANDS THATS WHY I DIDNT INCLUDE A # WITH IT, ITS JUST AN EXPLANATION OF THE FOR LOOP IN ENGLISH-
This is what happens for the first iteration of that loop. After this it repeats 11 more times to make the rest:
echo "===ab1===", THIS IS FOR A TEXT HEADER SO I KNOW ITS WORKING ON Ith ITERATION
dd if=/dev/zero/ bs=1M count=300 | bar -s 300m -of ab1, THIS IS TO MAKE A 300 MEG FILE WITH A PROGRESS BAR THE FILE WILL JUST BE ALL 0 BITS
losetup /dev/loop1 /ab1, THIS IS TO ASSOCIATE THE IMAGE WITH A LOOPBACK DEVICE - A NECESSARY STEP FOR IT TO BE A DEVICE (but not too. necessary for mdadm - but I wont go into that - because I already did above in some parenthsis sets.)
echo "===ab2==="
dd if=/dev/zero/ bs=1M count=300 | bar -s 300m -of ab2
losetup /dev/loop2 /ab2
echo "===ab3==="
dd if=/dev/zero/ bs=1M count=300 | bar -s 300m -of ab3
losetup /dev/loop3 /ab3

ECTERA... ECTERA... YOU GET THE IDEA REPEATS UNTIL LOOPBACK 12... 

===MAKE RAIDS (four RAID5s)===
# mdadm -Cv -c64 -l5 -n3 -e1.2 /dev/md1 /dev/loop1 /dev/loop2 /dev/loop3
# mdadm -Cv -c64 -l5 -n3 -e1.2 /dev/md2 /dev/loop4 /dev/loop5 /dev/loop6
# mdadm -Cv -c64 -l5 -n3 -e1.2 /dev/md3 /dev/loop7 /dev/loop8 /dev/loop9
# mdadm -Cv -c64 -l5 -n3 -e1.2 /dev/md4 /dev/loop10 /dev/loop11 /dev/loop12

===MAKE HYBRID RAID 55===
# mdadm -Cv -c128 -l5 -n4 -e1.2 /dev/md0 /dev/md1 /dev/md2 /dev/md3 /dev/md4

===CONFIRM===
# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md0 : active (auto-read-only) raid5 md4[4](S) md3[2] md2[1] md1[0]
      1842624 blocks super 1.2 level 5, 64k chunk, algorithm 2 [4/3] [UUU_]

md4 : active raid5 loop12[3] loop11[1] loop10[0]
      614272 blocks super 1.2 level 5, 64k chunk, algorithm 2 [3/2] [UU_]
      [=====>...............]  recovery = 25.0% (76800/307136) finish=2.6min speed=1449K/sec

md3 : active raid5 loop9[3] loop8[1] loop7[0]
      614272 blocks super 1.2 level 5, 64k chunk, algorithm 2 [3/2] [UU_]
      [======>..............]  recovery = 31.1% (96000/307136) finish=2.1min speed=1609K/sec

md2 : active raid5 loop6[3] loop5[1] loop4[0]
      614272 blocks super 1.2 level 5, 64k chunk, algorithm 2 [3/2] [UU_]
      [=======>.............]  recovery = 38.3% (118000/307136) finish=1.7min speed=1764K/sec

md1 : active raid5 loop3[3] loop2[1] loop1[0]
      614272 blocks super 1.2 level 5, 64k chunk, algorithm 2 [3/2] [UU_]
      [=========>...........]  recovery = 45.3% (139504/307136) finish=1.4min speed=1893K/sec

unused devices: <none>

FAIL!! BUT NO WORRIES WE CAN FIX
NOTICE WITH MD0 THE RAID WE ARE CONCERNED WITH DISK MARKED AS SPARE AND ALSO auto-read-only

===MAKE SPARE AN ACTIVE MEMBER AND NOT A SPARE===

SO STOP RAID AND RECREATE WITH --assume-clean (if had data on it would recreate the raid and not hurt the data - make sure device order is the same, if not and you do an add - add causes resync - you will destroy the data - BUT if you do assume-clean and put wrong order of members just try again with another assume-clean, as long as you dont "--add" a drive you will not resync so data will not get hurt, Just cant resync with wrong order.) Anyhow..

# mdadm -S /dev/md0
# mdadm -Cv /dev/md0  --assume-clean --level=5 --chunk=128 --raid-devices=4 /dev/md1 /dev/md2 /dev/md3 missing
 
# mdadm -a /dev/md0 /dev/md4

cat /proc/mdstat
(or)
watch -n2 cat /proc/mdstat
(or)
watch -n2 "(cat /proc/mdstat | grep recovery)"

cat /proc/mdstat

LOOK ITS ALL FIXED AND ITS NOT AUTO READ ONLY

Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md0 : active raid5 md4[4] md3[2] md2[1] md1[0]
      1842432 blocks super 1.2 level 5, 128k chunk, algorithm 2 [4/4] [UUUU]

md4 : active raid5 loop12[3] loop11[1] loop10[0]
      614272 blocks super 1.2 level 5, 64k chunk, algorithm 2 [3/3] [UUU]

md3 : active raid5 loop9[3] loop8[1] loop7[0]
      614272 blocks super 1.2 level 5, 64k chunk, algorithm 2 [3/3] [UUU]

md2 : active raid5 loop6[3] loop5[1] loop4[0]
      614272 blocks super 1.2 level 5, 64k chunk, algorithm 2 [3/3] [UUU]

md1 : active raid5 loop3[3] loop2[1] loop1[0]
      614272 blocks super 1.2 level 5, 64k chunk, algorithm 2 [3/3] [UUU]
 
WIN!!!
 
NOTE THE SYSTEM IS CHILL:

# w
 06:31:37 up 17 days,  1:15,  4 users,  load average: 0.28, 2.18, 2.03
USER     TTY      FROM              LOGIN@   IDLE   JCPU   PCPU WHAT
root     pts/1    172.20.18.115:S. 00:01    3:13m  0.86s  0.85s /bin/bash
root     pts/4    172.20.18.115:S. 00:44    5:35m  0.01s  0.01s /bin/bash
root     pts/5    172.20.18.115:S. 00:46    3:03m  0.02s  0.02s /bin/bash
root     pts/6    172.20.18.115:S. 03:28    0.00s  0.12s  0.00s w

################################
===2 PATHS: ext3 AND dm-crypt===
################################

##########
---EXT3---
##########

# mkfs.ext3 /dev/md0
# mkdir /mnt/r55
# mount /dev/md0 /mnt/r55
# df -h | grep -i "filesys\|md0"
Filesystem            Size  Used Avail Use% Mounted on
/dev/md0              1.8G   35M  1.7G   3% /mnt/r55

Very slow build but it works :)
Here are the numbers :-)

--TEST SPEED WITH EXT3--

# cd /mnt/r55

-WRITE TEST:-

ZEROS DATA:
# dd if=/dev/zero of=testfile0 bs=8k count=10000; sync;
Hit CONTROL-C after 5 seconds to get results
30703616 bytes (31 MB) copied, 7.94884 s, 3.9 MB/s

RANDOM DATA:
# dd if=/dev/urandom of=testfileR bs=8k count=10000; sync;
Hit CONTROL-C after 5 seconds to get results
65994752 bytes (66 MB) copied, 21.8919 s, 3.0 MB/s

-READ TEST:-

ZEROS DATa:
# dd if=testfile0 of=/dev/null
29368320 bytes (29 MB) copied, 0.780934 s, 37.6 MB/s

RANDOM DATA:
dd if=testfileR of=/dev/null 
30842880 bytes (31 MB) copied, 1.05709 s, 29.2 MB/s

OTHER READ TEST:
# hdparm -Tt /dev/md0
 Timing cached reads:   16280 MB in  2.00 seconds = 8158.98 MB/sec
 Timing buffered disk reads:  42 MB in  3.13 seconds =  13.42 MB/sec
# hdparm -Tt --direct /dev/md0
 Timing O_DIRECT cached reads:   6928 MB in  2.00 seconds = 3466.79 MB/sec
 Timing O_DIRECT disk reads:  58 MB in  3.30 seconds =  17.55 MB/sec
 
Now to prepare for next section with cryptography:
First remove the other ext3
# cd /
# umount /dev/dm0

################################
---CRYPTO-SYSTEM: BLOCK LEVEL---
################################

Citation 1: http://www.cyberciti.biz/hardware/howto-linux-hard-disk-encryption-with-luks-cryptsetup-command/
Citation 2: http://www.markus-gattol.name/ws/dm-crypt_luks.html

This is block level encryption so it will take /dev/md0 and encrypt it, then when we decrypt it it will give us a new device with a name of our choosing "crypt55" in this example. Then we can lay a file system on top of it, but before that write zeros to it just to ensure healthy encryption.

The crypt setup starts here
# apt-get install cryptsetup

Default options for cryptsetup -y for verify passphrase so we know we are typing the right password. -v for verbose. Notice you have to type YES in capital letters to continue. passphrase I chose was 1234 nothing special. Note when you type the password/passphrase its blank and doesnt display what you type, I just included it here for no reason.

# cryptsetup -y -v luksFormat /dev/md0
WARNING!
========
This will overwrite data on /dev/md0 irrevocably.
Are you sure? (Type uppercase yes): YES
Enter LUKS passphrase: 1234
Verify passphrase: 1234
Command successful.

Then we need to make a mapping to the device we will use. We can give it any name this, I called it "crypt55". So basically "crypt55" will map to /dev/md0. /dev/md0 is the encrypted device, and crypt55 is the unecrypted device which gets used. So basically UNENCRYPTED /dev/mapper/crypt55 --> ENCRYPTED /dev/md0 . We need to enter the passphrase accurately. This is essentially where we "login" with our passphrase. Just for fun at first I put the wrong password to make sure it fails - like it would not lol :-) - and it did so then it asks for it and I enter the correct 1234

# cryptsetup luksOpen /dev/md0 crypt55
Enter passphrase for /dev/md0: 4321
No key available with this passphrase.
Enter passphrase for /dev/md0: 1234

Note you can see all your opened crypt drives after you map them - they are in dev mapper. Before the luksOpen command this folder only had the "control" device. The control device is always there.

# ls -l /dev/mapper
total 0
crw------- 1 root root 10, 59 Feb 16 05:16 control
lrwxrwxrwx 1 root root      7 Mar  5 07:02 crypt55 -> ../dm-0

To see information on the cryptsetup and get an idea of what the default encryptions are:

# cryptsetup -v status crypt55
/dev/mapper/crypt55 is active:
  cipher:  aes-cbc-essiv:sha256
  keysize: 256 bits
  device:  /dev/md0
  offset:  2304 sectors
  size:    3682560 sectors
  mode:    read/write
Command successful.

First we are going to write all zeros to this "crypt55" drive which will inturn put alot of random data on /dev/md0.

# dd if=/dev/zero of=/dev/mapper/crypt55

(or even better)

BUT THAT WILL TAKE LONG AND WHO KNOWS WHEN IT WILL BE DONE LETS MONITOR WITH pv ANOTHER TOOL bar WORKS TOO - EXAMPLE OF bar WAS ABOVE. "pv -h" SHOWS ALL THE OPTIONS WE ARE GOING TO USE THESE:
 -p, --progress           show progress bar
 -t, --timer              show elapsed time
 -r, --rate               show data transfer rate counter
 -e, --eta                show estimated time of arrival (completion)
 -b, --bytes              show number of bytes transferred
 
 # pv -tpreb /dev/zero | dd of=/dev/mapper/crypt55 bs=128M
 38.6MB 0:00:07[3.57MB/s][               <=>                    ]
 
Note how the <=> is the location of the progress it starts on the left and goes to the right just like any progress bar. HOWEVER it comes back and then it goes the forward. AHH its not a progress bar. Its not a progress bar, its a mistake!
The speeds im seeing: 3,9,2,3,11 Mb/s
On a lighter note - I opened up another shell to see CPU use:
# w
load average: 12.68, 6.26, 2.78
12.68!!!!!!!!!!! wow that means to do the current job without queuing the processor this crappy processor would have to be 13 times stronger/faster/harder working.
Memory use was like this - regular memory almost completly used up and cache/swap free to run around:
# free -m
             total       used       free     shared    buffers     cached
Mem:           375        365         10          0         37        234
-/+ buffers/cache:         93        282
Swap:         1339         51       1288

When it was over this is how it looked. Notice how the progress bar is like smack down in some random spot :/ I realize I could probably use another pv arg. to make it more progressy but eh.
1.76GB 0:10:34 [2.83MB/s][    <=>                               ]
0+14386 records in
0+14385 records out
1885470720 bytes (1.9 GB) copied, 639.592 s, 2.9 MB/s
Total Time 10 minutes 34 seconds at 2.9 MB /s just at writing zeros. Sounds just as fast as the ext3 filesystem from above without any encryption - however this encryption is the default encryption its probably slow.

Just to heat things up Im making ext4 for this one. After all it should be "better" then ext3. If you ask me its all about "zfs" as soon as they get all the quirks out of it. zfs has encryption and raid built into it, so thats pretty sweet. Ill make an article on it later.

# mkfs.ext4 /dev/mapper/crypt55
# mount /dev/mapper/crypt55 /mnt/r55
Right after the mount it was still happy "w" showed a load average of 0.17 for the 1st minute average (thats the number on the very left on the load average triple digit numbers - so the full load average numbers were:  load average: 0.17, 0.26, 1.52 - 1st minute average, 5 min average, 15 min average

--SPEED TESTS--
# cd /mnt/r55

-WRITE TEST:-
ZEROS DATA:
# dd if=/dev/zero of=testfile0 bs=8k count=10000; sync;
Hit CONTROL-C after 5 seconds to get results
81920000 bytes (82 MB) copied, 35.2622 s, 2.3 MB/s

RANDOM DATA:
# dd if=/dev/urandom of=testfileR bs=8k count=10000; sync;
Hit CONTROL-C after 5 seconds to get results
81920000 bytes (82 MB) copied, 68.4873 s, 1.2 MB/s

-READ TEST:-

ZEROS DATa:
# dd if=testfile0 of=/dev/null
81920000 bytes (82 MB) copied, 14.6555 s, 5.6 MB/s

RANDOM DATA:
# dd if=testfileR of=/dev/null 
81920000 bytes (82 MB) copied, 5.95958 s, 13.7 MB/s

OTHER READ TEST:
# hdparm -Tt /dev/md0
Timing cached reads:   16864 MB in  2.00 seconds = 8445.45 MB/sec
Timing buffered disk reads:  16 MB in  3.84 seconds =   4.16 MB/sec
# hdparm -Tt --direct /dev/md0
Timing O_DIRECT cached reads:   6710 MB in  2.00 seconds = 3358.27 MB/sec
Timing O_DIRECT disk reads:  34 MB in  3.15 seconds =  10.79 MB/sec
# hdparm -Tt --direct /dev/mapper/crypt55
Timing O_DIRECT cached reads:   238 MB in  2.01 seconds = 118.19 MB/sec
Timing O_DIRECT disk reads:  58 MB in  3.11 seconds =  18.67 MB/sec
# hdparm -Tt /dev/mapper/crypt55
Timing cached reads:   16476 MB in  1.99 seconds = 8262.49 MB/sec
Timing buffered disk reads:  24 MB in  3.21 seconds =   7.48 MB/sec

THROUGH OUT THE TEST THIS IS THE CPU USAGE I HAD:
# w 
 07:55:59 up 17 days,  2:39,  2 users,  load average: 3.31, 2.94, 2.18

5 MINUTES LATER THIS WAS THE CPU USAGE:
# w
 08:00:34 up 17 days,  2:44,  2 users,  load average: 0.15, 1.27, 1.66


---SOME COMMON QUESTIONS AND ANSWERS FOR ENCRYPTION ON LUKS AND A RECAP---

WHATS LUKSFORMAT?
Its the best - google it.

HOW DO I MAKE AN ENCRYPTED DEVICE FROM SCRATCH?
# apt-get update
# apt-get install cryptsetup
Format is as such:
"cryptsetup -y -v <DEVICE-THAT-WILL-BE-ENCRYPTED-DATA-WILL-BE-LOST-ON-IT>"
Example:
# cryptsetup -y -v /dev/md0

HOW DO I MOUNT OR REMOUNT ENCRYPTED PARTITION?
Format is as such:
"cryptsetup luksOpen <ENCRYPTED-DEVICE> <MAPPED-DEVICE-JUST-THE-NAME-NO-NEED-FOR-/dev/mapper/NAME-JUST-NAME>"
"mount /dev/mapper/<NAME> <MOUNT-POINT>"
Example:
# cryptsetup luksOpen /dev/md0 crypt55
# mount /dev/mapper/crypt55 /mnt/r55
# df -H
# mount

HOW DO I UNMOUNT AND SECURE DATA?
Format is as such:
"umount <MOUNT-POINT>"
"cryptsetup luksClose <MAPPING-DEVICE-THE-UNENCRYPTED-ONE>"
Example:
# umount /mnt/r55
# cryptsetup luksClose crypt55

HOW TO CHECK STATUS?
Format is as such:
"cryptsetup -v status <MAPPING-DEVICE-THE-UNENCRYPTED-ONE>"
Example:
#cryptsetup -v status crypt55

DO FILE SYSTEM CHECKS?
Luks is the block level encryption and not the filesystem. The filesystem is ext4 so fsck will work. People might ask why does e2fsck - which is what fsck points at - work with ext4 and ext3? The argument is good because e2fsck implies ext2 with its e2 as the prefix. The answer is. ext3 is just ext2 with the journaling option. And ext4 is just ext2 with the journaling option plus some other extra options. So ext4 is logically ext3 plus some extra other options.
Below is only for ext2,3,and 4
Format:
"e2fsck -fn <MAPPING-DEVICE-THE-UNENCRYPTED-ONE>"
(or)
"nohup e2fsck -fn -C 0 <FULL-NAME-OF-MAPPING-DEVICE-THE-UNENCRYPTED-ONE> &" To run f.s. check in the background and in nohup - nohup allows you to close the shell while f.s. check continues
"tail -f nohup.out" Then CONTROL-C to exit out of view, the fsck will continue in background, can also close the shell and it will still run
Example:
# e2fsck -fn <MAPPING-DEVICE-THE-UNENCRYPTED-ONE>
(or)
# nohup e2fsck -fn -C 0 /dev/mapper/crypt55 &
# tail -f nohup.out

==READ SPEED TESTS OF THE OTHER MDS==
# hdparm -Tt --direct /dev/md1
 Timing O_DIRECT cached reads:   6726 MB in  2.00 seconds = 3364.91 MB/sec
 Timing O_DIRECT disk reads:  66 MB in  3.00 seconds =  21.98 MB/sec
# hdparm -Tt --direct /dev/md2
 Timing O_DIRECT cached reads:   6814 MB in  2.00 seconds = 3413.16 MB/sec
 Timing O_DIRECT disk reads: 102 MB in  3.16 seconds =  32.28 MB/sec
# hdparm -Tt --direct /dev/md3
 Timing O_DIRECT cached reads:   6678 MB in  2.00 seconds = 3339.93 MB/sec
 Timing O_DIRECT disk reads:  48 MB in  3.02 seconds =  15.87 MB/sec
# hdparm -Tt --direct /dev/md4
 Timing O_DIRECT cached reads:   6830 MB in  2.00 seconds = 3416.88 MB/sec
 Timing O_DIRECT disk reads:  50 MB in  3.11 seconds =  16.07 MB/sec
 
 ##############
 ===Summary:===
 ##############
 
 Imagine your the government and you had 12 gov. agents with their usb sticks - alone they are useless - together they can combine into raids that make more raid that needs to be decrypted by a 13th agent with his 1234 password. That then Mister Double-Oh-Thirteen can launch a nuke to Jupiter at whopping 3 Megabytes / Sec. Wow what a party!
Comments