IO performance improvement by setting up a proper block size

Performance tuning is one of the major activities for a Unix Administrator. Today i will show you a practical scenario where performance can be tunned by setting up

proper block size while creating filesystem. When a file is created, system allocates set of blocks to store the files. Filesystem with larger block size containing

only smaller files, will have lot of space un-utilized.

In database where files are of bigger size, for optimal performance its better to create filesystem of larger block size.

So before creating a filesystem, find out what would be the average size of the files to be stored in the filesystem, based on that decide on that size of filesystem

blocksize

Check the block size of the filesystem

[root@linuxserver ~]# dumpe2fs /dev/sda1 | grep -i "Block size"
dumpe2fs 1.41.12 (17-May-2010)
Block size:               1024
[root@linuxserver ~]# dumpe2fs /dev/sda2 | grep -i "Block size"
dumpe2fs 1.41.12 (17-May-2010)
Block size:               4096


In this demo i am going to create a 2 filesystem with blocksize of 1024 and 4096 and create 20 files of 10MB size

[root@linuxserver /]# mkfs -t ext3 -b 1024 /dev/sdc1 - /app1 - mountpoint


#!/bin/sh
no=0
while [ $no -le 20 ]
do
dd if=/dev/zero of=test$no bs=10M count=1
no=`expr $no + 1`
done

Below is the time required to create 20 files of 10MB size

real    0m8.162s
user    0m0.005s
sys     0m1.881s

[root@linuxserver /]# mkfs -t ext3   /dev/sdb1 - /app - mountpoint
By default it creates filesystem using blocksize of 4096 bytes

#!/bin/sh
no=0
while [ $no -le 20 ]
do
dd if=/dev/zero of=test$no bs=10M count=1
no=`expr $no + 1`
done

real    0m1.268s
user    0m0.014s
sys     0m1.239s

Exporting ZFS filesystem to non-global zone

There are 2 ways zfs filesystem can be assigned to a non-global zone. 

Method 1:

1. Create a zfs filesystem on global zone 

     

        zfs create orapool/app

Where orapool - is the zpool name and app is the zfs filesystem name

2. Export the above the zfs filesystem to non-global zone called appzone

zonecfg -z appzone

[root@server1]# zonecfg -z appzone
zonecfg:appzone> add fs
zonecfg:appzone:fs> set type=zfs
zonecfg:appzone:fs> set special=orapool/app
zonecfg:appzone:fs> set dir=/zonefs
zonecfg:appzone:fs> end
zonecfg:appzone> commit
zonecfg:appzone> exit

In  the above method, non-global zone administrator does not have any control over the exported the filesystem, means can not set any attribute for the filesystem


Method 2:

In this method , the non-global zone administrator has complete control over the filesystem

[root@server1]# zonecfg -z appzone
zonecfg:appzone> add dataset
zonecfg:appzone:dataset> set name=orapool/app
zonecfg:appzone:dataset> end
zonecfg:appzone> commit
zonecfg:appzone> exit

Once its exported, you can login to non-global zone and go ahead with the creating any no of filesystem on the exported the dataset

For example on orapoo/app you need orapool/app/test , orapool/app/test1,  orapool/app/tes3

Use the following command to create the above filesystem on non-global zone

#zfs create  orapool/app/test

#zfs create orapool/app/test1

#zfs create orapool/app/test2

So second approach is always best when you need to create multiple mountpoints on a dataset on non-global zone.

Other info:

 

Realtime issue faced:

I came across this issue when i was trying to set the mountpoint properties of previously exported dataset from global zone. I was getting error message as "Dataset is busy , its already exported"

Reason:

 When you export a dataset from a global zone, it sets  dataset's zone attribute,  which does not get turned off even if you remove the dataset from zone configuration using zonefcg command. You can view all the attributes of zone using

#zfs get orapool/app

Solution:

Turn of the zone attribute

#zfs set zone=off orapool/app

 

 

Persitent Binding for HBA Card

Persistent binding : As the name implies persistent binding enables LUNs zoned from a Storage frame will be assigned with a specific target ID. Now next question which may pops up in your mind "Why do we need that"?

I don't have any strong answer for it. After googling, got few clarifications which are not much convincing.

Below are the few points which i feel could be a reasonable answer for the above question.

1. In a cluster environment where the same LUN is assigned to multiple machine should be configured with persistent target ID.

2. Its a recommended steps for emulex branded HBA card where drivers software are provided by Emulex and same applies for Q Logic branded card.

Here in this, i would be taking only Emulex and Qlogic card as these cards are the most commonly used card on Solaris Box.

NOTE : You do not need to configure persistent binding for Sun Branded Emulex and Qlogic card. 

So HBA cards are categorized as Sun-branded and Sun-non-branded.

Below are the tips to identify the sun-branded and non-branded qlogic and emulex card.

Emulex Card:

LPxxxxxx-S are SUN Emulex HBA Cards  (ie has a “-S” for Sun at end).

For Solaris 10 fcinfo hba-port would provide you the details and for previous version of solaris prtpicl command can be used to get the HBA details

If you see any of the following references they are NON-SUN HBA Cards:

NON-SUN Qlogic:

• qlaxxxx
• QLGC,qla
• QLGC,qlc
• SUNW,qla

NON-SUN Emulex:

• emlx (emlxs is SUN)
• lpfc
  
  Click on the below link for the detailed explanation.
  
  Identify HBA card
  
  So we have got some idea on HBA cards identification now lets go ahead and configure the persistent for emulex card. As i have previously mentioned that persistent binding is needed only for SUN non-branded emulex card.
  
  To configure persistent binding, there are 2 files needed to be modified.
  
  /kernel/drv/lpfc.conf
  /kernel/drv/sd.conf
  
  lpfc.conf - Kernel configuration file for lpfc driver modules
  sd.conf - kernel configuration file for sd driver module
  
  lpfc.conf file is where you configure the persistent binding and sd.conf is where you should add enough entry to allow OS to scan for more devices.
  
  For persistent binding you need to find the HBA card instance you can easily determine this by using lputil commands
  
  Here is an example
  
  bash-3.00# /usr/sbin/lpfc/lputil
  LightPulse Common Utility for Solaris/SPARC. Version 2.0a13 (1/3/2006).
  Copyright (c) 2005, Emulex Corporation
  
  Emulex Fibre Channel Host Adapters Detected: 3
  Host Adapter 0 (lpfc3) is an LP11002-E (Ready Mode)
  Host Adapter 1 (lpfc4) is an LP11002-E (Ready Mode)
  Host Adapter 2 (lpfc2) is an LP9802 (Ready Mode)
  
  
  As you can see above there are 3 cards. LP11002-E looks like a dual port card that can be confirmed by looking at /etc/path_to_inst file . Here is extract from the the file
  
  
  "/pci@1c,600000/lpfc@1,1" 1 "emlxs"
  "/pci@1c,600000/lpfc@1,1" 1 "lpfc"
  "/pci@1c,600000/lpfc@1,1/fp@0,0" 1 "fp"
  "/pci@1c,600000/fibre-channel@1" 3 "lpfc"
  "/pci@1c,600000/fibre-channel@1,1" 4 "lpfc"
  "/pci@1d,700000/lpfc@2" 2 "emlxs"
  "/pci@1d,700000/lpfc@2" 2 "lpfc"
  "/pci@1d,700000/lpfc@2/fp@0,0" 2 "fp"
  
  From the above 
•  
• "/pci@1c,600000/fibre-channel@1" 3 "lpfc"
  "/pci@1c,600000/fibre-channel@1,1" 4 "lpfc"
  
  
  See the physical path of the lpfc which is same with the instance as 3 and 4 so its a single HBA card having dual port so lpfc3 and lpfc4 is the instance of dual port HBA card.
  
  Sample sd.conf file:
  
  name="sd" parent="lpfc" target=0 lun=0 hba="lpfc0";
  name="sd" parent="lpfc" target=0 lun=1 hba="lpfc0";
  name="sd" parent="lpfc" target=0 lun=2 hba="lpfc0";
  name="sd" parent="lpfc" target=0 lun=3 hba="lpfc0";
  name="sd" parent="lpfc" target=0 lun=4 hba="lpfc0";
  name="sd" parent="lpfc" target=0 lun=5 hba="lpfc0";
  name="sd" parent="lpfc" target=0 lun=6 hba="lpfc0";
  name="sd" parent="lpfc" target=0 lun=7 hba="lpfc0";
  
  name="sd" parent="lpfc" target=1 lun=0 hba="lpfc1";
  name="sd" parent="lpfc" target=1 lun=1 hba="lpfc1";
  name="sd" parent="lpfc" target=1 lun=2 hba="lpfc1";
  name="sd" parent="lpfc" target=1 lun=3 hba="lpfc1";
  name="sd" parent="lpfc" target=1 lun=4 hba="lpfc1";
  name="sd" parent="lpfc" target=1 lun=5 hba="lpfc1";
  name="sd" parent="lpfc" target=1 lun=6 hba="lpfc1";
  name="sd" parent="lpfc" target=1 lun=7 hba="lpfc1";
  
  
  
  
  
  Now in the above output assigning lun no is a tricky one. You need to work with storage team for it. Get the HEX code for each LUN and convert it to decimal which needs to be specified as lun no.
  
  Another example of /etc/path_to_inst
  
  > grep -i lpfc /etc/path_to_inst
  "/pci@1e,600000/pci@0/pci@1/pci@0/pci@8/fibre-channel@1" 0 "lpfc"
  "/pci@1e,600000/pci@0/pci@1/pci@0/pci@8/fibre-channel@1,1" 1 "lpfc"
  "/pci@1e,600000/pci@0/pci@8/fibre-channel@0" 4 "lpfc"
  "/pci@1e,600000/pci@0/pci@8/fibre-channel@0,1" 5 "lpfc"
  "/pci@1f,700000/pci@0/pci@2/pci@0/pci@8/fibre-channel@2" 2 "lpfc"
  "/pci@1f,700000/pci@0/pci@2/pci@0/pci@8/fibre-channel@2,1" 3 "lpfc"
  "/pci@1f,700000/pci@0/pci@9/fibre-channel@0" 6 "lpfc"
  "/pci@1f,700000/pci@0/pci@9/fibre-channel@0,1" 7 "lpfc"
  
  From the above you can find out that there 4 emulex dual ports HBA cards
  
  
  
  
  
  
  
  
  
  

Persitent Binding for HBA Card

Persistent binding : As the name implies persistent binding enables LUNs zoned from a Storage frame will be assigned with a specific target ID. Now next question which might pops up in your mind "Why do we need that"

I don't have any strong answer for it. After googling, got few clarifications which is not much convincing.

Below are the few points which i feel could be a reasonable answer for the above question.

1. In a cluster environment where the same LUN is assigned to multiple machine should be configured with persistent target ID.

2. Its a recommended steps for emulex branded HBA card where drivers software are provided by Emulex and same applies for Q Logic branded card.

Here in this, i would be taking only Emulex and Qlogic card as these cards are the most commonly used card on Solaris Box.

NOTE : You do not need to configure persistent binding for Sun Branded Emulex and Qlogic card.

Physical to Virtual server migration(P2V)

First create a flar image of the source system  . 

Following are the various option for flarcreate command

1. flarcreate -n archive1 -c archive1.flar ---------- create flar archive which is a exact duplicate of master system and and compressed

2. flar info -l archive1.flar - To check the structure of flar archive

Create an archive that contains large files

-------------------------------------------------

The default archiving utiltiy in flar is cpio which cannot handle large file of more than 4G . Instead of that -L pax copy to be invoked when any individual file is larger than 4 G

#flarcreate -L pax -n archive1 -c archive1.flar

To check : #flar info -l archive1.flar

Create an archive from alternate root filesystem.

-------------------------------------------------------


#flarcreate -n archive1.flar -c -R /x/yy/zz archive1.flar


 

# flarcreate -n archive3 -i 20000131221409 -m pumbaa  -e "Solaris 8 Print Server" -a "Mighty Matt" -U "Internal Finance"  -T server archive3.flar - provide description

flarcreate -S -n archive1 -L cpio archive1.flar - best command -S option will avoild disk size checking


P2V

Oracle Solaris Zones Physical-to-Virtual (P2V)
A new feature for Solaris 10 9/10.This feature provides the ability to build a Solaris 10 images from physical
system and migrate it into a virtualized operating system environment
There are three main steps using this tool

1. Image creation on the source system, this image includes the operating system and optionally the software in which we want to include within the image.
2. Preparing the target system by configuring a new zone that will host the new image.
3. Image installation on the target system using the image we created on step 1.

Once the image is ready and zone is configured then performs the below steps.

#zoneadm -z solaris10-up9-zone install -u -a /var/tmp/solaris_10_up9.flar - which will perform sys-unconfig. To preserve the sysidcfg setting use the below command

#zoneadm -z solaris10-up9-zone install -p -a /var/tmp/solaris_10_up9.flar