Wednesday, March 6, 2013

Exporting ZFS filesystem to non-global zone

There are 2 ways zfs filesystem can be assigned to a non-global zone. 

Method 1:

1. Create a zfs filesystem on global zone 
     
        zfs create orapool/app

Where orapool - is the zpool name and app is the zfs filesystem name

2. Export the above the zfs filesystem to non-global zone called appzone

zonecfg -z appzone

[root@server1]# zonecfg -z appzone
zonecfg:appzone> add fs
zonecfg:
appzone:fs> set type=zfs
zonecfg:
appzone:fs> set special=orapool/app
zonecfg:
appzone:fs> set dir=/zonefs
zonecfg:
appzone:fs> end
zonecfg:
appzone> commit
zonecfg:
appzone> exit

In  the above method, non-global zone administrator does not have any control over the exported the filesystem, means can not set any attribute for the filesystem


Method 2:

In this method , the non-global zone administrator has complete control over the filesystem


[root@server1]# zonecfg -z appzone
zonecfg:
appzone> add dataset
zonecfg:
appzone:dataset> set name=orapool/app
zonecfg:
appzone:dataset> end
zonecfg:
appzone> commit
zonecfg:
appzone> exit

Once its exported, you can login to non-global zone and go ahead with the creating any no of filesystem on the exported the dataset

For example on orapoo/app you need orapool/app/test , orapool/app/test1,  orapool/app/tes3

Use the following command to create the above filesystem on non-global zone


#zfs create  orapool/app/test
#zfs create orapool/app/test1
#zfs create orapool/app/test2

So second approach is always best when you need to create multiple mountpoints on a dataset on non-global zone.

Other info:
 
Realtime issue faced:

I came across this issue when i was trying to set the mountpoint properties of previously exported dataset from global zone. I was getting error message as "Dataset is busy , its already exported"

Reason:

 When you export a dataset from a global zone, it sets  dataset's zone attribute,  which does not get turned off even if you remove the dataset from zone configuration using zonefcg command. You can view all the attributes of zone using

#zfs get orapool/app

Solution:

Turn of the zone attribute

#zfs set zone=off orapool/app


 

 







Monday, March 4, 2013

Persitent Binding for HBA Card

Persistent binding : As the name implies persistent binding enables LUNs zoned from a Storage frame will be assigned with a specific target ID. Now next question which may pops up in your mind "Why do we need that"?

I don't have any strong answer for it. After googling, got few clarifications which are not much convincing.


Below are the few points which i feel could be a reasonable answer for the above question.


1. In a cluster environment where the same LUN is assigned to multiple machine should be configured with persistent target ID.


2. Its a recommended steps for emulex branded HBA card where drivers software are provided by Emulex and same applies for Q Logic branded card.


Here in this, i would be taking only Emulex and Qlogic card as these cards are the most commonly used card on Solaris Box.


NOTE : You do not need to configure persistent binding for Sun Branded Emulex and Qlogic card. 
So HBA cards are categorized as Sun-branded and Sun-non-branded.

Below are the tips to identify the sun-branded and non-branded qlogic and emulex card.

Emulex Card:

LPxxxxxx-S are SUN Emulex HBA Cards  (ie has a “-S” for Sun at end).

For Solaris 10 fcinfo hba-port would provide you the details and for previous version of solaris prtpicl command can be used to get the HBA details

If you see any of the following references they are NON-SUN HBA Cards:

NON-SUN Qlogic:
  • qlaxxxx
  • QLGC,qla
  • QLGC,qlc
  • SUNW,qla

NON-SUN Emulex:
  • emlx (emlxs is SUN)
  • lpfc

    Click on the below link for the detailed explanation.

    Identify HBA card

    So we have got some idea on HBA cards identification now lets go ahead and configure the persistent for emulex card. As i have previously mentioned that persistent binding is needed only for SUN non-branded emulex card.

    To configure persistent binding, there are 2 files needed to be modified.

    /kernel/drv/lpfc.conf
    /kernel/drv/sd.conf

    lpfc.conf - Kernel configuration file for lpfc driver modules
    sd.conf - kernel configuration file for sd driver module

    lpfc.conf file is where you configure the persistent binding and sd.conf is where you should add enough entry to allow OS to scan for more devices.

    For persistent binding you need to find the HBA card instance you can easily determine this by using lputil commands

    Here is an example

    bash-3.00# /usr/sbin/lpfc/lputil
    LightPulse Common Utility for Solaris/SPARC. Version 2.0a13 (1/3/2006).
    Copyright (c) 2005, Emulex Corporation

    Emulex Fibre Channel Host Adapters Detected: 3
    Host Adapter 0 (lpfc3) is an LP11002-E (Ready Mode)
    Host Adapter 1 (lpfc4) is an LP11002-E (Ready Mode)
    Host Adapter 2 (lpfc2) is an LP9802 (Ready Mode)


    As you can see above there are 3 cards. LP11002-E looks like a dual port card that can be confirmed by looking at /etc/path_to_inst file . Here is extract from the the file


    "/pci@1c,600000/lpfc@1,1" 1 "emlxs"
    "/pci@1c,600000/lpfc@1,1" 1 "lpfc"
    "/pci@1c,600000/lpfc@1,1/fp@0,0" 1 "fp"
    "/pci@1c,600000/fibre-channel@1" 3 "lpfc"
    "/pci@1c,600000/fibre-channel@1,1" 4 "lpfc"
    "/pci@1d,700000/lpfc@2" 2 "emlxs"
    "/pci@1d,700000/lpfc@2" 2 "lpfc"
    "/pci@1d,700000/lpfc@2/fp@0,0" 2 "fp"

    From the above 
  •  
  • "/pci@1c,600000/fibre-channel@1" 3 "lpfc"
    "/pci@1c,600000/fibre-channel@1,1" 4 "lpfc"


    See the physical path of the lpfc which is same with the instance as 3 and 4 so its a single HBA card having dual port so lpfc3 and lpfc4 is the instance of dual port HBA card.

    Sample sd.conf file:

    name="sd" parent="lpfc" target=0 lun=0 hba="lpfc0";
    name="sd" parent="lpfc" target=0 lun=1 hba="lpfc0";
    name="sd" parent="lpfc" target=0 lun=2 hba="lpfc0";
    name="sd" parent="lpfc" target=0 lun=3 hba="lpfc0";
    name="sd" parent="lpfc" target=0 lun=4 hba="lpfc0";
    name="sd" parent="lpfc" target=0 lun=5 hba="lpfc0";
    name="sd" parent="lpfc" target=0 lun=6 hba="lpfc0";
    name="sd" parent="lpfc" target=0 lun=7 hba="lpfc0";

    name="sd" parent="lpfc" target=1 lun=0 hba="lpfc1";
    name="sd" parent="lpfc" target=1 lun=1 hba="lpfc1";
    name="sd" parent="lpfc" target=1 lun=2 hba="lpfc1";
    name="sd" parent="lpfc" target=1 lun=3 hba="lpfc1";
    name="sd" parent="lpfc" target=1 lun=4 hba="lpfc1";
    name="sd" parent="lpfc" target=1 lun=5 hba="lpfc1";
    name="sd" parent="lpfc" target=1 lun=6 hba="lpfc1";
    name="sd" parent="lpfc" target=1 lun=7 hba="lpfc1";





    Now in the above output assigning lun no is a tricky one. You need to work with storage team for it. Get the HEX code for each LUN and convert it to decimal which needs to be specified as lun no.

    Another example of /etc/path_to_inst

    > grep -i lpfc /etc/path_to_inst
    "/pci@1e,600000/pci@0/pci@1/pci@0/pci@8/fibre-channel@1" 0 "lpfc"
    "/pci@1e,600000/pci@0/pci@1/pci@0/pci@8/fibre-channel@1,1" 1 "lpfc"
    "/pci@1e,600000/pci@0/pci@8/fibre-channel@0" 4 "lpfc"
    "/pci@1e,600000/pci@0/pci@8/fibre-channel@0,1" 5 "lpfc"
    "/pci@1f,700000/pci@0/pci@2/pci@0/pci@8/fibre-channel@2" 2 "lpfc"
    "/pci@1f,700000/pci@0/pci@2/pci@0/pci@8/fibre-channel@2,1" 3 "lpfc"
    "/pci@1f,700000/pci@0/pci@9/fibre-channel@0" 6 "lpfc"
    "/pci@1f,700000/pci@0/pci@9/fibre-channel@0,1" 7 "lpfc"

    From the above you can find out that there 4 emulex dual ports HBA cards









Persitent Binding for HBA Card

Persistent binding : As the name implies persistent binding enables LUNs zoned from a Storage frame will be assigned with a specific target ID. Now next question which might pops up in your mind "Why do we need that"

I don't have any strong answer for it. After googling, got few clarifications which is not much convincing.

Below are the few points which i feel could be a reasonable answer for the above question.

1. In a cluster environment where the same LUN is assigned to multiple machine should be configured with persistent target ID.

2. Its a recommended steps for emulex branded HBA card where drivers software are provided by Emulex and same applies for Q Logic branded card.

Here in this, i would be taking only Emulex and Qlogic card as these cards are the most commonly used card on Solaris Box.

NOTE : You do not need to configure persistent binding for Sun Branded Emulex and Qlogic card.