Wednesday, 17 October 2012

SOLARIS KERNEL PATCHING PROCEDURE


  1. We need to prepare patch document procedure with back-out plan, and needs to get the approval by change management. Once we get the approval, we can start our change in scheduled window
  2. Download suitable our platform bundle patch(Recommended Solaris Cluster Patch) from https://support.oracle.com
#isainfo -b  [ to check the bit version]
  1. Take the necessary backups of the server.
#uname –a
# df –k
#metastat –p
#metadb –i
#netastat –nr
#ifconfig –a
#prtconf
#prtdiag –v
#metastat -ac
#metastat –t
#eeprom
#echo | format
/etc/system
/etc/vfstab

NOTE: Better to run SUN Explorer before preceding this activity
NOTE: Take the outputs before detaching the mirror else you would have to take the backups separately for your detached disk
NOTE: Zones must be halted, while doing patch on active boot enviromnet
            #zoneadm list –vc
NOTE: Before proceeding with patching, make sure our application & data base are down
  1. Break the mirror, and it will isolate from root disk.
c1t0d0 —> Root disk
c1t1d0 —> Root-mirror disk
·         First delete state database replicas from root-mirror disk
#metadb –i
a    p  luo        16              8192            /dev/dsk/c1t1d0s7
#metadb –d –f  /dev/dsk/ c1t1d0s7
·         Proceed with the mirror detach for your root-mirror disk
# metastat –p
            d1 -m d20 d21 1
d20 1 1 c1t0d0s1
d21 1 1 c1t1d0s1
d0 -m d10 d11 1
d10 1 1 c1t0d0s0
d11 1 1 c1t1d0s0
#metadetach d1 d21
#metadetach d0 d11
#metaclear d21
#metaclear d11
#metastat –p
            d1 -m d20
d20 1 1 c1t0d0s1
                                    d0 -m d10
d10 1 1 c1t0d0s0

  1. Mount detached root-mirror disk on mount point /mnt and edit /mnt/etc/vfstab & /mnt/etc/vfstab
#mount /dev/dsk/c1t1d0s0 /mnt
#vi /mnt/etc/vfstab     [change md entries to native entries]
/dev/md/dsk/d1   à /dev/dsk/c1t1d0s1
/dev/md/dsk/d0   à /dev/dsk/c1t1d0s0
# grep -i md /mnt/etc/system
* Begin MDD root info (do not edit)
rootdev:/pseudo/md@0:0,0,blk    [comment with *]
* End MDD root info (do not edit)

  1. Unmount root-mirror disk and fsck
#umount /mnt
#fsck /dev/rdsk/c1t1d0s0
NOTE:
1.) Please keep in mind that all changes are applicable to root-mirror disk. Do not do any changes on other disk (i.e. root-disk).
2.) It is recommended to run fsck on root-mirror disk (slice 0, rootFS slice) so that it won’t give any errors while booting.
3.) Before proceeding with the patching make sure that /var FS have enough free space as all logs will be written there. else you will get errors while patching.
  1. Now our root-disk is under SVM control and root-mirror disk is under native raw Solaris disk. By doing this step we are making sure that in case if anything goes against our action plan then we will be able to recover our server from other disk.  
  2. We have to test the booting of our server from both the disks to make sure that the server will come up with both of the disks without any issues.
#init 0
Ok>devalias root-mirror
/pci@1f,0/pci@1/scsi@8/disk@1,0:a
Ok>boot root-mirror 
Once system come-up with root-mirror, needs do the post check like df –h, ifconfig etc
Now boot the server from root-disk as well
#init 0
Ok>devalias rootdisk
/pci@1f,0/pci@1/scsi@8/disk@0,0:a
            Ok>boot rootdisk
Once system come-up with rootdisk, needs do the post check like df –h, ifconfig etc

  1. Now we can go ahead with patching, but server should be single user mode
#cd /var/tmp
# digest -a md5 -v /var/tmp/10_x86_Recommended.zip [will check md5 value]
#unzip -q 10_Recommended.zip
NOTE: for unzipping of recommended patch, we need 5 GB of disk space
#grep PASSCODE 10_Recommended.README
*************** PASSCODE **************
PASSCODE: s10patchset
#shutdown -g0 -y –is
#who –r
.       run-level S  Oct 25 12:39     S      0  3
NOTE: For installation recommended patch, we need 3 GB of disk space
#./installpatchset --s10patchset  
Setup ..........
Recommended OS Patchset Solaris 10 x86 (2011.09.14)

The patch set will complete installation in this session. No intermediate
reboots are required.

Application of patches started : 2011.09.16 20:24:45
Applying 120901-03 (  1 of 302) ... skipped
Applying 121334-04 (  2 of 302) ... skipped
Applying 119255-81 (  3 of 302) ... skipped
.
. <similar output omitted>
.
Applying 147379-01 (300 of 302) ... success
Applying 147435-01 (301 of 302) ... success
Applying 147441-01 (302 of 302) ... success
NOTE: A symlink to 'installpatchset' is provided in place of the legacy 'installcluster' script for now

  1. Once patching has done, needs to run reconfiguration reboot on server and check the kernel version
#touch /reconfigure
#reboot -- -r
#uname -a
SunOS Release 5.10 Version Generic_147441-01 32-bit
NOTE: Do not re-mirror the root-mirror disk immediately; wait for 5 to 7 days to notice any issues with the OS or application. If all is good then re-mirror the disks for redundancy.

NOTE: If patch fails, for roll-back you need to boot from the root-mirror disk and create all the SVM mirrors back on root-mirror disk from start and then re-mirror root-disk back with root-mirror disk.


Tuesday, 16 October 2012

SOFT & HARD LINKS



Soft Link:
            Symbolic link is a file that points to another file, and can create symbolic links for files, directories, and device files also

            Syntax: ln -s <source_file> <target_file>
                          #ln –s /var/file1 link1

  1. The size of the symbolic link always matches the number of characters in the path name it contains. EX: bin -> ./usr/bin  [ size of bin directory is : 9]
  2. Inode values are different but data block are same for soft link.
  3. Across file system or network or another disk we can create the symbolic links
  4. We can identify the soft link by using command ls –F which it displays the @ symbol at the end of the link
# ls -F
Desktop/     bin@         devices/     export/      lib/         net/         proc/        tmp/        
Documents/   boot/        etc/         home/        lost+found/  opt/         sbin/        usr/
anonFTP/     dev/         evolution/   kernel/      mnt/         platform/    system/      var/
# ls -l bin
lrwxrwxrwx   1 root     root           9 Sep  8 19:06 bin -> ./usr/bin
  1. We can create the soft links for non existence source also#ln -s sur1 sub2 [here both files non existence files]
  1. We can remove symbolic links by using rm command#rm link1
Hard Links:
  1. We can create hard links for files only, and can’t create the hard links across the file system
  2. Inode values are always same but data block are differect and will occupy the disk space
  3. We can identify hard links by using link count more than 1
# ls -li /etc/rc2.d/K05appserv
      5425 -rwxr--r--   5 root     sys          824 Aug 16  2006 /etc/rc2.d/K05appserv
#find / -mount –inum 5425 –exec ls –ltr  {} \;
5425    1 -rwxr--r--   5 root     sys           824 Aug 16  2006 /etc/init.d/appserv
 5425    1 -rwxr--r--   5 root     sys           824 Aug 16  2006 /etc/rc0.d/K05appserv
 5425    1 -rwxr--r--   5 root     sys           824 Aug 16  2006 /etc/rc1.d/K05appserv
 5425    1 -rwxr--r--   5 root     sys           824 Aug 16  2006 /etc/rc2.d/K05appserv
 5425    1 -rwxr--r--   5 root     sys           824 Aug 16  2006 /etc/rc3.d/S84appserv
Or
# find / -inum 5425
/etc/init.d/appserv
/etc/rc0.d/K05appserv
/etc/rc1.d/K05appserv
/etc/rc2.d/K05appserv
/etc/rc3.d/S84appserv


Flipkart.com

SVM ROOT MIRRORING



c1t0d0 —> Root disk
c1t1d0 —> Root-mirror disk
Sub mirrors for Root disk —> d10, d20
Sub mirrors for Root-mirror disk —> d11, d21
Mirror name would be —> d0, d1  (root, swap)

1.     Need to take the backup of required files
#cp /etc/system /etc/system.bkp
#cp /etc/vfstab /etc/vfstab.bkp
#df -h>df.bkp
#echo|format>format.bkp
2.     We have to select the new disk as same geometry of root disk, and also need to label the new disk by using root disk
# fmthard -s /dev/null /dev/rdsk/c1t1d0s2  [empty labeling on new disk]
#prtvtoc /dev/rdsk/c1t0d0s2|fmthard -s - /dev/rdsk/c1t1d0s2 [labeling new disk]
3.     Need to create 3 meta database replicas on each disk [should allocate 50MB for 7th slice on each disk, and no need run newfs]
# metadb -a -f -c 3 /dev/rdsk/c1t0d0s7
# metadb -a -f -c 3 /dev/rdsk/c1t1d0s7
#metadb   [to check metadb values]
4.     Create RAID-0 volumes on root disk for sub-mirrors
# metainit -f d10 1 1 c1t0d0s0    [for root]
# metainit -f d20 1 1 c1t0d0s1    [for swap]
5.     Create second RAID-0 volumes on root-mirror disk for sub-mirrors
# metainit -f d11 1 1 c1t1d0s0
# metainit -f d21 1 1 c1t1d0s1
6.     Create RAID-1 one-way mirror volumes using RAID-0 volumes on root disk
# metainit -f d0 -m d10
# metainit -f d1 -m d20
7.     Execute “metaroot” command on root mirror volume, and it will update the /etc/system file & /etc/vfstab file
Note: Before executing “metaroot” just check system & vfstab for knowing difference after this
        #metaroot d0
NOTE: Check /etc/vfstab & /etc/system files, and above command will update only for root file system with “md” entries for rest of other file systems(i.e., swap, usr, var etc..) we need to made it as manual “md” entries.
        #cat /etc/vfstab
            /dev/dsk/c1t0d0s1               -                           -       swap    -        no      -
             /dev/md/dsk/d0      /dev/md/rdsk/d0  /       ufs       1        no      -
         #vi /etc/vfstab   [need to change with “md” entries for other FS]
            /dev/md/dsk/d1               -                   -       swap    -        no      -
             /dev/md/dsk/d0      /dev/md/rdsk/d0  /       ufs       1        no      -

8.     For file system level alters, we must reboot the system[i.e. will effect in df –h with md entries]
 #shutdown –y –g0 –i6
9.     Once system came up, need to attach second submirrors to main mirror
#metattach d0 d11
#metattach d1 d21
Check the status of syncing
#metastat|grep –i sync
10.  Install “bootblk” on root mirrored disk
#installboot /usr/platform/`uname –i`/lib/fs/ufs/bootblk  /dev/rdsk/c1t1d0s0
#prtconf -pv | grep -i bootpath
                 bootpath:  '/pci@1f,0/pci@1/scsi@8/disk@0,0:a'
NOTE: for x86 systems old versions [should be on s2 slice]
#installboot /usr/platform/i86pc/lib/fs/ufs/pboot \
/usr/platform/i86pc/lib/fs/ufs/bootblk /dev/rdsk/c1t1d0s2
------------------------------------------------------------------------------------------------------------------------
NOTE: From Solaris 10 x86 U1 version, the installboot command is obselete. So, use the installgrub command, to make the second disk bootable. It is better to run this command once, to make sure the secondary disk is bootable.
# installgrub -fm /boot/grub/stage1 /boot/grub/stage2 /dev/rdsk/c1t1d0s0
11.  Record second submirror name from event of failures
#eeprom boot-device
        boot-device=rootdisk  net
#ls -l /dev/dsk/c1t1d0s0
     /pci@1f,0/pci@1/scsi@8/sd@1,0:a
Ok>       [stop+A]
Ok> show-disks                       [select the root-mirror disk  ]
Ok>nvalias root-mirror  /pci@1f,0/pci@1/scsi@8/disk@1,0:a
NOTE: needs to boot with each drive for testing
Ok>setenv boot-device rootdisk root-mirror net
Ok>printenv boot-device
    rootdisk   root-mirror   net
Ok>boot
NOTE: x86 systems
# eeprom bootpath
bootpath=/pci@0,0/pci15ad,1976@10/sd@0,0:a
# ls -l /dev/dsk/c1t1d0s0
lrwxrwxrwx   1 root     root          46 Feb  7 16:48 /dev/dsk/c1t1d0s0 -> ../../devices/pci@0,0/pci15ad,1976@10/sd@1,0:a
# eeprom altbootpath=/pci@0,0/pci15ad,1976@10/sd@1,0:a
NOTE: Booting with 50% of available replicas is possible/optional with the /etc/system setting “set md:mirrored_root_flag=1” i.e to maintain quorum also add the mentioned line in /etc/system file.
UNMIRRORING PROCEDURE
1.     Un-record second sub-mirror from “boot-device”
2.     Need to detach root-mirror disk sub-mirrors from main mirror
#metadetach d0 d11   [for root]
#metadetach d1 d21   [for swap]
3.     Execute metaroot on root file system block device
#metaroot /dev/dsk/c1t0d0s0
Check /etc/vfstab & /etc/system file, and above command will update for root FS
#grep c1t0d0s0 /etc/vfstab
/dev/dsk/c1t0d0s0  /dev/rdsk/c1t0d0s0  /   ufs   1   no  -
NOTE: For other File systems[swap,opt,usr], and we need to edit manually, and make it as native entries
NOTE: Here's the caveat(caution) for mirrored swap - you're probably using /dev/md/dsk/d1 for your dump device. Let's fix that now. First run
dumpadm | grep '/md/'
If that returns any output, then run this (using your single-disk slice for swap):
dumpadm -s /var/crash/`hostname` -d /dev/dsk/c1t0d0s1

4.     Reboot the system
#shutdown –y –g0 –i6
5.     Now, we can clear main mirror as well as sub-mirror under root disk
#metaclear –r d0
#metaclear –r d1
Flipkart.com