Cloning Oracle 11g Grid Home
Cloning oracle RAC binaries has become common practice but same is not always true with the Grid!! We recently patched one of our RAC env and need to apply the same patches to the QA.
One way was to repeat the process on env to be patched. Second was to clone env and use it for RAC DBs.
Since the Cloning Grid was not very common and was looking challenging, i decided to give it a try.
On the source node, create a copy of the Oracle grid infrastructure home. To keep the installed Oracle grid infrastructure home as a working home, make a full copy of the source Oracle grid infrastructure home and remove the unnecessary files from the copy. For example, as root on Linux systems, run the cp command:
[oracle@appsractest bin]$ cp -prf Grid_home location_of_the_copy_of_Grid_home
Delete unnecessary files from the copy.
The Oracle grid infrastructure home contains files that are relevant only to the source node, so you can remove the unnecessary files from the copy in the log, crs/init, and cdata directories. Check following to remove the unnecessary files from the copy of the Oracle grid infrastructure home:
[oracle@appsractest bin]$ cd /u01/app/11.2.0.2/grid
[oracle@appsractest grid]$ rm -rf /u01/app/11.2.0.2/grid/log/host_name
[oracle@appsractest grid]$ rm -rf crs/init
[oracle@appsractest grid]$ rm -rf cdata
[oracle@appsractest grid]$ rm -rf gpnp/*
[oracle@appsractest grid]$ rm -rf network/admin/*.ora
[oracle@appsractest grid]$ find . -name '*.ouibak' -exec rm {} \;
[oracle@appsractest grid]$ find . -name '*.ouibak.1' -exec rm {} \;
[oracle@appsractest grid]$ rm -rf root.sh*
[oracle@appsractest grid]$ rm -rf /u01/app/11.2.0.2/grid/inventory/ContentsXML/oraclehomeproperties.xml
[oracle@appsractest grid]$ cd cfgtoollogs
[root@node1 cfgtoollogs]# find . -type f -exec rm -f {} \;
Create a compressed copy of the previously copied Oracle grid infrastructure home using tar or gzip on Linux and UNIX systems. Ensure that the tool you use preserves the permissions and file time-stamps.
The steps to create a cluster through cloning are as follows:
Prepare the new cluster nodes
Deploy Oracle Clusterware on the destination nodes
Run the clone.pl script on each destination node
Prepare crsconfig_params file on all nodes
Run the orainstRoot.sh script on each node
Run the Grid_home/root.sh script
Run the configuration assistants and CVU
Step 1: Prepare the New Cluster Nodes
On each destination node, perform the following preinstallation steps:
Specify the kernel parameters
Configure block devices for Oracle Clusterware devices
Ensure that you have set the block device permissions correctly
Use short, nondomain-qualified names for all of the names in the Hosts file
Test whether the interconnect interfaces are reachable using the ping command
Verify that the VIP addresses are not active at the start of the cloning process by using the ping command (the ping command of the VIP address must fail)
Delete all files in the Grid_home/gpnp folder
Note:
If the Grid_home/gpnp folder contains any files, then cluster creation fails. All resources are added to the existing cluster, instead.
Run CVU to verify your hardware and operating system environment
-- Create the grid home directory on destination home
[root@appsractest ]## mkdir -p /u01/app/11.2.0/grid
[root@appsractest ]# cd /u01/app/11.2.0/grid
[root@appsractest ]# tar -zxvf /home/oracle/gridHome.tgz
In this example, path_name represents the directory structure in which you want to install the Oracle grid infrastructure home. Note that you can change the Grid Home location as part of the clone process
Change the ownership of all of the files to belong to the oracle:oinstall group, and create a directory for the Oracle Inventory. The following example shows the commands to do this on a Linux system:
[root@appsractest grid]# chown -R oracle:oinstall /u01/app/11.2.0/grid
[root@appsractest grid]# mkdir -p /u01/app/oraInventory
[root@appsractest grid]# chown oracle:oinstall /u01/app/oracle/oraInventory
It is important to remove any Oracle network files from the /u01/app/11.2.0.2/grid/network/admin directory on both nodes before continuing. For example, remove any tnsnames.ora, listener.ora or sqlnet.ora files.
[oracle@appsractest bin]$ /home/oracle/clone.sh
./runInstaller -clone -waitForCompletion "ORACLE_BASE=/u01/app/oracle" "ORACLE_HOME=/u01/app/11.2.0.2/grid" "ORACLE_HOME_NAME=OraHome1Grid" "INVENTORY_LOCATION=/u01/app/oraInventory" "CLUSTER_NODES={appsractest}" "LOCAL_NODE=appsractest" -silent -noConfig -nowait
Starting Oracle Universal Installer...
Checking swap space: must be greater than 500 MB. Actual 2047 MB Passed
Preparing to launch Oracle Universal Installer from /tmp/OraInstall2012-10-16_05-26-26AM. Please wait ...Oracle Universal Installer, Version 11.2.0.2.0 Production
Copyright (C) 1999, 2010, Oracle. All rights reserved.
You can find the log of this install session at:
/u01/app/oraInventory/logs/cloneActions2012-10-16_05-26-26AM.log
...
Performing tests to see whether nodes are available
............................................................... 100% Done.
Installation in progress (Tuesday, October 16, 2012 5:27:02 AM IST)
......................................................................... 73% Done.
Install successful
Linking in progress (Tuesday, October 16, 2012 5:27:20 AM IST)
Link successful
Setup in progress (Tuesday, October 16, 2012 5:29:49 AM IST)
................ 100% Done.
Setup successful
End of install phases.(Tuesday, October 16, 2012 5:30:10 AM IST)
WARNING:
The following configuration scripts need to be executed as the "root" user in each cluster node.
/u01/app/11.2.0.2/grid/rootupgrade.sh #On nodes appsractest
To execute the configuration scripts:
1. Open a terminal window
2. Log in as "root"
3. Run the scripts in each cluster node
The cloning of OraHome1Grid was successful.
Please check '/u01/app/oraInventory/logs/cloneActions2012-10-16_05-26-26AM.log' for more details.
-- Repeat the above steps for all the nodes in cluster
Run the script on the local node first. After successful completion, you can run the script in parallel on all the other nodes, except a node you designate as the last node. When all the nodes except the last node are done successfully, run the script on the last node.
Execution of rootupgrade.sh log file -
Running Oracle 11g root script...
The following environment variables are set as:
ORACLE_OWNER= oracle
ORACLE_HOME= /u01/app/11.2.0.2/grid
Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
To configure Grid Infrastructure for a Cluster execute the following command:
/u01/app/11.2.0.2/grid/crs/config/config.sh
This command launches the Grid Infrastructure Configuration Wizard. The wizard also supports silent operation, and the parameters can be passed through the response file that is available in the installation media.
-- Run orainstRoot.sh script as follows
[root@appsractest ~]# cd /u01/app/oraInventory/
[root@appsractest oraInventory]# ./orainstRoot.sh
Changing permissions of /u01/app/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.
Changing groupname of /u01/app/oraInventory to oinstall.
The execution of the script is complete.
[root@appsractest oraInventory]#
[root@appsractest oraInventory]# echo Then run the following command:
Then run the following command:
-- For our process we need to prepare the following crsconfig file and then execute rootcrs.pl script as follows
-- prepare crsconfig_params file /u01/app/11.2.0.2/grid/crs/install/crsconfig_params
[root@appsractest grid]# /u01/app/11.2.0.2/grid/perl/bin/perl -I/u01/app/11.2.0.2/grid/perl/lib -I/u01/app/11.2.0.2/grid/crs/install /u01/app/11.2.0.2/grid/crs/install/rootcrs.pl
Using configuration parameter file: /u01/app/11.2.0.2/grid/crs/install/crsconfig_params
-- so its using the file crsconfig file we created earlier...
LOCAL ADD MODE
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
OLR initialization - successful
root wallet
root wallet cert
root cert export
peer wallet
profile reader wallet
pa wallet
peer wallet keys
pa wallet keys
peer cert request
pa cert request
peer cert
pa cert
peer root cert TP
profile reader root cert TP
pa root cert TP
peer pa cert TP
pa peer cert TP
profile reader pa cert TP
profile reader peer cert TP
peer user cert
pa user cert
Adding daemon to inittab
ACFS-9200: Supported
ACFS-9300: ADVM/ACFS distribution files found.
ACFS-9307: Installing requested ADVM/ACFS software.
ACFS-9308: Loading installed ADVM/ACFS drivers.
ACFS-9321: Creating udev for ADVM/ACFS.
ACFS-9323: Creating module dependencies - this may take some time.
ACFS-9327: Verifying ADVM/ACFS devices.
ACFS-9327: Verifying ADVM/ACFS devices.
ACFS-9309: ADVM/ACFS installation correctness verified.
CRS-2672: Attempting to start 'ora.mdnsd' on 'appsractest'
CRS-2676: Start of 'ora.mdnsd' on 'appsractest' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'appsractest'
CRS-2676: Start of 'ora.gpnpd' on 'appsractest' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'appsractest'
CRS-2672: Attempting to start 'ora.gipcd' on 'appsractest'
CRS-2676: Start of 'ora.gipcd' on 'appsractest' succeeded
CRS-2676: Start of 'ora.cssdmonitor' on 'appsractest' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'appsractest'
CRS-2672: Attempting to start 'ora.diskmon' on 'appsractest'
CRS-2676: Start of 'ora.diskmon' on 'appsractest' succeeded
CRS-2676: Start of 'ora.cssd' on 'appsractest' succeeded
Disk Group OCR_DISK creation failed with the following message:
ORA-15018: diskgroup cannot be created
ORA-15031: disk specification '/dev/hdd2' matches no disks
ORA-15025: could not open disk "/dev/hdd2"
ORA-15056: additional error message
Configuration of ASM ... failed
see asmca logs at /u01/app/oracle/cfgtoollogs/asmca for details
Did not succssfully configure and start ASM at /u01/app/11.2.0.2/grid/crs/install/crsconfig_lib.pm line 6464.
The issue here is that oracleasm_disks parameter having invalid value.
fix - modified parameter for oracleasm_disks -
from /dev/hdd1 to /dev/oracleasm/OCR_DISK disks
-- next run
[root@appsractest grid]# /u01/app/11.2.0.2/grid/perl/bin/perl -I/u01/app/11.2.0.2/grid/perl/lib -I/u01/app/11.2.0.2/grid/crs/install /u01/app/11.2.0.2/grid/crs/install/rootcrs.pl
Using configuration parameter file: /u01/app/11.2.0.2/grid/crs/install/crsconfig_params
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'appsractest'
CRS-2676: Start of 'ora.cssdmonitor' on 'appsractest' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'appsractest'
CRS-2672: Attempting to start 'ora.diskmon' on 'appsractest'
CRS-2676: Start of 'ora.diskmon' on 'appsractest' succeeded
CRS-2676: Start of 'ora.cssd' on 'appsractest' succeeded
You need disks from at least two different failure groups, excluding quorum disks and quorum failure groups, to create a Disk Group with normal redundancy
Configuration of ASM ... failed
-- Since I'm using external redundancy that parameter needs to get modify from NORMAL to EXTERNAL
fix - modified ASM_REDUNDANCY=EXTERNAL
-- Next run
Disk Group OCR_DISK creation failed with the following message:
ORA-15018: diskgroup cannot be created
ORA-15031: disk specification '/dev/oracleasm/disks' matches no disks
ORA-15014: path '/dev/oracleasm/disks/OCR_DISK' is not in the discovery set
ORA-15014: path '/dev/oracleasm/disks/DATA' is not in the discovery set
-- Disk discovery string needs to be fully qualified path as follows
fix - modified ASM_DISCOVERY_STRING=/dev/oracleasm/* to ASM_DISCOVERY_STRING=/dev/oracleasm/disks/*
[root@appsractest grid]# /u01/app/11.2.0.2/grid/perl/bin/perl -I/u01/app/11.2.0.2/grid/perl/lib -I/u01/app/11.2.0.2/grid/crs/install /u01/app/11.2.0.2/grid/crs/install/rootcrs.pl
Using configuration parameter file: /u01/app/11.2.0.2/grid/crs/install/crsconfig_params
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'appsractest'
CRS-2676: Start of 'ora.cssdmonitor' on 'appsractest' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'appsractest'
CRS-2672: Attempting to start 'ora.diskmon' on 'appsractest'
CRS-2676: Start of 'ora.diskmon' on 'appsractest' succeeded
CRS-2676: Start of 'ora.cssd' on 'appsractest' succeeded
ASM created and started successfully.
Disk Group OCR_DISK created successfully.
clscfg: -install mode specified
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
CRS-4256: Updating the profile
Successful addition of voting disk 6b227d8c9a314fe5bf3ce025c322a476.
Successfully replaced voting disk group with +OCR_DISK.
CRS-4256: Updating the profile
CRS-4266: Voting file(s) successfully replaced
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE 6b227d8c9a314fe5bf3ce025c322a476 (/dev/oracleasm/disks/OCR_DISK) [OCR_DISK]
Located 1 voting disk(s).
CRS-2672: Attempting to start 'ora.asm' on 'appsractest'
CRS-2676: Start of 'ora.asm' on 'appsractest' succeeded
CRS-2672: Attempting to start 'ora.OCR_DISK.dg' on 'appsractest'
CRS-2676: Start of 'ora.OCR_DISK.dg' on 'appsractest' succeeded
ACFS-9200: Supported
ACFS-9200: Supported
CRS-2672: Attempting to start 'ora.registry.acfs' on 'appsractest'
CRS-2676: Start of 'ora.registry.acfs' on 'appsractest' succeeded
Preparing packages for installation...
cvuqdisk-1.0.9-1
Configure Oracle Grid Infrastructure for a Cluster ... succeeded
At the end of the Oracle Clusterware installation, on each new node, manually run the configuration assistants and CVU. The following commands should be run from the first node only.
[oracle] $ /u01/app/11.2.0/grid/bin/netca \
/orahome /u01/app/11.2.0/grid \
/orahnam OraGridHome1 \
/instype typical \
/inscomp client,oraclenet,javavm,server\
/insprtcl tcp \
/cfg local \
/authadp NO_VALUE \
/responseFile /u01/app/11.2.0/grid/network/install/netca_typ.rsp \
/silent
[oracle@appsractest bin]$ /u01/app/11.2.0.2/grid/bin/netca /orahome /u01/app/11.2.0.2/grid /orahnam OraHome1Grid /instype typical /inscomp client,oraclenet,javavm,server /insprtcl tcp /cfg local /authadp NO_VALUE /responseFile /u01/app/11.2.0.2/grid/network/install/netca_typ.rsp /silent
Parsing command line arguments:
Parameter "orahome" = /u01/app/11.2.0.2/grid
Parameter "orahnam" = OraHome1Grid
Parameter "instype" = typical
Parameter "inscomp" = client,oraclenet,javavm,server
Parameter "insprtcl" = tcp
Parameter "cfg" = local
Parameter "authadp" = NO_VALUE
Parameter "responsefile" = /u01/app/11.2.0.2/grid/network/install/netca_typ.rsp
Parameter "silent" = true
Done parsing command line arguments.
Oracle Net Services Configuration:
Oracle Net Listener Startup:
Listener started successfully.
Listener configuration complete.
Oracle Net Services configuration successful. The exit code is 0
-- If you use a supported cluster file system, then you do not need to execute this command. If you are using Oracle ASM, then run the following command:
[oracle] $ /u01/app/11.2.0/grid/bin/asmca -silent -postConfigureASM -sysAsmPassword oracle -asmsnmpPassword oracle
-- Setup Oracle ASM password..
[oracle@appsractest bin]$ /u01/app/11.2.0.2/grid/bin/asmca -silent -postConfigureASM -sysAsmPassword oracle -asmsnmpPassword oracle
PostConfiguration completed successfully
Now if you check your Grid with CRS is up and running. Next step will be to clone Oracle RAC Binaries or Install them as per your need. I will put that topic in my next post.
One last thing is that you have to repeat this process for all the nodes on your destination cluster, so make sure you follow all the above mentioned steps. You can do some of this steps at a time on multiple nodes but doing them one by one will give you better control and ease of trouble shooting.
Note - If you a plan to run a pre-11g release 2 (11.2) database on this cluster, then you should run oifcfg as described in the Oracle Database 11g release 2 (11.2) documentation
