elcarOnOsdnaH: January 2014

Upgrade OEM 11g to 12C - One System Approach

There are two different ways to upgrade OEM Grid Control environment.

1. One system upgrade

2. Two system upgrade.

One system upgrade is nothing but the upgrading the existing environment. In this note, I will describe how to do a 1-System Upgrade Approach to upgrade an existing 11.1.0.1 Grid Control environment to OEM 12c.

We will then see how to upgrade one of the existing 11g Management Agents to 12c.

Note that the OMS (Oracle Management Service) 12c communicates only with Oracle Management Agent 12c.

1. The first step here is to apply the PSU on existing 11g Environment.

At time of this application I have applied PSU 8. However currently PSU 10 is also available so you can also apply PSU 10.

Running the PSU pre-requisite

a)[oracle@Host11G software]$ export ORACLE_HOME=/u01/app/oracle/OEM11g/Middleware/oms11g

Shutdown OMS before applying the patch

b)[oracle@Host11G software]$ /u01/app/oracle/OEM11g/Middleware/oms11g/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /orabkup/oracle/software/p10065631_111010_Generic/10065631

Invoking OPatch 11.1.0.8.0

Oracle Interim Patch Installer version 11.1.0.8.0

Patch history file: /u01/app/oracle/OEM11g/Middleware/oms11g/cfgtoollogs/opatch/opatch_history.txt

OPatch detects the Middleware Home as "/u01/app/oracle/OEM11g/Middleware"

…………………..

Prereq "checkConflictAgainstOHWithDetail" passed.

OPatch succeeded.

c) Applying the PSU

Shutdown OMS before applying the patch

[oracle@Host11G bin]$ ./emctl stop oms

[oracle@Host11G 10065631]$ $ORACLE_HOME/OPatch/opatch apply

Invoking OPatch 11.1.0.8.0

Oracle Home : /u01/app/oracle/OEM11g/Middleware/oms11g

Central Inventory : /u01/app/oraInventory

from : /etc/oraInst.loc

OPatch version : 11.1.0.8.0

OUI version : 11.1.0.8.0

OUI location : /u01/app/oracle/OEM11g/Middleware/oms11g/oui

Log file location : /u01/app/oracle/OEM11g/Middleware/oms11g/cfgtoollogs/opatch/opatch2013-06-28_05-32-20AM.log

Running prerequisite checks...

Patching component oracle.sysman.oms.core, 11.1.0.1.0...

Updating jar file "/u01/app/oracle/OEM11g/Middleware/oms11g/sysman/jlib/emclikit.jar" with "/sysman/jlib/emclikit.jar/emCLI.jar/oracle/sysman/emCLI/CLILogger.class"

Copying file to "/u01/app/oracle/OEM11g/Middleware/oms11g/sysman/plugins/connector/install_remedycntr.pl"

Copying file to "/u01/app/oracle/OEM11g/Middleware/oms11g/sysman/omsca/scripts/wls/create_domain.py"

Copying file to "/u01/app/oracle/OEM11g/Middleware/oms11g/bin/SecureOMSCmds.pm"

Copying file to "/u01/app/oracle/OEM11g/Middleware/oms11g/sysman/emdrep/scripts/SecureAgent_oms.pl"

ApplySession adding interim patch '10065631' to inventory

Verifying the update...

Inventory check OK: Patch ID 10065631 is registered in Oracle Home inventory with proper meta-data.

Files check OK: Files from Patch ID 10065631 are present in Oracle Home.

Execution of 'sh /orabkup/oracle/software/p10065631_111010_Generic/10065631/custom/scripts/post -apply 10065631 ':

Return Code = 0

OPatch succeeded.

[oracle@Host11G 10065631]$

d) [oracle@Host11G 10065631]$ echo $PWD

/orabkup/oracle/software/p10065631_111010_Generic/10065631

$ORACLE_HOME/bin/rcuJDBCEngine sys/xxxx@<REPO-HOST>:1521:omsupg JDBC_SCRIPT post_install_script.sql $PWD $ORACLE_HOME

Statement Type: 'BEGIN/END Anonymous Block'

Completed SQL script execution normally.

1 scripts were processed

[oracle@Host11G 10065631]$ $ORACLE_HOME/bin/rcuJDBCEngine sys/n4u2ck@<REPO-HOST>:1521:omsupg JDBC_SCRIPT post_install_patch.sql $PWD $ORACLE_HOME

output:

Extracting Statement from File Name: 'post_install_patch.sql' Line Number: 316

Extracted SQL Statement: [truncate table bundle_component_files]

Statement Type: 'DDL Statement'

Executing SQL statement: truncate table bundle_component_files

Extracting Statement from File Name: 'post_install_patch.sql' Line Number: 317

Extracted SQL Statement: [drop table bundle_component_files]

Statement Type: 'DROP Statement'

Completed SQL script execution normally.

8 scripts were processed

e) [oracle@Host11G bin]$ ./emctl start oms

Oracle Enterprise Manager 11g Release 1 Grid Control

Starting WebTier...

WebTier Successfully Started

Starting Oracle Management Server...

Oracle Management Server Successfully Started

Oracle Management Server is Up

WARNING: Limit of open file descriptors is found to be 1024.

The OMS has been started but it may run out of descriptors under heavy usage.

For proper functioning of OMS, please set "ulimit -n" to be at least 4096.

f) Then again stop the OMS

[oracle@Host11G bin]$ ./emctl stop oms

Oracle Enterprise Manager 11g Release 1 Grid Control

Stopping WebTier...

WebTier Successfully Stopped

Stopping Oracle Management Server...

Oracle Management Server Successfully Stopped

Oracle Management Server is Down

g.) Installing the Oracle Grid Control 12c Pre-upgrade Patch

Once PSU is applied the next step is to install 12c pre-upgrade patch. This patch introduces the grid upgrade console in 11g console.

[oracle@Host11G 14375077]$ $ORACLE_HOME/OPatch/opatch apply

Invoking OPatch 11.1.0.8.0

Oracle Interim Patch Installer version 11.1.0.8.0

Oracle Home : /u01/app/oracle/OEM11g/Middleware/oms11g

Central Inventory : /u01/app/oraInventory

from : /etc/oraInst.loc

OPatch version : 11.1.0.8.0

OUI version : 11.1.0.8.0

OUI location : /u01/app/oracle/OEM11g/Middleware/oms11g/oui

Log file location : /u01/app/oracle/OEM11g/Middleware/oms11g/cfgtoollogs/opatch/opatch2013-06-28_05-58-07AM.log

Patch history file: /u01/app/oracle/OEM11g/Middleware/oms11g/cfgtoollogs/opatch/opatch_history.txt

OPatch detects the Middleware Home as "/u01/app/oracle/OEM11g/Middleware"

ApplySession applying interim patch '14375077' to OH '/u01/app/oracle/OEM11g/Middleware/oms11g'

Running prerequisite checks...

OPatch detected non-cluster Oracle Home from the inventory and will patch the local system only.

Backing up files and inventory (not for auto-rollback) for the Oracle Home

Backing up files affected by the patch '14375077' for restore. This might take a while...

Backing up files affected by the patch '14375077' for rollback. This might take a while...

………………………………………

Verifying the update...

Inventory check OK: Patch ID 14375077 is registered in Oracle Home inventory with proper meta-data.

Files check OK: Files from Patch ID 14375077 are present in Oracle Home.

OPatch succeeded.

g.) Run Post Upgrade scripts

[oracle@Host11G bin]$rcuJDBCEngine sys/XXX@<REPO-HOST>:1521:omsupg JDBC_SCRIPT /u01/app/oracle/OEM11g/Middleware/oms11g/sysman/preupgc/puc_dblink_pkgdef.sql

Example -

[oracle@Host11G bin]$ rcuJDBCEngine sys/XXXX@<REPO-HOST>:1521:omsupg JDBC_SCRIPT /u01/app/oracle/OEM11g/Middleware/oms11g/sysman/preupgc/puc_dblink_pkgdef.sql

Extracting Statement from File Name: '/u01/app/oracle/OEM11g/Middleware/oms11g/sysman/preupgc/puc_dblink_pkgdef.sql' Line Number: 20

………………….

Statement Type: 'PL/SQL Statement - Object Type:PACKAGE Object Name:PRE_UPG_DB_LINKS'

Completed SQL script execution normally.

1 scripts were processed

[oracle@Host11G bin]$

[oracle@Host11G bin]$rcuJDBCEngine sys/xxxxxx@<REPO-HOST>:1521:omsupg JDBC_SCRIPT /u01/app/oracle/OEM11g/Middleware/oms11g/sysman/preupgc/puc_dblink_pkgbody.sql

Example -

[oracle@Host11G bin]$ ./rcuJDBCEngine sys/xxxxxx@<REPO-HOST>:1521:omsupg JDBC_SCRIPT /u01/app/oracle/OEM11g/Middleware/oms11g/sysman/preupgc/puc_dblink_pkgbody.sql

Extracting Statement from File Name: '/u01/app/oracle/OEM11g/Middleware/oms11g/sysman/preupgc/puc_dblink_pkgbody.sql' Line Number: 20

Extracted SQL Statement: [CREATE OR REPLACE PACKAGE BODY PRE_UPG_DB_LINKS

Statement Type: 'PL/SQL Statement - Object Type:PACKAGE BODY Object Name:PRE_UPG_DB_LINKS'

Completed SQL script execution normally.

1 scripts were processed

[oracle@Host11G bin]$

[oracle@Host11G 14375077]$rcuJDBCEngine sys/xxxx@<REPO-HOST>:1521:omsupg JDBC_SCRIPT /u01/app/oracle/OEM11g/Middleware/oms11g/sysman/preupgc/pre_upg_console.sql

.sql

Error:

>>>A SQL Error occurred!: Error creating PL/SQL Object EM_PLUGIN_UPG_SUPPORT:PACKAGE BODY

oracle.sysman.assistants.common.dbutil.SQLFatalErrorException: Error creating PL/SQL Object EM_PLUGIN_UPG_SUPPORT:PACKAGE BODY

at oracle.sysman.assistants.common.dbutil.jdbc.JDBCEngine.parseNexecuteScript(JDBCEngine.java:1139)

at oracle.sysman.assistants.common.dbutil.jdbc.JDBCEngine.main(JDBCEngine.java:1992)

Workaround: Use sysman account instead of sys account;

[oracle@Host11G bin]$./rcuJDBCEngine sysman/XXXXX@<REPO-HOST>:1521:omsupg JDBC_SCRIPT /u01/app/oracle/OEM11g/Middleware/oms11g/sysman/preupgc/pre_upg_console.sql

Output:

Extracting Statement from File Name: '/u01/app/oracle/OEM11g/Middleware/oms11g/sysman/preupgc/RefreshAndValidateJob.sql' Line Number: 346

Extracted SQL Statement: [COMMIT;]

………….

Skipping Unsupported Statement

Statement Type: 'Oracle RCU NotSupported SQLPlus Statement'

Completed SQL script execution normally.

23 scripts were processed

[oracle@Host11G bin]$

h.) Start the OMS and check for the upgrade console

[oracle@Host11G bin]$ ./emctl start oms

Oracle Enterprise Manager 11g Release 1 Grid Control

Starting WebTier...

WebTier Successfully Started

Starting Oracle Management Server...

Oracle Management Server Successfully Started

Oracle Management Server is Up

WARNING: Limit of open file descriptors is found to be 1024.

The OMS has been started but it may run out of descriptors under heavy usage.

For proper functioning of OMS, please set "ulimit -n" to be at least 4096.

[oracle@Host11G bin]$

i.) [oracle@Host11G bin]$ ./emctl status agent

Oracle Enterprise Manager Cloud Control 12c Release 2

---------------------------------------------------------------

Agent Version : 12.1.0.2.0

OMS Version : (unknown)

Protocol Version : 12.1.0.1.0

Agent Home : /u01/app/oracle/OEM11g/Middleware/agent12g/agent_inst

Agent Binaries : /u01/app/oracle/OEM11g/Middleware/agent12g/core/12.1.0.2.0

Agent Process ID : 27785

Parent Process ID : 27620

Agent URL : https://Host11G.searshc.com:1830/emd/main/

Repository URL : https://Host11G.searshc.com:4900/empbs/upload

Started at : 2013-06-28 09:34:36

Started by user : oracle

Last Reload : (none)

Last successful upload : (none)

Last attempted upload : (none)

Total Megabytes of XML files uploaded so far : 0

Number of XML files pending upload : 95

Size of XML files pending upload(MB) : 0.08

Available disk space on upload filesystem : 36.57%

Collection Status : Collections enabled

Heartbeat Status : OMS is unreachable

Last attempted heartbeat to OMS : 2013-06-28 09:38:45

Last successful heartbeat to OMS : (none)

Next scheduled heartbeat to OMS : 2013-06-28 09:39:15

---------------------------------------------------------------

Agent is Running and Ready

[oracle@Host11G bin]$

ONCE THIS IS DONE YOUR PRE-UPGRADE PART IS DONE.

NOW ONE NEEDS TO DOWNLOAD THE PLUGINS REQUIRED FOR 12c FROM OTN AND MAKE THEM AVAILABLE TO 11G OMS HOST SO THAT THEY CAN BE VALIDATED.

ONCE THEY ARE VALIDATED YOU NEED TO CHECK UPGRADE CONSOLE LINK FROM 11G OMS AND MAKE SURE IT LOOKS LIKE FOLLOWING SCREEN. THERE SHOULD BE NO YELLOW OR RED ONES.

UPGRADE TO OEM12c USING ONE SYSTEM APPROACH

HERE THE EXISTING OMS IS GOING TO GET UPDATED HENCE ENTERPRISE MONITORING WILL BE COMPLETELY DOWN DURING UPGRADE PERIOD WHICH DIRECTLY DEPENDS ON THE SIZE OF YOUR ENVIRONMENT I.E. NO OF MONITORED TARGETS.

[oracle@Host11G em12c]$ ./runInstaller

Starting Oracle Universal Installer...

Error -

The below specified Management Plug-Ins (in the format "PluginID:PluginVersion:PluginUpdate") are registered with the 12c Upgrade Console, but are not present for installation of the current Management Service. In order to proceed, you need to install the following plugins :

oracle.sysman.db:12.1.0.3.0:20130402

Ensure that you download and make the software for those Management Plug-Ins available. Once they are available, for installing these required plugins, exit the installer and invoke the installer by passing the "-pluginLocation <absolute_path_to_plugin_archives_location>" to the runInstaller or setup.exe

Fix – download disk1 also and unzip it

MAKE SURE YOU ATTEND ALL THE WARNINGS.

ANY AVOIDED WARNING MAY RESULT IN FAILED ISNTALLATION.

[oracle@ushofsvpracbl18 dbs]$ !sqlp

sqlplus "/ as sysdba"

SQL*Plus: Release 11.2.0.3.0 Production on Fri Jun 28 12:09:47 2013

Connected to:

Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production

With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,

Data Mining and Real Application Testing options

SQL> grant execute on dbms_random to dbsnmp;

Grant succeeded.

SQL> grant execute on dbms_random to sysman;

Grant succeeded.

SQL> alter user oracle_ocm account unlock identified by oracle_ocm;

User altered.

[root@Host11G ~]# cd /u01/app/oracle/OEM12c/Middleware/oms/

[root@Host11G oms]# ./allroot.sh

Starting to execute allroot.sh .........

Starting to execute /u01/app/oracle/OEM12c/Middleware/oms/root.sh ......

Running Oracle 11g root.sh script...

The following environment variables are set as:

ORACLE_OWNER= oracle

ORACLE_HOME= /u01/app/oracle/OEM12c/Middleware/oms

Enter the full pathname of the local bin directory: [/usr/local/bin]:

The file "dbhome" already exists in /usr/local/bin. Overwrite it? (y/n)

[n]:

The file "oraenv" already exists in /usr/local/bin. Overwrite it? (y/n)

[n]:

The file "coraenv" already exists in /usr/local/bin. Overwrite it? (y/n)

[n]:

Entries will be added to the /etc/oratab file as needed by

Database Configuration Assistant when a database is created

Finished running generic part of root.sh script.

Now product-specific root actions will be performed.

/etc exist

/u01/app/oracle/OEM12c/Middleware/oms

Finished execution of /u01/app/oracle/OEM12c/Middleware/oms/root.sh ......

This information is also available at:

/u01/app/oracle/OEM12c/Middleware/oms/install/setupinfo.txt

See below for information pertaining to your Enterprise Manager installation:

Use the following URL to access:

1. Enterprise Manager Cloud Control URL: https://Host11G.searshc.com:7799/em

2. Admin Server URL: https://Host11G.searshc.com:7101/console

The following details need to be provided during the additional OMS install:

1. Admin Server Hostname: Host11G.searshc.com

2. Admin Server Port: 7101

NOTE:

An encryption key has been generated to encrypt sensitive data in the Management Repository. If this key is lost, all encrypted data in the Repository becomes unusable. Please run following command to backup the Management Service configuration including the emkey and keep the backup on another system:

emctl exportconfig oms -dir <backup location>

You can either set the "oracle.sysman.prov.agentpush.enablePty" property to true in the "/u01/app/oracle/OEM12c/Middleware/oms/sysman/prov/agentpush/agentpush.properties" file, pass -S argument to the sudo command, or enable the "visiblepw" flag in the sudoers file.

[oracle@Host11G agentpush]$ pwd

/u01/app/oracle/OEM12c/Middleware/oms/sysman/prov/agentpush

[oracle@Host11G agentpush]$ vi agentpush.properties

#The value of this property decides whether a pseudo terminal needs to be allocated while executing the pdp utility over ssh

oracle.sysman.prov.agentpush.enablePty=true

THIS WILL COMPLETE YOUR ONE SYSTEM UPGRADE.
In next post I will describe how to perform 2-system upgrade...

Tuesday, January 14, 2014

Upgrade OEM 11g to 12C - One System Approach