{"id":2064,"date":"2013-03-10T08:21:59","date_gmt":"2013-03-10T08:21:59","guid":{"rendered":"http:\/\/www.oratraining.com\/blog\/?p=2064"},"modified":"2013-03-19T09:33:53","modified_gmt":"2013-03-19T09:33:53","slug":"installing-11g-rac-on-linux-vm-install-oracle-clusterware","status":"publish","type":"post","link":"https:\/\/www.oratraining.com\/blog\/2013\/03\/installing-11g-rac-on-linux-vm-install-oracle-clusterware\/","title":{"rendered":"Installing 11g RAC on Linux VM: Install Oracle Clusterware"},"content":{"rendered":"<p style=\"text-align: justify;\"><strong>Previous:<\/strong>\u00a0<a title=\"Copy\/clone this virtual machine to create second node and modify host details\" href=\"http:\/\/www.oratraining.com\/blog\/2013\/03\/copyclone-this-virtual-machine-to-create-second-node-and-modify-host-details\/\"><strong>Copy\/clone this virtual machine to create second node and modify host details<\/strong><\/a><\/p>\n<p style=\"text-align: justify;\">The clusterware or database installation needs to be started only on one node since it will propagate the files to remote node automatically during the installation.<\/p>\n<p style=\"text-align: justify;\">If you have the setup files on your host machine, you can share the setup folder to the VM using VMWare or Oracle VirtualBox shared folder option.<\/p>\n<p style=\"text-align: justify;\">Following screen shows how to share a folder to VM using VMWare. This can be done even when the VM is online.<\/p>\n<p style=\"text-align: justify;\"><img decoding=\"async\" alt=\"\" src=\"http:\/\/www.oratraining.com\/blog\/wp-content\/uploads\/2013\/03\/031813_0817_Installing11.png\" \/><\/p>\n<p style=\"text-align: justify;\">The files which you share using above option will be available by default at \/mnt\/hgfs\/ directory under Linux.<\/p>\n<p style=\"text-align: justify;\">Now let us start the Oracle Clusterware installation.<\/p>\n<p style=\"text-align: justify;\">Login with <strong>oracle<\/strong> user (grid owner)<\/p>\n<p style=\"text-align: justify;\">[oracle@dbhost1 ~]$ <strong>cd \/mnt\/hgfs\/setup\/grid\/<\/strong><\/p>\n<p style=\"text-align: justify;\">Start the installation using .\/runInstaller script<\/p>\n<p style=\"text-align: justify;\">[oracle@dbhost1 grid]$ <strong>.\/runInstaller<\/strong><\/p>\n<p style=\"text-align: justify;\">\u2026<\/p>\n<p>&nbsp;<\/p>\n<p><img decoding=\"async\" alt=\"\" src=\"http:\/\/www.oratraining.com\/blog\/wp-content\/uploads\/2013\/03\/031813_0817_Installing12.png\" \/><\/p>\n<p>&nbsp;<\/p>\n<p><img decoding=\"async\" alt=\"\" src=\"http:\/\/www.oratraining.com\/blog\/wp-content\/uploads\/2013\/03\/031813_0817_Installing13.png\" \/><\/p>\n<p>&nbsp;<\/p>\n<p>Select &#8220;<strong>Install and Configure Grid Infrastructure for a Cluster<\/strong>&#8221; and click <strong>Next<\/strong><\/p>\n<p>&nbsp;<\/p>\n<p><img decoding=\"async\" alt=\"\" src=\"http:\/\/www.oratraining.com\/blog\/wp-content\/uploads\/2013\/03\/031813_0817_Installing14.png\" \/><\/p>\n<p>&nbsp;<\/p>\n<p>Select &#8220;<strong>Advanced Installation<\/strong>&#8221; and click <strong>Next<\/strong><\/p>\n<p>&nbsp;<\/p>\n<p><img decoding=\"async\" alt=\"\" src=\"http:\/\/www.oratraining.com\/blog\/wp-content\/uploads\/2013\/03\/031813_0817_Installing15.png\" \/><\/p>\n<p>&nbsp;<\/p>\n<p>Click <strong>Next<\/strong><\/p>\n<p>&nbsp;<\/p>\n<p><img decoding=\"async\" alt=\"\" src=\"http:\/\/www.oratraining.com\/blog\/wp-content\/uploads\/2013\/03\/031813_0817_Installing16.png\" \/><\/p>\n<p>&nbsp;<\/p>\n<p>Enter details as follows and click <strong>Next. <\/strong>You can change the values as you want but make sure to use same values as you entered if required in other screens.<\/p>\n<p>&nbsp;<\/p>\n<p>Cluster Name: dbhost-cluster<\/p>\n<p>SCAN Name: dbhost-scan.paramlabs.com<\/p>\n<p>SCAN Port: 1521<\/p>\n<p>&nbsp;<\/p>\n<p><img decoding=\"async\" alt=\"\" src=\"http:\/\/www.oratraining.com\/blog\/wp-content\/uploads\/2013\/03\/031813_0817_Installing17.png\" \/><\/p>\n<p>&nbsp;<\/p>\n<p>It will validate the name entered for SCAN.<\/p>\n<p>&nbsp;<\/p>\n<p><img decoding=\"async\" alt=\"\" src=\"http:\/\/www.oratraining.com\/blog\/wp-content\/uploads\/2013\/03\/031813_0817_Installing18.png\" \/><br \/>\nSince we have started installation on node1 and it yet does not recognize node2, it will only show 1 node here. Now we need to manually add another node in the cluster. Click <strong>Add<\/strong><\/p>\n<p>&nbsp;<\/p>\n<p><img decoding=\"async\" alt=\"\" src=\"http:\/\/www.oratraining.com\/blog\/wp-content\/uploads\/2013\/03\/031813_0817_Installing19.png\" \/><\/p>\n<p>&nbsp;<\/p>\n<p>Enter hostname and VIP name for second node. Click <strong>Ok<\/strong><\/p>\n<p>&nbsp;<\/p>\n<p><img decoding=\"async\" alt=\"\" src=\"http:\/\/www.oratraining.com\/blog\/wp-content\/uploads\/2013\/03\/031813_0817_Installing110.png\" \/><\/p>\n<p>&nbsp;<\/p>\n<p>Now both nodes will appear on the screen. Make sure to select both values and then click <strong>Next<\/strong><\/p>\n<p>&nbsp;<\/p>\n<p><img decoding=\"async\" alt=\"\" src=\"http:\/\/www.oratraining.com\/blog\/wp-content\/uploads\/2013\/03\/031813_0817_Installing111.png\" \/><\/p>\n<p>&nbsp;<\/p>\n<p>It will do various tests including ssh connectivity, node readiness, user equivalence and check for existing public\/private interfaces on the hosts.<\/p>\n<p>&nbsp;<\/p>\n<p><img decoding=\"async\" alt=\"\" src=\"http:\/\/www.oratraining.com\/blog\/wp-content\/uploads\/2013\/03\/031813_0817_Installing112.png\" \/><\/p>\n<p>&nbsp;<\/p>\n<p><img decoding=\"async\" alt=\"\" src=\"http:\/\/www.oratraining.com\/blog\/wp-content\/uploads\/2013\/03\/031813_0817_Installing113.png\" \/><\/p>\n<p>&nbsp;<\/p>\n<p><img decoding=\"async\" alt=\"\" src=\"http:\/\/www.oratraining.com\/blog\/wp-content\/uploads\/2013\/03\/031813_0817_Installing114.png\" \/><\/p>\n<p>&nbsp;<\/p>\n<p><img decoding=\"async\" alt=\"\" src=\"http:\/\/www.oratraining.com\/blog\/wp-content\/uploads\/2013\/03\/031813_0817_Installing115.png\" \/><\/p>\n<p>&nbsp;<\/p>\n<p>It will detect eth0 as public and eth1 as private. This is exactly what we want. If not detected automatically, set as above and click <strong>Next<\/strong><\/p>\n<p>&nbsp;<\/p>\n<p><img decoding=\"async\" alt=\"\" src=\"http:\/\/www.oratraining.com\/blog\/wp-content\/uploads\/2013\/03\/031813_0817_Installing116.png\" \/><\/p>\n<p>&nbsp;<\/p>\n<p>Select <strong>Shared File system<\/strong> and click <strong>Next<\/strong><\/p>\n<p>&nbsp;<\/p>\n<p><img decoding=\"async\" alt=\"\" src=\"http:\/\/www.oratraining.com\/blog\/wp-content\/uploads\/2013\/03\/031813_0817_Installing117.png\" \/><\/p>\n<p>&nbsp;<\/p>\n<p><img decoding=\"async\" alt=\"\" src=\"http:\/\/www.oratraining.com\/blog\/wp-content\/uploads\/2013\/03\/031813_0817_Installing118.png\" \/><\/p>\n<p>&nbsp;<\/p>\n<p><span style=\"color: red;\"><strong>Important Note:<\/strong> Since having normal redundancy increases the load on VM to make sure NFS communicates and keeps all these files in sync all the time, we might face issues during normal run of the Oracle RAC.<br \/>\n<\/span><\/p>\n<p>&nbsp;<\/p>\n<p><span style=\"color: red;\">So please select <strong>EXTERNAL REDUNDANCY<\/strong> which means that you are going to mirror this disk using external methods. Though we are not using any external mirroring here but still for the non-production and learning purpose we can take this risk.<br \/>\n<\/span><\/p>\n<p>&nbsp;<\/p>\n<p>Enter following value in External Redundancy box. Click <strong>Next<\/strong><\/p>\n<p><strong>\/u01\/storage\/ocr<br \/>\n<\/strong><\/p>\n<p><img decoding=\"async\" alt=\"\" src=\"http:\/\/www.oratraining.com\/blog\/wp-content\/uploads\/2013\/03\/031813_0817_Installing119.png\" \/><\/p>\n<p>&nbsp;<\/p>\n<p><span style=\"color: red;\"><strong>Important Note:<\/strong> Since having normal redundancy increases the load on VM to make sure NFS communicates and keeps all these files in sync all the time, we might face issues during normal run of the Oracle RAC.<br \/>\n<\/span><\/p>\n<p>&nbsp;<\/p>\n<p><span style=\"color: red;\">So please select <strong>EXTERNAL REDUNDANCY<\/strong> which means that you are going to mirror this disk using external methods. Though we are not using any external mirroring here but still for the non-production and learning purpose we can take this risk.<br \/>\n<\/span><\/p>\n<p>&nbsp;<\/p>\n<p>Enter following value in External Redundancy box. Click <strong>Next<\/strong><\/p>\n<p><strong>\/u01\/storage\/vdsk<br \/>\n<\/strong><\/p>\n<p>&nbsp;<\/p>\n<p><img decoding=\"async\" alt=\"\" src=\"http:\/\/www.oratraining.com\/blog\/wp-content\/uploads\/2013\/03\/031813_0817_Installing120.png\" \/><\/p>\n<p>&nbsp;<\/p>\n<p>Select <strong>Do not use IPMI <\/strong>and click <strong>Next<\/strong><\/p>\n<p>&nbsp;<\/p>\n<p><img decoding=\"async\" alt=\"\" src=\"http:\/\/www.oratraining.com\/blog\/wp-content\/uploads\/2013\/03\/031813_0817_Installing121.png\" \/><\/p>\n<p>&nbsp;<\/p>\n<p>We have selected <strong>dba<\/strong> for all the above groups, you can choose different if you wish to. Click <strong>Next<\/strong><\/p>\n<p>&nbsp;<\/p>\n<p><img decoding=\"async\" alt=\"\" src=\"http:\/\/www.oratraining.com\/blog\/wp-content\/uploads\/2013\/03\/031813_0817_Installing122.png\" \/><\/p>\n<p>&nbsp;<\/p>\n<p>If you chose <strong>dba<\/strong> as all above groups, you might see above message box, click <strong>Yes<\/strong><\/p>\n<p>&nbsp;<\/p>\n<p><img decoding=\"async\" alt=\"\" src=\"http:\/\/www.oratraining.com\/blog\/wp-content\/uploads\/2013\/03\/031813_0817_Installing123.png\" \/><\/p>\n<p>&nbsp;<\/p>\n<p>Enter following values and click <strong>Next<\/strong><br \/>\nOracle Base: <strong>\/app\/oracle<\/strong><\/p>\n<p>Software Location: <strong>\/app\/11.2.0\/grid<\/strong><\/p>\n<p>&nbsp;<\/p>\n<p><img decoding=\"async\" alt=\"\" src=\"http:\/\/www.oratraining.com\/blog\/wp-content\/uploads\/2013\/03\/031813_0817_Installing124.png\" \/><\/p>\n<p>&nbsp;<\/p>\n<p><img decoding=\"async\" alt=\"\" src=\"http:\/\/www.oratraining.com\/blog\/wp-content\/uploads\/2013\/03\/031813_0817_Installing125.png\" \/><\/p>\n<p>&nbsp;<\/p>\n<p>Enter <strong>\/app\/oraInventory<\/strong> for the Inventory location. Click <strong>Next<\/strong><\/p>\n<p>&nbsp;<\/p>\n<p><img decoding=\"async\" alt=\"\" src=\"http:\/\/www.oratraining.com\/blog\/wp-content\/uploads\/2013\/03\/031813_0817_Installing126.png\" \/><\/p>\n<p>&nbsp;<\/p>\n<p><img decoding=\"async\" alt=\"\" src=\"http:\/\/www.oratraining.com\/blog\/wp-content\/uploads\/2013\/03\/031813_0817_Installing127.png\" \/><\/p>\n<p>&nbsp;<\/p>\n<p>You might see above failed pre-requisites. We have specifically not applied the above pre-reqs just to show you that now Oracle will generate a script to fix all required pre-reqs in this screen where the value in column &#8220;<strong>Fixable<\/strong>&#8221; is <strong>Yes<\/strong><\/p>\n<p>&nbsp;<\/p>\n<p>Regarding physical memory and swap, we can ignore these. Click on &#8220;<strong>Fix and Check Again<\/strong>&#8221; to generate the fix script<\/p>\n<p>&nbsp;<\/p>\n<p><img decoding=\"async\" alt=\"\" src=\"http:\/\/www.oratraining.com\/blog\/wp-content\/uploads\/2013\/03\/031813_0817_Installing128.png\" \/><\/p>\n<p>&nbsp;<\/p>\n<p>It will show above screen with location of <strong>runfixup.sh<\/strong> script which you need to <strong>run as root<\/strong><br \/>\n<strong>on both nodes<\/strong>.<\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify;\">[root@dbhost1 ~]# <strong>\/tmp\/CVU_11.2.0.1.0_grid\/runfixup.sh<\/strong><\/p>\n<p style=\"text-align: justify;\">Response file being used is :\/tmp\/CVU_11.2.0.1.0_grid\/fixup.response<\/p>\n<p style=\"text-align: justify;\">Enable file being used is :\/tmp\/CVU_11.2.0.1.0_grid\/fixup.enable<\/p>\n<p style=\"text-align: justify;\">Log file location: \/tmp\/CVU_11.2.0.1.0_grid\/orarun.log<\/p>\n<p style=\"text-align: justify;\">uid=54322(grid) gid=54322(dba) groups=54322(dba),54321(oinstall)<\/p>\n<p style=\"text-align: justify;\">[root@dbhost2 ~]# <strong>\/tmp\/CVU_11.2.0.1.0_grid\/runfixup.sh<\/strong><\/p>\n<p style=\"text-align: justify;\">Response file being used is :\/tmp\/CVU_11.2.0.1.0_grid\/fixup.response<\/p>\n<p style=\"text-align: justify;\">Enable file being used is :\/tmp\/CVU_11.2.0.1.0_grid\/fixup.enable<\/p>\n<p style=\"text-align: justify;\">Log file location: \/tmp\/CVU_11.2.0.1.0_grid\/orarun.log<\/p>\n<p style=\"text-align: justify;\">uid=54322(grid) gid=54322(dba) groups=54322(dba),54321(oinstall)<\/p>\n<p>&nbsp;<\/p>\n<p><img decoding=\"async\" alt=\"\" src=\"http:\/\/www.oratraining.com\/blog\/wp-content\/uploads\/2013\/03\/031813_0817_Installing129.png\" \/><\/p>\n<p>&nbsp;<\/p>\n<p>Now it will only show above 2 memory related errors. We can ignore them. <strong>Check Ignore All<\/strong><\/p>\n<p>&nbsp;<\/p>\n<p><img decoding=\"async\" alt=\"\" src=\"http:\/\/www.oratraining.com\/blog\/wp-content\/uploads\/2013\/03\/031813_0817_Installing130.png\" \/><\/p>\n<p>&nbsp;<\/p>\n<p>Click <strong>Next<br \/>\n<\/strong><\/p>\n<p>&nbsp;<\/p>\n<p><img decoding=\"async\" alt=\"\" src=\"http:\/\/www.oratraining.com\/blog\/wp-content\/uploads\/2013\/03\/031813_0817_Installing131.png\" \/><\/p>\n<p>&nbsp;<\/p>\n<p>Review the summary and click <strong>Finish<\/strong> to begin installation. If you wish you can save the response file as follows before clicking on finish.<\/p>\n<p>&nbsp;<\/p>\n<p><img decoding=\"async\" alt=\"\" src=\"http:\/\/www.oratraining.com\/blog\/wp-content\/uploads\/2013\/03\/031813_0817_Installing132.png\" \/><\/p>\n<p>&nbsp;<\/p>\n<p><img decoding=\"async\" alt=\"\" src=\"http:\/\/www.oratraining.com\/blog\/wp-content\/uploads\/2013\/03\/031813_0817_Installing133.png\" \/><\/p>\n<p>&nbsp;<\/p>\n<p>After installation is finished on node 1, it will propagate the files on node 2. After which it will need to run some scripts as root user.<\/p>\n<p>&nbsp;<\/p>\n<p><img decoding=\"async\" alt=\"\" src=\"http:\/\/www.oratraining.com\/blog\/wp-content\/uploads\/2013\/03\/031813_0817_Installing134.png\" \/><\/p>\n<p>&nbsp;<\/p>\n<p>Run first script on both nodes and then second script on both nodes.<\/p>\n<p>&nbsp;<\/p>\n<p>[root@dbhost1 ~]# <strong>\/app\/oraInventory\/orainstRoot.sh<\/strong><\/p>\n<p>Changing permissions of \/app\/oraInventory.<\/p>\n<p>Adding read,write permissions for group.<\/p>\n<p>Removing read,write,execute permissions for world.<\/p>\n<p>&nbsp;<\/p>\n<p>Changing groupname of \/app\/oraInventory to dba.<\/p>\n<p>The execution of the script is complete.<\/p>\n<p>&nbsp;<\/p>\n<p>[root@dbhost2 ~]# <strong>\/app\/oraInventory\/orainstRoot.sh<\/strong><\/p>\n<p>Changing permissions of \/app\/oraInventory.<\/p>\n<p>Adding read,write permissions for group.<\/p>\n<p>Removing read,write,execute permissions for world.<\/p>\n<p>&nbsp;<\/p>\n<p>Changing groupname of \/app\/oraInventory to dba.<\/p>\n<p>The execution of the script is complete.<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>[root@dbhost1 ~]# <strong>\/app\/11.2.0\/grid\/root.sh<\/strong><\/p>\n<p>Running Oracle 11g root.sh script&#8230;<\/p>\n<p>&nbsp;<\/p>\n<p>The following environment variables are set as:<\/p>\n<p>ORACLE_OWNER= oracle<\/p>\n<p>ORACLE_HOME= \/app\/11.2.0\/grid<\/p>\n<p>&nbsp;<\/p>\n<p>Enter the full pathname of the local bin directory: [\/usr\/local\/bin]:<\/p>\n<p>Copying dbhome to \/usr\/local\/bin &#8230;<\/p>\n<p>Copying oraenv to \/usr\/local\/bin &#8230;<\/p>\n<p>Copying coraenv to \/usr\/local\/bin &#8230;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>Creating \/etc\/oratab file&#8230;<\/p>\n<p>Entries will be added to the \/etc\/oratab file as needed by<\/p>\n<p>Database Configuration Assistant when a database is created<\/p>\n<p>Finished running generic part of root.sh script.<\/p>\n<p>Now product-specific root actions will be performed.<\/p>\n<p>2013-02-19 12:35:15: Parsing the host name<\/p>\n<p>2013-02-19 12:35:15: Checking for super user privileges<\/p>\n<p>2013-02-19 12:35:15: User has super user privileges<\/p>\n<p>Using configuration parameter file: \/app\/11.2.0\/grid\/crs\/install\/crsconfig_params<\/p>\n<p>Creating trace directory<\/p>\n<p>LOCAL ADD MODE<\/p>\n<p>Creating OCR keys for user &#8216;root&#8217;, privgrp &#8216;root&#8217;..<\/p>\n<p>Operation successful.<\/p>\n<p>root wallet<\/p>\n<p>root wallet cert<\/p>\n<p>root cert export<\/p>\n<p>peer wallet<\/p>\n<p>profile reader wallet<\/p>\n<p>pa wallet<\/p>\n<p>peer wallet keys<\/p>\n<p>pa wallet keys<\/p>\n<p>peer cert request<\/p>\n<p>pa cert request<\/p>\n<p>peer cert<\/p>\n<p>pa cert<\/p>\n<p>peer root cert TP<\/p>\n<p>profile reader root cert TP<\/p>\n<p>pa root cert TP<\/p>\n<p>peer pa cert TP<\/p>\n<p>pa peer cert TP<\/p>\n<p>profile reader pa cert TP<\/p>\n<p>profile reader peer cert TP<\/p>\n<p>peer user cert<\/p>\n<p>pa user cert<\/p>\n<p>Adding daemon to inittab<\/p>\n<p>CRS-4123: Oracle High Availability Services has been started.<\/p>\n<p>ohasd is starting<\/p>\n<p>acfsroot: ACFS-9301: ADVM\/ACFS installation can not proceed:<\/p>\n<p>&nbsp;<\/p>\n<p>acfsroot: ACFS-9302: No installation files found at \/app\/11.2.0\/grid\/install\/usm\/EL5\/x86_64\/2.6.18-8\/2.6.18-8.el5uek-x86_64\/bin.<\/p>\n<p>&nbsp;<\/p>\n<p>CRS-2672: Attempting to start &#8216;ora.gipcd&#8217; on &#8216;dbhost1&#8217;<\/p>\n<p>CRS-2672: Attempting to start &#8216;ora.mdnsd&#8217; on &#8216;dbhost1&#8217;<\/p>\n<p>CRS-2676: Start of &#8216;ora.gipcd&#8217; on &#8216;dbhost1&#8217; succeeded<\/p>\n<p>CRS-2676: Start of &#8216;ora.mdnsd&#8217; on &#8216;dbhost1&#8217; succeeded<\/p>\n<p>CRS-2672: Attempting to start &#8216;ora.gpnpd&#8217; on &#8216;dbhost1&#8217;<\/p>\n<p>CRS-2676: Start of &#8216;ora.gpnpd&#8217; on &#8216;dbhost1&#8217; succeeded<\/p>\n<p>CRS-2672: Attempting to start &#8216;ora.cssdmonitor&#8217; on &#8216;dbhost1&#8217;<\/p>\n<p>CRS-2676: Start of &#8216;ora.cssdmonitor&#8217; on &#8216;dbhost1&#8217; succeeded<\/p>\n<p>CRS-2672: Attempting to start &#8216;ora.cssd&#8217; on &#8216;dbhost1&#8217;<\/p>\n<p>CRS-2672: Attempting to start &#8216;ora.diskmon&#8217; on &#8216;dbhost1&#8217;<\/p>\n<p>CRS-2676: Start of &#8216;ora.diskmon&#8217; on &#8216;dbhost1&#8217; succeeded<\/p>\n<p>CRS-2676: Start of &#8216;ora.cssd&#8217; on &#8216;dbhost1&#8217; succeeded<\/p>\n<p>CRS-2672: Attempting to start &#8216;ora.ctssd&#8217; on &#8216;dbhost1&#8217;<\/p>\n<p>CRS-2676: Start of &#8216;ora.ctssd&#8217; on &#8216;dbhost1&#8217; succeeded<\/p>\n<p>clscfg: -install mode specified<\/p>\n<p>Successfully accumulated necessary OCR keys.<\/p>\n<p>Creating OCR keys for user &#8216;root&#8217;, privgrp &#8216;root&#8217;..<\/p>\n<p>Operation successful.<\/p>\n<p>CRS-2672: Attempting to start &#8216;ora.crsd&#8217; on &#8216;dbhost1&#8217;<\/p>\n<p>CRS-2676: Start of &#8216;ora.crsd&#8217; on &#8216;dbhost1&#8217; succeeded<\/p>\n<p>Now formatting voting disk: \/u01\/cluster\/vdsk1.<\/p>\n<p>Now formatting voting disk: \/u02\/cluster\/vdsk2.<\/p>\n<p>Now formatting voting disk: \/u03\/cluster\/vdsk3.<\/p>\n<p>CRS-4603: Successful addition of voting disk \/u01\/cluster\/vdsk1.<\/p>\n<p>CRS-4603: Successful addition of voting disk \/u02\/cluster\/vdsk2.<\/p>\n<p>CRS-4603: Successful addition of voting disk \/u03\/cluster\/vdsk3.<\/p>\n<p>## STATE File Universal Id File Name Disk group<\/p>\n<p>&#8212; &#8212;&#8211; &#8212;&#8212;&#8212;&#8212;&#8212;&#8211; &#8212;&#8212;&#8212; &#8212;&#8212;&#8212;<\/p>\n<p>1. ONLINE 91ce08c1a7254ff5bfe2d1125bafd956 (\/u01\/cluster\/vdsk1) []<\/p>\n<p>2. ONLINE 44f4d1a582e54ffdbf600efd4fb30cff (\/u02\/cluster\/vdsk2) []<\/p>\n<p>3. ONLINE 60b10b42b1334f2fbf753c9a4a4e85d2 (\/u03\/cluster\/vdsk3) []<\/p>\n<p>Located 3 voting disk(s).<\/p>\n<p>CRS-2673: Attempting to stop &#8216;ora.crsd&#8217; on &#8216;dbhost1&#8217;<\/p>\n<p>CRS-2677: Stop of &#8216;ora.crsd&#8217; on &#8216;dbhost1&#8217; succeeded<\/p>\n<p>CRS-2673: Attempting to stop &#8216;ora.ctssd&#8217; on &#8216;dbhost1&#8217;<\/p>\n<p>CRS-2677: Stop of &#8216;ora.ctssd&#8217; on &#8216;dbhost1&#8217; succeeded<\/p>\n<p>CRS-2673: Attempting to stop &#8216;ora.cssdmonitor&#8217; on &#8216;dbhost1&#8217;<\/p>\n<p>CRS-2677: Stop of &#8216;ora.cssdmonitor&#8217; on &#8216;dbhost1&#8217; succeeded<\/p>\n<p>CRS-2673: Attempting to stop &#8216;ora.cssd&#8217; on &#8216;dbhost1&#8217;<\/p>\n<p>CRS-2677: Stop of &#8216;ora.cssd&#8217; on &#8216;dbhost1&#8217; succeeded<\/p>\n<p>CRS-2673: Attempting to stop &#8216;ora.gpnpd&#8217; on &#8216;dbhost1&#8217;<\/p>\n<p>CRS-2677: Stop of &#8216;ora.gpnpd&#8217; on &#8216;dbhost1&#8217; succeeded<\/p>\n<p>CRS-2673: Attempting to stop &#8216;ora.gipcd&#8217; on &#8216;dbhost1&#8217;<\/p>\n<p>CRS-2677: Stop of &#8216;ora.gipcd&#8217; on &#8216;dbhost1&#8217; succeeded<\/p>\n<p>CRS-2673: Attempting to stop &#8216;ora.mdnsd&#8217; on &#8216;dbhost1&#8217;<\/p>\n<p>CRS-2677: Stop of &#8216;ora.mdnsd&#8217; on &#8216;dbhost1&#8217; succeeded<\/p>\n<p>CRS-2672: Attempting to start &#8216;ora.mdnsd&#8217; on &#8216;dbhost1&#8217;<\/p>\n<p>CRS-2676: Start of &#8216;ora.mdnsd&#8217; on &#8216;dbhost1&#8217; succeeded<\/p>\n<p>CRS-2672: Attempting to start &#8216;ora.gipcd&#8217; on &#8216;dbhost1&#8217;<\/p>\n<p>CRS-2676: Start of &#8216;ora.gipcd&#8217; on &#8216;dbhost1&#8217; succeeded<\/p>\n<p>CRS-2672: Attempting to start &#8216;ora.gpnpd&#8217; on &#8216;dbhost1&#8217;<\/p>\n<p>CRS-2676: Start of &#8216;ora.gpnpd&#8217; on &#8216;dbhost1&#8217; succeeded<\/p>\n<p>&nbsp;<\/p>\n<p>CRS-2672: Attempting to start &#8216;ora.cssdmonitor&#8217; on &#8216;dbhost1&#8217;<\/p>\n<p>CRS-2676: Start of &#8216;ora.cssdmonitor&#8217; on &#8216;dbhost1&#8217; succeeded<\/p>\n<p>CRS-2672: Attempting to start &#8216;ora.cssd&#8217; on &#8216;dbhost1&#8217;<\/p>\n<p>CRS-2672: Attempting to start &#8216;ora.diskmon&#8217; on &#8216;dbhost1&#8217;<\/p>\n<p>CRS-2676: Start of &#8216;ora.diskmon&#8217; on &#8216;dbhost1&#8217; succeeded<\/p>\n<p>CRS-2676: Start of &#8216;ora.cssd&#8217; on &#8216;dbhost1&#8217; succeeded<\/p>\n<p>CRS-2672: Attempting to start &#8216;ora.ctssd&#8217; on &#8216;dbhost1&#8217;<\/p>\n<p>CRS-2676: Start of &#8216;ora.ctssd&#8217; on &#8216;dbhost1&#8217; succeeded<\/p>\n<p>CRS-2672: Attempting to start &#8216;ora.crsd&#8217; on &#8216;dbhost1&#8217;<\/p>\n<p>CRS-2676: Start of &#8216;ora.crsd&#8217; on &#8216;dbhost1&#8217; succeeded<\/p>\n<p>CRS-2672: Attempting to start &#8216;ora.evmd&#8217; on &#8216;dbhost1&#8217;<\/p>\n<p>CRS-2676: Start of &#8216;ora.evmd&#8217; on &#8216;dbhost1&#8217; succeeded<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>dbhost1 2013\/02\/19 12:42:00 \/app\/11.2.0\/grid\/cdata\/dbhost1\/backup_20130219_124200.olr<\/p>\n<p>Preparing packages for installation&#8230;<\/p>\n<p>cvuqdisk-1.0.7-1<\/p>\n<p>Configure Oracle Grid Infrastructure for a Cluster &#8230; succeeded<\/p>\n<p>Updating inventory properties for clusterware<\/p>\n<p>Starting Oracle Universal Installer&#8230;<\/p>\n<p>&nbsp;<\/p>\n<p>Checking swap space: must be greater than 500 MB. Actual 8178 MB Passed<\/p>\n<p>The inventory pointer is located at \/etc\/oraInst.loc<\/p>\n<p>The inventory is located at \/app\/oraInventory<\/p>\n<p>&#8216;UpdateNodeList&#8217; was successful.<\/p>\n<p>&nbsp;<\/p>\n<p>====<\/p>\n<p>&nbsp;<\/p>\n<p>[root@dbhost2 ~]# <strong>\/app\/11.2.0\/grid\/root.sh<\/strong><\/p>\n<p>Running Oracle 11g root.sh script&#8230;<\/p>\n<p>&nbsp;<\/p>\n<p>The following environment variables are set as:<\/p>\n<p>ORACLE_OWNER= oracle<\/p>\n<p>ORACLE_HOME= \/app\/11.2.0\/grid<\/p>\n<p>&nbsp;<\/p>\n<p>Enter the full pathname of the local bin directory: [\/usr\/local\/bin]:<\/p>\n<p>Copying dbhome to \/usr\/local\/bin &#8230;<\/p>\n<p>Copying oraenv to \/usr\/local\/bin &#8230;<\/p>\n<p>Copying coraenv to \/usr\/local\/bin &#8230;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>Creating \/etc\/oratab file&#8230;<\/p>\n<p>Entries will be added to the \/etc\/oratab file as needed by<\/p>\n<p>Database Configuration Assistant when a database is created<\/p>\n<p>Finished running generic part of root.sh script.<\/p>\n<p>Now product-specific root actions will be performed.<\/p>\n<p>2013-02-19 12:44:08: Parsing the host name<\/p>\n<p>2013-02-19 12:44:08: Checking for super user privileges<\/p>\n<p>2013-02-19 12:44:08: User has super user privileges<\/p>\n<p>Using configuration parameter file: \/app\/11.2.0\/grid\/crs\/install\/crsconfig_params<\/p>\n<p>Creating trace directory<\/p>\n<p>LOCAL ADD MODE<\/p>\n<p>Creating OCR keys for user &#8216;root&#8217;, privgrp &#8216;root&#8217;..<\/p>\n<p>Operation successful.<\/p>\n<p>Adding daemon to inittab<\/p>\n<p>CRS-4123: Oracle High Availability Services has been started.<\/p>\n<p>ohasd is starting<\/p>\n<p>acfsroot: ACFS-9301: ADVM\/ACFS installation can not proceed:<\/p>\n<p>&nbsp;<\/p>\n<p>acfsroot: ACFS-9302: No installation files found at \/app\/11.2.0\/grid\/install\/usm\/EL5\/x86_64\/2.6.18-8\/2.6.18-8.el5uek-x86_64\/bin.<\/p>\n<p>&nbsp;<\/p>\n<p>CRS-4402: The CSS daemon was started in exclusive mode but found an active CSS daemon on node dbhost1, number 1, and is terminating<\/p>\n<p>CRS-2673: Attempting to stop &#8216;ora.cssdmonitor&#8217; on &#8216;dbhost2&#8217;<\/p>\n<p>CRS-2677: Stop of &#8216;ora.cssdmonitor&#8217; on &#8216;dbhost2&#8217; succeeded<\/p>\n<p>An active cluster was found during exclusive startup, restarting to join the cluster<\/p>\n<p>CRS-2672: Attempting to start &#8216;ora.mdnsd&#8217; on &#8216;dbhost2&#8217;<\/p>\n<p>CRS-2676: Start of &#8216;ora.mdnsd&#8217; on &#8216;dbhost2&#8217; succeeded<\/p>\n<p>CRS-2672: Attempting to start &#8216;ora.gipcd&#8217; on &#8216;dbhost2&#8217;<\/p>\n<p>CRS-2676: Start of &#8216;ora.gipcd&#8217; on &#8216;dbhost2&#8217; succeeded<\/p>\n<p>CRS-2672: Attempting to start &#8216;ora.gpnpd&#8217; on &#8216;dbhost2&#8217;<\/p>\n<p>CRS-2676: Start of &#8216;ora.gpnpd&#8217; on &#8216;dbhost2&#8217; succeeded<\/p>\n<p>CRS-2672: Attempting to start &#8216;ora.cssdmonitor&#8217; on &#8216;dbhost2&#8217;<\/p>\n<p>CRS-2676: Start of &#8216;ora.cssdmonitor&#8217; on &#8216;dbhost2&#8217; succeeded<\/p>\n<p>CRS-2672: Attempting to start &#8216;ora.cssd&#8217; on &#8216;dbhost2&#8217;<\/p>\n<p>CRS-2672: Attempting to start &#8216;ora.diskmon&#8217; on &#8216;dbhost2&#8217;<\/p>\n<p>CRS-2676: Start of &#8216;ora.diskmon&#8217; on &#8216;dbhost2&#8217; succeeded<\/p>\n<p>CRS-2676: Start of &#8216;ora.cssd&#8217; on &#8216;dbhost2&#8217; succeeded<\/p>\n<p>CRS-2672: Attempting to start &#8216;ora.ctssd&#8217; on &#8216;dbhost2&#8217;<\/p>\n<p>CRS-2676: Start of &#8216;ora.ctssd&#8217; on &#8216;dbhost2&#8217; succeeded<\/p>\n<p>CRS-2672: Attempting to start &#8216;ora.crsd&#8217; on &#8216;dbhost2&#8217;<\/p>\n<p>CRS-2676: Start of &#8216;ora.crsd&#8217; on &#8216;dbhost2&#8217; succeeded<\/p>\n<p>CRS-2672: Attempting to start &#8216;ora.evmd&#8217; on &#8216;dbhost2&#8217;<\/p>\n<p>CRS-2676: Start of &#8216;ora.evmd&#8217; on &#8216;dbhost2&#8217; succeeded<\/p>\n<p>&nbsp;<\/p>\n<p>dbhost2 2013\/02\/19 12:48:37 \/app\/11.2.0\/grid\/cdata\/dbhost2\/backup_20130219_124837.olr<\/p>\n<p>Preparing packages for installation&#8230;<\/p>\n<p>cvuqdisk-1.0.7-1<\/p>\n<p>Configure Oracle Grid Infrastructure for a Cluster &#8230; succeeded<\/p>\n<p>Updating inventory properties for clusterware<\/p>\n<p>Starting Oracle Universal Installer&#8230;<\/p>\n<p>&nbsp;<\/p>\n<p>Checking swap space: must be greater than 500 MB. Actual 8188 MB Passed<\/p>\n<p>The inventory pointer is located at \/etc\/oraInst.loc<\/p>\n<p>The inventory is located at \/app\/oraInventory<\/p>\n<p>&#8216;UpdateNodeList&#8217; was successful.<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p><img decoding=\"async\" alt=\"\" src=\"http:\/\/www.oratraining.com\/blog\/wp-content\/uploads\/2013\/03\/031813_0817_Installing135.png\" \/><\/p>\n<p>&nbsp;<\/p>\n<p>If you are not using DNS to resolve host names and only using \/etc\/hosts, you might see following failed step at &#8220;Oracle Cluster Verification Utility&#8221;. This is known issue and you can ignore it.<\/p>\n<p>&nbsp;<\/p>\n<p><img decoding=\"async\" alt=\"\" src=\"http:\/\/www.oratraining.com\/blog\/wp-content\/uploads\/2013\/03\/031813_0817_Installing136.png\" \/><\/p>\n<p>&nbsp;<\/p>\n<p><img decoding=\"async\" alt=\"\" src=\"http:\/\/www.oratraining.com\/blog\/wp-content\/uploads\/2013\/03\/031813_0817_Installing137.png\" \/><\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>Click Skip and it will change the status as Ignored. Click <strong>Next<br \/>\n<\/strong><\/p>\n<p>&nbsp;<\/p>\n<p><img decoding=\"async\" alt=\"\" src=\"http:\/\/www.oratraining.com\/blog\/wp-content\/uploads\/2013\/03\/031813_0817_Installing138.png\" \/><\/p>\n<p>&nbsp;<\/p>\n<p>Click <strong>Close<\/strong> to finish the installation.<\/p>\n<p>&nbsp;<\/p>\n<p>Verify that the cluster services are started properly on both nodes.<\/p>\n<p>&nbsp;<\/p>\n<p>[root@dbhost2 ~]# <strong>\/app\/11.2.0\/grid\/bin\/srvctl status nodeapps<\/strong><\/p>\n<p>VIP dbhost1-vip is enabled<\/p>\n<p>VIP dbhost1-vip is running on node: dbhost1<\/p>\n<p>VIP dbhost2-vip is enabled<\/p>\n<p>VIP dbhost2-vip is running on node: dbhost2<\/p>\n<p>Network is enabled<\/p>\n<p>Network is running on node: dbhost1<\/p>\n<p>Network is running on node: dbhost2<\/p>\n<p>GSD is disabled<\/p>\n<p>GSD is not running on node: dbhost1<\/p>\n<p>GSD is not running on node: dbhost2<\/p>\n<p>ONS is enabled<\/p>\n<p>ONS daemon is running on node: dbhost1<\/p>\n<p>ONS daemon is running on node: dbhost2<\/p>\n<p>eONS is enabled<\/p>\n<p>eONS daemon is running on node: dbhost1<\/p>\n<p>eONS daemon is running on node: dbhost2<\/p>\n<p>&nbsp;<\/p>\n<p><strong>Next:<\/strong>\u00a0<b><a title=\"Installing 11g RAC on Linux VM: Install Oracle Database software and create RAC database\" href=\"http:\/\/www.oratraining.com\/blog\/2013\/03\/installing-11g-rac-on-linux-vm-install-oracle-database-software-and-create-rac-database-2\/\">Install Oracle Database software and create RAC database<\/a><\/b><\/p>\n<p>&nbsp;<\/p>\n<p><strong><a title=\"Installing 11g Release 2 Real Application Clusters (11gR2 RAC) on Linux x86-64 Virtual Machine (VM) \u2013 Step by step installation guide\" href=\"http:\/\/www.oratraining.com\/blog\/2013\/03\/installing-11g-release-2-real-application-clusters-11gr2-rac-on-linux-x86-64-virtual-machine-vm-step-by-step-installation-guide\/\">Installing 11g Release 2 Real Application Clusters (11gR2 RAC) on Linux x86-64 Virtual Machine (VM) \u2013 Steps<\/a><\/strong><\/p>\n<p>1.<strong>\u00a0<a title=\"Creating Oracle VirtualBox Virtual Machine with Oracle Linux 5\" href=\"http:\/\/www.oratraining.com\/blog\/2012\/02\/creating-oracle-virtualbox-virtual-machine-with-oracle-linux\/\">Create Virtual Machine and install 64 bit Linux<\/a>\u00a0(generic step from previous post, not specific to this guide)<\/strong><\/p>\n<p>2.\u00a0<a title=\"Add additional virtual Ethernet card and perform prerequisites in Linux\" href=\"http:\/\/www.oratraining.com\/blog\/2013\/03\/add-additional-virtual-ethernet-card-and-perform-prerequisites-in-linux\/\"><strong>Add additional virtual Ethernet card and perform prerequisites in Linux<\/strong><\/a><\/p>\n<p>3.\u00a0<a title=\"Copy\/clone this virtual machine to create second node and modify host details\" href=\"http:\/\/www.oratraining.com\/blog\/2013\/03\/copyclone-this-virtual-machine-to-create-second-node-and-modify-host-details\/\"><strong>Copy\/clone this virtual machine to create second node and modify host details<\/strong><\/a><\/p>\n<p>4.\u00a0<a title=\"Setup shared file system and other pre-requisites\" href=\"http:\/\/www.oratraining.com\/blog\/2013\/03\/setup-shared-file-system-and-other-pre-requisites\/\"><strong>Setup shared file system and other pre-requisites<\/strong><\/a><\/p>\n<p>5.\u00a0<a title=\"Installing 11g RAC on Linux VM: Install Oracle Clusterware\" href=\"http:\/\/www.oratraining.com\/blog\/2013\/03\/installing-11g-rac-on-linux-vm-install-oracle-clusterware\/\"><strong>Install Oracle Clusterware<\/strong><\/a><\/p>\n<p>6.\u00a0<b><a title=\"Installing 11g RAC on Linux VM: Install Oracle Database software and create RAC database\" href=\"http:\/\/www.oratraining.com\/blog\/2013\/03\/installing-11g-rac-on-linux-vm-install-oracle-database-software-and-create-rac-database-2\/\">Install Oracle Database software and create RAC database<\/a><\/b><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Previous:\u00a0Copy\/clone this virtual machine to create second node and modify host details The clusterware or database installation needs to be started only on one node since it will propagate the files to remote node automatically during the installation. If you have the setup files on your host machine, you can share the setup folder to [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":""},"categories":[1],"tags":[],"class_list":["post-2064","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/www.oratraining.com\/blog\/wp-json\/wp\/v2\/posts\/2064","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.oratraining.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.oratraining.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.oratraining.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.oratraining.com\/blog\/wp-json\/wp\/v2\/comments?post=2064"}],"version-history":[{"count":0,"href":"https:\/\/www.oratraining.com\/blog\/wp-json\/wp\/v2\/posts\/2064\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.oratraining.com\/blog\/wp-json\/wp\/v2\/media?parent=2064"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.oratraining.com\/blog\/wp-json\/wp\/v2\/categories?post=2064"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.oratraining.com\/blog\/wp-json\/wp\/v2\/tags?post=2064"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}