Lunar的oracle实验室 - 世间所有相遇都是久别重逢

linux 误删除文件恢复

发表于 2013 年 6 月 3 日由 Lunar

[oracle@lunar ~]$ sqlplus / as sysdba

SQL*Plus: Release 11.2.0.3.0 Production on Fri Mar 15 23:46:35 2013

Copyright (c) 1982, 2011, Oracle.  All rights reserved.


Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options

SQL> select tablespace_name from dba_tablespaces;

TABLESPACE_NAME
------------------------------------------------------------
SYSTEM
SYSAUX
UNDOTBS1					
TEMP
USERS
EZIO

6 rows selected.

创建测试表空间

SQL> create tablespace lunar_test datafile '/test/lunar_test01.dbf' size 10M;

Tablespace created.

SQL> alter user lunar default tablespace lunar_test;

User altered.

创建表插入数据

SQL> conn lunar/lunar
Connected.
SQL> create table test_table (name varchar2(300)) tablespace lunar_test;

Table created.

SQL> insert into test_table values ('Love you forever, my baby, Happy Childrens Day!');

1 row created.

SQL> commit;

Commit complete.

SQL> 
SQL> select * from test_table;

NAME
--------------------------------------------------------------------------------
Love you forever, my baby, Happy Childrens Day!

删除datafile

SQL> !rm -rf /test/lunar_test01.dbf 

SQL> !ls -lrt /test/lunar_test01.dbf 
ls: /test/lunar_test01.dbf: No such file or directory

SQL>

数据还在，因为从buffer cache中读到的

SQL> conn lunar/lunar
Connected.
SQL> select * from test_table;

NAME
--------------------------------------------------------------------------------
Love you forever, my baby, Happy Childrens Day!

执行flush buffer cache

SQL> conn / as sysdba
Connected.
SQL> alter system flush buffer_cache;

System altered.

可以看见，再次查询，报错文件状态不对了（找不到了）

SQL> conn lunar/lunar
Connected.
SQL> select * from test_table;
select * from test_table
*
ERROR at line 1:
ORA-01116: error in opening database file 6
ORA-01110: data file 6: '/test/lunar_test01.dbf'
ORA-27041: unable to open file
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3

检查dbwr进程的spid

SQL> !ps -ef|grep dbw
oracle    2757     1  0 12:29 ?        00:00:14 ora_dbw0_bb
oracle    8912  8872  0 23:50 pts/3    00:00:00 /bin/bash -c ps -ef|grep dbw
oracle    8914  8912  0 23:50 pts/3    00:00:00 grep dbw

找到dbwr的句柄

SQL> !/usr/sbin/lsof -p 2757
COMMAND  PID   USER   FD   TYPE DEVICE   SIZE/OFF    NODE NAME
oracle  2757 oracle  cwd    DIR  253,0       4096 6947181 /u01/app/oracle/product/11.2.0.3/dbhome_1/dbs
oracle  2757 oracle  rtd    DIR  253,0       4096       2 /
oracle  2757 oracle  txt    REG  253,0  232399473 1179781 /u01/app/oracle/product/11.2.0.3/dbhome_1/bin/oracle
oracle  2757 oracle  DEL    REG    0,4            1933317 /SYSV00000000
oracle  2757 oracle  DEL    REG    0,4            1966086 /SYSV00000000
oracle  2757 oracle  DEL    REG    0,4            1998855 /SYSV2b7268c4
oracle  2757 oracle  mem    REG  253,0     144776 2293762 /lib64/ld-2.5.so
oracle  2757 oracle  mem    REG  253,0       5328 4822547 /usr/lib64/libaio.so.1.0.1
oracle  2757 oracle  mem    REG  253,0    1722328 2293764 /lib64/libc-2.5.so
oracle  2757 oracle  mem    REG  253,0      23360 2293771 /lib64/libdl-2.5.so
oracle  2757 oracle  mem    REG  253,0     615136 2293809 /lib64/libm-2.5.so
oracle  2757 oracle  mem    REG  253,0     145872 2293775 /lib64/libpthread-2.5.so
oracle  2757 oracle  mem    REG  253,0      53448 2293781 /lib64/librt-2.5.so
oracle  2757 oracle  mem    REG  253,0     114352 2293986 /lib64/libnsl-2.5.so
oracle  2757 oracle  mem    REG  253,0      58949 6955646 /u01/app/oracle/product/11.2.0.3/dbhome_1/lib/libnque11.so
oracle  2757 oracle  mem    CHR    1,5               4302 /dev/zero
oracle  2757 oracle  mem    REG  253,0       1544 6951342 /u01/app/oracle/product/11.2.0.3/dbhome_1/dbs/hc_bb.dat
oracle  2757 oracle  mem    REG  253,0      53880 2293787 /lib64/libnss_files-2.5.so
oracle  2757 oracle  mem    REG  253,0      22368 4831836 /usr/lib64/libnuma.so.1
oracle  2757 oracle  mem    REG  253,0     153574 6976596 /u01/app/oracle/product/11.2.0.3/dbhome_1/lib/libocrutl11.so
oracle  2757 oracle  mem    REG  253,0    3319072 6976595 /u01/app/oracle/product/11.2.0.3/dbhome_1/lib/libocrb11.so
oracle  2757 oracle  mem    REG  253,0    1590995 6976594 /u01/app/oracle/product/11.2.0.3/dbhome_1/lib/libocr11.so
oracle  2757 oracle  mem    REG  253,0      12755 6953392 /u01/app/oracle/product/11.2.0.3/dbhome_1/lib/libskgxn2.so
oracle  2757 oracle  mem    REG  253,0   17319952 6976562 /u01/app/oracle/product/11.2.0.3/dbhome_1/lib/libhasgen11.so
oracle  2757 oracle  mem    REG  253,0     161764 6953093 /u01/app/oracle/product/11.2.0.3/dbhome_1/lib/libdbcfg11.so
oracle  2757 oracle  mem    REG  253,0     228765 6976564 /u01/app/oracle/product/11.2.0.3/dbhome_1/lib/libclsra11.so
oracle  2757 oracle  mem    REG  253,0    7955322 6976828 /u01/app/oracle/product/11.2.0.3/dbhome_1/lib/libnnz11.so
oracle  2757 oracle  mem    REG  253,0    1010297 4456454 /u01/app/oracle/product/11.2.0.3/dbhome_1/lib/libskgxp11.so
oracle  2757 oracle  mem    REG  253,0     589359 6977008 /u01/app/oracle/product/11.2.0.3/dbhome_1/lib/libcell11.so
oracle  2757 oracle  mem    REG  253,0      12259 6950100 /u01/app/oracle/product/11.2.0.3/dbhome_1/lib/libodmd11.so
oracle  2757 oracle    0r   CHR    1,3        0t0    4300 /dev/null
oracle  2757 oracle    1w   CHR    1,3        0t0    4300 /dev/null
oracle  2757 oracle    2w   CHR    1,3        0t0    4300 /dev/null
oracle  2757 oracle    3r   CHR    1,3        0t0    4300 /dev/null
oracle  2757 oracle    4r   CHR    1,3        0t0    4300 /dev/null
oracle  2757 oracle    5r   CHR    1,3        0t0    4300 /dev/null
oracle  2757 oracle    6r   CHR    1,3        0t0    4300 /dev/null
oracle  2757 oracle    7u   REG  253,0       1544 6951342 /u01/app/oracle/product/11.2.0.3/dbhome_1/dbs/hc_bb.dat
oracle  2757 oracle    8r   CHR    1,3        0t0    4300 /dev/null
oracle  2757 oracle    9r   CHR    1,3        0t0    4300 /dev/null
oracle  2757 oracle   10r   CHR    1,5        0t0    4302 /dev/zero
oracle  2757 oracle   11r   CHR    1,5        0t0    4302 /dev/zero
oracle  2757 oracle   12u   REG  253,0       1544 6951342 /u01/app/oracle/product/11.2.0.3/dbhome_1/dbs/hc_bb.dat
oracle  2757 oracle   13r   REG  253,0    1101312 6962758 /u01/app/oracle/product/11.2.0.3/dbhome_1/rdbms/mesg/oraus.msb
oracle  2757 oracle   14r   DIR    0,3          0   10741 /proc/2757/fd
oracle  2757 oracle   15r   CHR    1,5        0t0    4302 /dev/zero
oracle  2757 oracle   16u   REG  253,0       1544 6951342 /u01/app/oracle/product/11.2.0.3/dbhome_1/dbs/hc_bb.dat
oracle  2757 oracle   17uR  REG  253,0         24 6951341 /u01/app/oracle/product/11.2.0.3/dbhome_1/dbs/lkBB
oracle  2757 oracle   18r   REG  253,0    1101312 6962758 /u01/app/oracle/product/11.2.0.3/dbhome_1/rdbms/mesg/oraus.msb
oracle  2757 oracle   19u  IPv6  42951        0t0     UDP *:29281 
oracle  2757 oracle  256u   REG  253,0   10043392 4784156 /u01/app/oracle/oradata/bb/control01.ctl
oracle  2757 oracle  257uW  REG  253,0  524296192 4784160 /u01/app/oracle/oradata/bb/system01.dbf
oracle  2757 oracle  258uW  REG  253,0 1048584192 4784161 /u01/app/oracle/oradata/bb/sysaux01.dbf
oracle  2757 oracle  259u   REG  253,0  581967872 4784162 /u01/app/oracle/oradata/bb/undotbs01.dbf
oracle  2757 oracle  260u   REG  253,0   85204992 4784165 /u01/app/oracle/oradata/bb/users01.dbf
oracle  2757 oracle  261uW  REG  253,0  209723392 4784784 /u01/app/oracle/oradata/bb/ezio01.dbf
oracle  2757 oracle  262uW  REG  253,0   87040000 4784725 /u01/app/oracle/oradata/bb/temp01.dbf
oracle  2757 oracle  263uW  REG  253,0   10493952 1639050 /test/lunar_test01.dbf (deleted)   可以看见文件被标识为delete

进入dbwr进程的File Descriptor number目录中

SQL> !ls -ltar /proc/2757/fd/
total 0
dr-xr-xr-x 7 oracle oinstall  0 Mar 15 12:29 ..
dr-x------ 2 oracle oinstall  0 Mar 15 12:29 .
lr-x------ 1 oracle oinstall 64 Mar 15 23:30 9 -> /dev/null
lr-x------ 1 oracle oinstall 64 Mar 15 23:30 8 -> /dev/null
lrwx------ 1 oracle oinstall 64 Mar 15 23:30 7 -> /u01/app/oracle/product/11.2.0.3/dbhome_1/dbs/hc_bb.dat
lr-x------ 1 oracle oinstall 64 Mar 15 23:30 6 -> /dev/null
lr-x------ 1 oracle oinstall 64 Mar 15 23:30 5 -> /dev/null
lr-x------ 1 oracle oinstall 64 Mar 15 23:30 4 -> /dev/null
lr-x------ 1 oracle oinstall 64 Mar 15 23:30 3 -> /dev/null
lrwx------ 1 oracle oinstall 64 Mar 15 23:30 263 -> /test/lunar_test01.dbf (deleted)   可以看见dbwr的进程的FD号
lrwx------ 1 oracle oinstall 64 Mar 15 23:30 262 -> /u01/app/oracle/oradata/bb/temp01.dbf
lrwx------ 1 oracle oinstall 64 Mar 15 23:30 261 -> /u01/app/oracle/oradata/bb/ezio01.dbf
lrwx------ 1 oracle oinstall 64 Mar 15 23:30 260 -> /u01/app/oracle/oradata/bb/users01.dbf
lrwx------ 1 oracle oinstall 64 Mar 15 23:30 259 -> /u01/app/oracle/oradata/bb/undotbs01.dbf
lrwx------ 1 oracle oinstall 64 Mar 15 23:30 258 -> /u01/app/oracle/oradata/bb/sysaux01.dbf
lrwx------ 1 oracle oinstall 64 Mar 15 23:30 257 -> /u01/app/oracle/oradata/bb/system01.dbf
lrwx------ 1 oracle oinstall 64 Mar 15 23:30 256 -> /u01/app/oracle/oradata/bb/control01.ctl
l-wx------ 1 oracle oinstall 64 Mar 15 23:30 2 -> /dev/null
lr-x------ 1 oracle oinstall 64 Mar 15 23:30 18 -> /u01/app/oracle/product/11.2.0.3/dbhome_1/rdbms/mesg/oraus.msb
lrwx------ 1 oracle oinstall 64 Mar 15 23:30 17 -> /u01/app/oracle/product/11.2.0.3/dbhome_1/dbs/lkBB
lrwx------ 1 oracle oinstall 64 Mar 15 23:30 16 -> /u01/app/oracle/product/11.2.0.3/dbhome_1/dbs/hc_bb.dat
lr-x------ 1 oracle oinstall 64 Mar 15 23:30 15 -> /dev/zero
lr-x------ 1 oracle oinstall 64 Mar 15 23:30 14 -> /proc/2757/fd
lr-x------ 1 oracle oinstall 64 Mar 15 23:30 13 -> /u01/app/oracle/product/11.2.0.3/dbhome_1/rdbms/mesg/oraus.msb
lrwx------ 1 oracle oinstall 64 Mar 15 23:30 12 -> /u01/app/oracle/product/11.2.0.3/dbhome_1/dbs/hc_bb.dat
lr-x------ 1 oracle oinstall 64 Mar 15 23:30 11 -> /dev/zero
lr-x------ 1 oracle oinstall 64 Mar 15 23:30 10 -> /dev/zero
l-wx------ 1 oracle oinstall 64 Mar 15 23:30 1 -> /dev/null
lr-x------ 1 oracle oinstall 64 Mar 15 23:30 0 -> /dev/null
lrwx------ 1 oracle oinstall 64 Mar 15 23:37 19 -> socket:[42951]

恢复过程

[oracle@lunar ~]$ cd /proc/2757/fd/
--根据File Descriptor number重建数据文件：
[oracle@lunar fd]$ cat 263 > /test/lunar01.dbf

检查下，文件已经恢复完成，大小为10m

[oracle@lunar fd]$ ls -lrt /test/lunar01.dbf
-rw-r--r-- 1 oracle oinstall 10493952 Mar 15 23:50 /test/lunar01.dbf

将数据文件offline

SQL> alter database datafile '/test/lunar_test01.dbf' offline;

Database altered.
[/sql
修改控制文件的文件信息，因为已经将rm的/test/lunar_test01.dbf，创建到/test/lunar01.dbf，因此要同步控制文件中的信息
1
SQL> alter database rename file '/test/lunar_test01.dbf' to '/test/lunar01.dbf';

Database altered.

恢复datafile

SQL> recover datafile '/test/lunar01.dbf';
Media recovery complete.

将数据文件online

SQL> alter database datafile '/test/lunar01.dbf' online;

Database altered.

好了，完成恢复了

SQL> conn lunar/lunar
Connected.
SQL> select * from test_table;

NAME
--------------------------------------------------------------------------------
Love you forever, my baby, Happy Childrens Day!

SQL> 
SQL> select tablespace_name from dba_tablespaces;

TABLESPACE_NAME
------------------------------------------------------------
SYSTEM
SYSAUX
UNDOTBS1
TEMP
USERS
EZIO
LUNAR_TEST

7 rows selected.

SQL> select name from v$datafile;

NAME
--------------------------------------------------------------------------------
/u01/app/oracle/oradata/bb/system01.dbf
/u01/app/oracle/oradata/bb/sysaux01.dbf
/u01/app/oracle/oradata/bb/undotbs01.dbf
/u01/app/oracle/oradata/bb/users01.dbf
/u01/app/oracle/oradata/bb/ezio01.dbf
/test/lunar01.dbf

6 rows selected.

发表在 backup&recovery | 标签为 linux, 恢复, 误删除数据 | 2 条评论

Smart Flash Cache on Exadata(4)—使用flash cache

发表于 2013 年 5 月 30 日由 Lunar

对于表和索引，可以在创建表时使用storage子句将表保存在flashcache中，如果表已经创建完成了，那么可以使用alert table或者alert index命令进行修改相应对象的storage属性，将对象混存在flash cache中。

我做了两张测试表，他们数据的内容基本一直，不过一张是压缩表，一张是非压缩表：

11:14:16 SQL> set autot off
11:16:40 SQL> col segment_name for a30
11:16:41 SQL> select /*+ parallel(a,24) */ SEGMENT_NAME,SUM(BYTES)/1024/1024/1024 GB 
11:16:41   2  FROM USER_EXTENTS 
11:16:41   3  WHERE SEGMENT_NAME LIKE '%COMPRESS%'
11:16:41   4  GROUP BY SEGMENT_NAME;

SEGMENT_NAME                           GB
------------------------------ ----------
LUNAR_UNCOMPRESS                  17.234436
LUNAR_COMPRESS                   1.27032471
	
11:31:22 SQL> select count(*) from LUNAR_UNCOMPRESS;

  COUNT(*)
----------
 155607456

Elapsed: 00:00:00.54
11:31:24 SQL> select count(*) from LUNAR_COMPRESS;

  COUNT(*)
----------
 155680188

Elapsed: 00:00:00.18
11:31:30 SQL>

SQL> select table_name, compression, compress_for FROM user_tables where table_name like '%COMPRESS%';

TABLE_NAME                     COMPRESS COMPRESS_FOR
------------------------------ -------- ------------
LUNAR_UNCOMPRESS                 DISABLED
LUNAR_COMPRESS                   ENABLED  QUERY HIGH

Elapsed: 00:00:00.07
SQL>

缓存对象到flash cache的语法如下（类似这样对象的IO，我们都称之为 ”Smart Scan I/Os” ）：

--对于分区对象，可以缓存部分或者全部分表数据：
CREATE TABLE lunar_par_tab (c1 number, c2 clob) 
TABLESPACE TBS_1 PARTITION BY RANGE(c1)
 ( PARTITION p1 VALUES LESS THAN (100) TABLESPACE TBS_2 STORAGE (CELL_FLASH_CACHE DEFAULT),
	PARTITION p2 VALUES LESS THAN (200) TABLESPACE TBS_3 STORAGE (CELL_FLASH_CACHE KEEP));

--对于lob对象，可以根据应用访问等具体情况确定是否需要缓存：
CREATE TABLE lunar_lob_test (col1 number, col_lob clob)
lob (col_lob) STORE AS securefile (cache nologging STORAGE (CELL_FLASH_CACHE NONE))
PCTFREE 0 
TABLESPACE tbs_93 STORAGE (initial 128K next 128K pctincrease 0);

取消对象在flash cache的缓存：

alter TABLE LUNAR_TEST_PAR STORAGE (CELL_FLASH_CACHE none);
alter INDEX IDX_LUNAR_TEST_PAR STORAGE (CELL_FLASH_CACHE none);

一般在POC或者生产上，我们会按照一定的条件（比如过滤掉超大的表或者分区等等）生成符合条件的表或者索引的keep 命令：
例如，使用下面的语句，将生成满足条件的表的缓存语句：

select 'alter table '||segment_name||' STORAGE (CELL_FLASH_CACHE keep);'
from dba_segments
where owner='LUNAR' 
  and segment_type in ('TABLE')
--	and TABLESPACE_NAME='DATA_DYNAMIC'
and segment_name not in ('TEST_TAB1','TEST_TAB2')
--group by owner||'.'||segment_name,segment_type
--having sum(bytes) > 1*1024*1024*1024
--order by sum(bytes)/1024/1024/1024;

alter table MM_LUNAR_TEMP_TEST STORAGE (CELL_FLASH_CACHE keep);
alter table MM_LUNAR_TEMP_TEST STORAGE (CELL_FLASH_CACHE keep);
…………

select 'alter INDEX '||segment_name||' STORAGE (CELL_FLASH_CACHE keep);'
from dba_segments
where owner='LUNAR' 
  and segment_type in ('INDEX')
--	and TABLESPACE_NAME='DATA_DYNAMIC'
--and segment_name not in ('TEST_TAB1','TEST_TAB2')
--group by owner||'.'||segment_name,segment_type
--having sum(bytes) > 1*1024*1024*1024
--order by sum(bytes)/1024/1024/1024;

alter INDEX IDX_LUNAR_TEST_PAR STORAGE (CELL_FLASH_CACHE keep);
alter INDEX IDX_LUNAR_TEST STORAGE (CELL_FLASH_CACHE keep);
…………

对于已经创建的对象，要修改其CELL_FLASH_CACHE属性，可以使用如下命令：

ALTER TABLE lunar_lob_test STORAGE( CELL_FLASH_CACHE DEFAULT);
ALTER TABLE lunar_lob_test MODIFY LOB (col_lob) (STORAGE (CELL_FLASH_CACHE KEEP));

要知道当前对象的设置，可以查询dba_tables(all_tables, user_tables)或者dba_indexes(all_indexes, user_indexes)的CELL_FLASH_CACHE列：

SELECT TABLESPACE_NAME, TABLE_NAME, CELL_FLASH_CACHE 
FROM user_tables 
WHERE table_name='LUNAR_LOB_TEST';

SELECT CELL_FLASH_CACHE 
FROM ALL_INDEXES 
WHERE index_name='IDX_LUNAR_LOB_TEST';

还可以在cell上使用cellcli工具和命令” LIST FLASHCACHECONTENT”查看：

--(注意，cell的版本是11.2.3.2.1，是上周刚刚升级上来的，o(∩_∩)o 哈哈)
[root@dm02cel09 ~]# cellcli
CellCLI: Release 11.2.3.2.1 - Production on Tue May 28 11:24:29 CST 2013

Copyright (c) 2007, 2012, Oracle.  All rights reserved.
Cell Efficiency Ratio: 416

CellCLI>
CellCLI> LIST FLASHCACHECONTENT WHERE objectNumber= 112912  DETAIL
--这说明该OBJECT_ID=112912对象没有被keep在flash cache上。

CellCLI> LIST FLASHCACHECONTENT WHERE objectNumber= 112913  DETAIL
         cachedKeepSize:         0
         cachedSize:             34340864
         dbID:                   722038086
         dbUniqueName:           DB06
         hitCount:               652
         missCount:              19377
         objectNumber:           112913
         tableSpaceNumber:       7

CellCLI>

我们注意到这里cachedKeepSize=0，表示这个表曾经被cache了，后来执行了类似“alter table XXX STORAGE (CELL_FLASH_CACHE none);”的命令，取消的cache到flash cache的操作。

而上面的objectNumber= 112912没有任何输出，表示这个表没有被缓存过。

通常，POC或者生产上，我们更多的是生成批量查看对象缓存内容的语句：

11:23:17 SQL> select 'LIST FLASHCACHECONTENT WHERE objectNumber= '||b.object_id||'  DETAIL'
11:23:18   2  from dba_segments a, dba_objects b
11:23:18   3  where a.owner='LUNAR'
11:23:18   4    and a.segment_name = b.OBJECT_NAME  
11:23:18   5    and a.segment_type in ('TABLE')
11:23:18   6    and a.segment_name in ('LUNAR_UNCOMPRESS','LUNAR_COMPRESS')
11:23:18   7  /

'LISTFLASHCACHECONTENTWHEREOBJECTNUMBER='||B.OBJECT_ID||'DETAIL'
-------------------------------------------------------------------------------------------
LIST FLASHCACHECONTENT WHERE objectNumber= 112912  DETAIL
LIST FLASHCACHECONTENT WHERE objectNumber= 112913  DETAIL

具体我们看一下” LIST FLASHCACHE DETAIL”和”“ LIST FLASHCACHECONTENT”的官方说明：

CellCLI> DESCRIBE FLASHCACHE
        name
        cellDisk
        creationTime
        degradedCelldisks
        effectiveCacheSize
        id
        size
        status

CellCLI>

具体含义如下：

例如：

CellCLI> LIST FLASHCACHE DETAIL
         name:                   dm02cel09_FLASHCACHE
         cellDisk:               						FD_08_dm02cel09,FD_01_dm02cel09,FD_06_dm02cel09,FD_11_dm02cel09,
		FD_00_dm02cel09,FD_12_dm02cel09,FD_05_dm02cel09,FD_07_dm02cel09,
		FD_15_dm02cel09,FD_04_dm02cel09,FD_02_dm02cel09,FD_09_dm02cel09,
		FD_13_dm02cel09,FD_10_dm02cel09,FD_14_dm02cel09,FD_03_dm02cel09
         creationTime:           2012-08-10T16:03:25+08:00
         degradedCelldisks:      需要关注这里是否有异常信息…………
         effectiveCacheSize:     364.75G
         id:                     26514ed2-fa29-4b05-91b7-835f6180d6f3
         size:                   364.75G
         status:                 normal

CellCLI>

查看Flash cache的内容：

CellCLI> DESCRIBE FLASHCACHECONTENT
        cachedKeepSize
        cachedSize
        dbID
        dbUniqueName
        hitCount
        hoursToExpiration
        missCount
        objectNumber
        tableSpaceNumber

CellCLI>

例如：

CellCLI> LIST FLASHCACHECONTENT WHERE objectNumber= 112913  DETAIL
         cachedKeepSize:         1315151872   被cache的对象大小为1.3G
         cachedSize:             1315258368   该对象被用keep模式缓存的大小
	dbID:                   722038086
         dbUniqueName:           DB06
         hitCount:               4541	
         hoursToExpiration:      24
         missCount:              20676       
         objectNumber:           112913
         tableSpaceNumber:       7

CellCLI> 

CellCLI> LIST FLASHCACHECONTENT WHERE objectNumber= 112912  DETAIL
         cachedKeepSize:         95485952
         cachedSize:             95485952
         dbID:                   722038086
         dbUniqueName:           DB06
         hitCount:               245
         hoursToExpiration:      24
         missCount:              545
         objectNumber:           112912
         tableSpaceNumber:       7

CellCLI>

当使用” STORAGE (CELL_FLASH_CACHE none)”子句取消对象在flash cache的缓存时：

SQL> ALTER table LUNAR_UNCOMPRESS STORAGE (CELL_FLASH_CACHE none);

Table altered.

Elapsed: 00:00:00.01
SQL> ALTER table LUNAR_COMPRESS STORAGE (CELL_FLASH_CACHE none);

Table altered.

Elapsed: 00:00:00.01
SQL>

可以看到cachedKeepSize的值立刻变成“0”：

CellCLI> LIST FLASHCACHECONTENT WHERE objectNumber= 112912  DETAIL
         cachedKeepSize:         0
         cachedSize:             95485952
         dbID:                   722038086
         dbUniqueName:           DB06
         hitCount:               245
         missCount:              545
         objectNumber:           112912
         tableSpaceNumber:       7

CellCLI> LIST FLASHCACHECONTENT WHERE objectNumber= 112913  DETAIL
         cachedKeepSize:         0
         cachedSize:             1315258368
         dbID:                   722038086
         dbUniqueName:           DB06
         hitCount:               4541
         missCount:              20676
         objectNumber:           112913
         tableSpaceNumber:       7

CellCLI>

看下这两个表的数据量，都在1.5亿行左右，没有索引情况下，count(*)的时间如下：
（当然，我这里测试不够严谨，应该每次执行前先执行清空buffer pool等）

SQL> select count(*) from LUNAR_COMPRESS;

  COUNT(*)
----------
 155680188

Elapsed: 00:00:23.67
SQL> 
SQL> select count(*) from LUNAR_UNCOMPRESS;

  COUNT(*)
----------
 155607456

Elapsed: 00:00:01.34
SQL>

修改为keep，再查看下：

SQL> ALTER table LUNAR_UNCOMPRESS STORAGE (CELL_FLASH_CACHE keep);
Table altered.
Elapsed: 00:00:00.07
SQL> ALTER table LUNAR_COMPRESS STORAGE (CELL_FLASH_CACHE keep);
Table altered.
Elapsed: 00:00:00.01
SQL> 

--因为还没有访问，因此，这里cachedKeepSize依然是0：
CellCLI> LIST FLASHCACHECONTENT WHERE objectNumber= 112912  DETAIL
         cachedKeepSize:         0
         cachedSize:             95485952
         dbID:                   722038086
         dbUniqueName:           DB06
         hitCount:               245
         missCount:              1174
         objectNumber:           112912
         tableSpaceNumber:       7

CellCLI> LIST FLASHCACHECONTENT WHERE objectNumber= 112913  DETAIL
         cachedKeepSize:         0
         cachedSize:             1315258368
         dbID:                   722038086
         dbUniqueName:           DB06
         hitCount:               4541
         missCount:              24459
         objectNumber:           112913
         tableSpaceNumber:       7

CellCLI>

修改下会话的parallel选项（不指定具体的并行度，系统将使用缺省并行度），准备插入1000w行数据，测试下速度：

SQL> alter session force parallel query;

Session altered.

Elapsed: 00:00:00.00
SQL> alter session force parallel dml;

Session altered.

Elapsed: 00:00:00.00
SQL> alter session force parallel ddl;

Session altered.

Elapsed: 00:00:00.00
SQL>

插入1000w行数据，看下时间：

SQL> insert /*+ append parallel */ into LUNAR_COMPRESS 
  2  select /*+ parallel  */ * from LUNAR_COMPRESS  
where rownum<=10000000;
  3  

10000000 rows created.

Elapsed: 00:00:31.70
SQL> SQL> 
SQL> commit;

Commit complete.

Elapsed: 00:00:00.06
SQL> 
SQL> insert /*+ append parallel */ into LUNAR_UNCOMPRESS 
  2  select /*+ parallel  */ * from LUNAR_UNCOMPRESS  
  3  where rownum<=10000000;

10000000 rows created.

Elapsed: 00:00:27.84
SQL> 
SQL> commit;

Commit complete.

Elapsed: 00:00:00.08
SQL>

查看下对象大小：

SQL> col segment_name for a30                                                
SQL> select SEGMENT_NAME,SUM(BYTES)/1024/1024/1024 GB  
  2  FROM USER_EXTENTS                                                       
  3  WHERE sEGMENT_nAME LIKE '%COMPRESS%'                                    
  4  GROUP BY SEGMENT_NAME;                                                  

SEGMENT_NAME                           GB
------------------------------ ----------
LUNAR_COMPRESS                   1.45855713
LUNAR_UNCOMPRESS                 19.4857178

Elapsed: 00:00:19.21
SQL>

可以查看下现在对象的cache情况：
可见，这时对象已经

CellCLI> LIST FLASHCACHECONTENT WHERE objectNumber= 112912  DETAIL
         cachedKeepSize:         103546880
         cachedSize:             103546880
         dbID:                   722038086
         dbUniqueName:           DB06
         hitCount:               553
         hoursToExpiration:      24
         missCount:              1226
         objectNumber:           112912
         tableSpaceNumber:       7

CellCLI> LIST FLASHCACHECONTENT WHERE objectNumber= 112913  DETAIL
         cachedKeepSize:         1495015424
         cachedSize:             1496629248
         dbID:                   722038086
         dbUniqueName:           DB06
         hitCount:               6234
         hoursToExpiration:      24
         missCount:              25126
         objectNumber:           112913
         tableSpaceNumber:       7

CellCLI> 

SQL> select count(*) from LUNAR_COMPRESS;

  COUNT(*)
----------
 175680188

Elapsed: 00:00:00.27
SQL> select count(*) from LUNAR_UNCOMPRESS;

  COUNT(*)
----------
 175607456

Elapsed: 00:00:00.62
SQL>

Direct path insert缺省是不会缓存到Flash cache的，因此WriteBack和WriteThough模式的性能无差别。
未完，待续……
下一节，看看flash cache的监控和阀值…………

发表在 FAQ, 体系架构, 硬件配置 | 标签为 Exadata, flash cache | 留下评论

Smart Flash Cache on Exadata(3)—Write-back

发表于 2013 年 5 月 29 日由 Lunar

在Exadata image 11.2.3.2.0以前的版本中，仅支持Write Through模式，该模式的读写流程已经在”Smart Flash Cache on Exadata(2)—Write through“种讨论过了。需要注意的是：这个模式的flash cache在Cellsrv重启之后Flash Cache上的数据变成Invalid，而Write-Back模式则不会，这是他们的工作原理决定的……。

从Image 11.2.3.2.0版本开始，原来仅支持Write Through 模式的Flash Cache现在可以支持Write Back模式了。但是Wtite Back模式不是缺省属性（缺省值还是Write through模式），需要手工修改来启用这个特性。Write Through和Write Back这两个模式之间的切换需要重启Cellsrv服务，否则会报错：

CELL-01552: Stop CELLSRV. Cell flashCacheMode cannot be modified while CELLSRV is running.

将Smart Flash Cache修改为Write Back模式的具体方法如下：

1.查看当前FlashCache的工作模式
list cell 命令来查询flashcachemode属性（缺省是WriteThrough模式）：
CellCLI&gt; list cell attributes name, flashcachemode
         dm02cel02        WriteThrough

2.enable Write-back模式

Drop FlashCache
cellcli -e drop flashcache
  
停止cellsrv服务
cellcli -e alter cell shutdown services cellsrv
 
修改flashcachemode
cellcli -e alter cell flashCacheMode=WriteBack
 
启动cellsrv
cellcli -e alter cell startup services cellsrv

创建FlashCache
cellcli -e create flashcache all

另外，要支持Write Back模式需要以下版本的介质（老版本的cellcli上使用DESCRIBE CELL命令也看不见flashCacheMode这个属性）：
 DB Patch for Exadata 11.2.0.3.x BP 9以上
 推荐Exadata 11.2.3.2.x + patch 16042459（取代了以前的patch 15841041），具体请参考MOS NOTES 1515324.1和 1485475.1）。
 patch 16042459的主要修复了如下问题，具体请参考MOS NOTES 1485475.1和1270094.1：

Write-back Smart Flash Cache fixes：
	。Systems running with Write-back Smart Flash Cache enabled should upgrade to this release (or later).
	。Note that a subset of the Write-back Smart Flash Cache fixes were backported andsupplied in one-off patch 16003324 (which superseded 15887843 and 15841041) for Exadata 11.2.3.2.0 to address Exadata critical issue EX11. Patch 16003324 is not required for 11.2.3.2.1. No
separate action during upgrade is required if patch 16003324 (or 15887843 or 15841041) is already installed.

另外，如果没有打patch的话，11.2.3.2.1版不带上面的FlashCacheMode选项。

顺便说一句，image 11.2.3.2.0 是X3-2出厂预设的最早版本。从2013年3月份左右，X3-2的image版本已经是11.2.3.2.1这个相对稳定的版本了。如果是X2或者以前的机器，那么可以升级image到指定的版本（11.2.3.2.0以上，并且apply patch 16003324），具体的更新信息请参考MOS NOTES 888828.1.

X3系列（image 11.2.3.2.0以后的版本）Exadata Smart Flash Cache Write-Back的特点：
1. 提高Write操作在多个场景的性能
 频繁更新表或索引的Case
 X3的话，可以达到磁盘20倍的Write IOPS
 V2或X2的话、磁盘10倍的Write IOPS

2. 对Database的Write IO直接缓存到flash
 Block根据LRU算法进行缓存
 缓存以后，Read与Write都在缓存上操作
 Block老化后再写入磁盘

3. 智能化缓存
 在不手工指定keep的情况下，只会缓存频繁访问的小表（非全表扫描的），他有一个算法，后面会具体描述
 以下IO不被缓存:
–Backup、DataPump I/O
–临时表空间的I/O
–Direct path insert（未设置Keep的对象）
–表的全扫描（未设置Keep的对象）
–表空间的格式化

4. Write Back模式的智能化缓存
 Write Back模式实现了永久性Write缓存（重启不丢失）
 Crash Recovery也更快
 Flash无需Recovery
 Data Guard的redo apply也更快（Redo Apply是依赖Write I/O的，据说速度大概是原来的3倍）
 Flash卡的故障由Exadata的ASM自动透明管理，缓存的内容通过其他Cell的镜像数据来进行恢复

Exadata Smart Flash Cache Write-Back:
Flash Cache作为Cell中磁盘的缓存，其冗余性由ASM来保障，其读写也是以4MB的AU为单位进行条带化和镜像的。Flash Cache的数据跨越多个Fail group（分布在各个Cell）写入primary copy和mirror copy（类似于磁盘数据）。

1.Flach Cache 总是作为磁盘缓存，冗余性由ASM来保障：

更新数据到DB buffer Cache，然后DBWR进程发起写数据的请求通过iDB协议给Cellsrv，Cellsrv收到请求后，直接写入Flash Cahche。注意DBWR进程会写两份数据，一个是Primary copy，一个是Secondary copy，流程都一样（这是ASM新特性的机制保证的）。此时，磁盘上依然保留着旧数据，当数据age out后，会被同步到磁盘。而这期间，flash cache的数据跟磁盘数据的工作模式一样：都是ASM来保证的。

2. 读数据的时候，直接读取Flash上的Primary Copy：

客户端发出请求，Server Process进程将请求通过iDB协议传给Cellsrv，Cellsrv检测到Flash Cache的Primary copy，于是将这个数据返回给server process进程。因此，读的时候都是直接读取Flash上的Primary Copy，Mirror Copy一般在Read的时候不需使用，当Secondry copy
数据Age Out的时候会被写入磁盘。当然，你也可以手工将数据agent out，以触发将flash cache写入磁盘的操作。

3、Backup 或 ASM Diskgroup恢复的时候，读取最新的数据：

 磁盘上有最新的数据的话，则从磁盘读取
 只在Flash cache上有最新的数据的话、从Flash cache读取。
 数据的写入则在Disk执行，这类IO不被缓存。

4、读取时发生的Flash cache数据的故障切换：

客户端发出请求，Server process进程将请求发给Cellsrv，Cellsrv发现flashcache有最新的数据，同时也发现这个flash cache出现故障了，不可读取，于是就直接从其他Cell的磁盘读取镜像数据。这个过程中，Cellsrv会直接自动重定向到保存着最新数据的Cell，因此不会返回应用错误。
这里别忘记了，上面“2. Read的时候，直接读取Flash上的Primary Copy”的时候介绍了，当数据age out后，会被从flash cache写入磁盘，或者手工出发一个age的操作，也会将数据同步到磁盘。

5、Resilvering Rebalance：

Flash发生故障时、将利用其他Cell上的镜像数据采取称之为”Resilvering”的数据处理。
这个操作不需要像磁盘一样手工做Drop disk和Add disk那样的Rebalance处理，但是你仍然可以通过 v$asm_operation 查询状态，并且其自动Rebalance的操作跟disk的类似，其Rebalance power可以通过初始化参数asm_power_limit、或者alter diskgroup power语句来影响。

下面是ASM的alert.log中将会记录resilvering rebalance操作：

Fri Nov 02 16:27:11 2012
NOTE: repairing group 2 file 268 extent 0
Errors in file /u01/app/oracle/diag/asm/+asm/+ASM2/trace/+ASM2_r000_123901.trc:
ORA-27603: Cell storage I/O error, I/O failed on disk o/192.168.20.54/DATA_L_CD_07_test04 at offset 24683479040 for data length 1048576
ORA-27626: Exadata error: 223 (Block needs to be resilvered)  提示块需要resilvering
WARNING: Read Failed. group:2 disk:41 AU:5885 offset:0 size:1048576
path:o/192.168.20.54/DATA_L_CD_07_test04
         incarnation:0xe9688e67 synchronous result:'I/O error'
         subsys:OSS iop:0x7f2b7246edc0 bufp:0x7f2b7235de00 osderr:0xdf osderr1:0x0
SUCCESS: extent 0 of file 268 group 2 repaired - all online mirror sides found readable, no repair required
Fri Nov 02 16:27:19 2012
Starting background process XDWK

XDWK是XD上ASM特有的进程，即Exadata Automation Worker进程，他负责执行由XDMG后台进程发起的请求。比如对磁盘进行ONLINE , DROP, ADD等操作的时候，XDMG进程就会发出请求让XDWK进程去具体落实了。XDWK进程在inactive 5分钟后，自动stop。简单的说，就是XDMG是监工，负责监控disk和cell的状态，当他们需要online，drop，add等操作的时候，XDMG就会去启动或者唤醒XDWK进程来干活)

Fri Nov 02 16:27:19 2012
XDWK started with pid=39, OS id=123956
SQL&gt; /* Exadata Auto Mgmt: Resilver diskgroup */   自动执行resilvering rebalance
alter diskgroup DATA_L
  rebalance nowait
NOTE: GroupBlock outside rolling migration privileged region
NOTE: requesting all-instance membership refresh for group=2
…..
SUCCESS: /* Exadata Auto Mgmt: Resilver diskgroup */

可以使用list griddisk查看当前flash cache使用的flash disk的相关信息：

X3-2的每块Flash Card（每块Card内有4个Flash Module）的数据映射到12块Celldisk其中的3块之上，这样的算法是为了最大化保护数据（就如同最小的ED配置是3个cell一个道理）。

可以使用“list griddisk attributes name, cachedby”命令查看：

出现故障时的状态：

待续……

发表在 FAQ, 体系架构, 硬件配置 | 标签为 Exadata, flash cache | 留下评论

Smart Flash Cache on Exadata(2)—Write through

发表于 2013 年 5 月 28 日由 Lunar

每个Exadata配备的Flash disk，这些flash disk可以创建flash grid disks来存放频繁访问的数据，也可以做成Exadata Smart Flash Cache，然后将频繁访问的数据cache在Exadata Smart Flash Cache。

缺省情况下，Create Cell命令会创建create flash cell disks，然后在这些flash celldisk上创建Exadata Smart Flash Cache。下面是使用onecommand安装过程中创建celldisk和griddisk的步骤，缺省是将flash disk创建为 Exadata Smart Flash Cache（不同版本的onecommand执行情况稍有不同）：

INFO: Running /usr/local/bin/dcli -g /opt/oracle.SupportTools/onecommand/cell_group -l root "cellcli -e create celldisk all " to create all celldisks...

然后创建 flashlog:

INFO: Running /usr/local/bin/dcli -g /opt/oracle.SupportTools/onecommand/cell_group -l root "cellcli -e create flashlog all " to create flashlog on cells...

然后创建 flashcache:

INFO: Running /usr/local/bin/dcli -g /opt/oracle.SupportTools/onecommand/cell_group -l root "cellcli -e create flashcache all " to create flashcache

当然，你可以根据情况创建为FlashDisk(而非Flash Cache):

一般情况下，我们有这样的对比数据（多次测试对比的结果和个人经验值）：

Disk的IO： 168块600G的光线盘，容量大概 100TB，吞吐量在 25GB/sec，左右
Flash的IO： 56 PCI Cards，每块96G，容量大概 22.4TB，吞吐量在 100GB/sec 左右
固态盘的IO: 16块256MB的 DIMM，容量大概 4TB，吞吐量在 200GB/sec 左右
当然具体场景要根据很多因素，比如性价比、应用场景，等等。

Flash PCI卡的作用就是消除磁盘控制器的瓶颈。根据Oracle官方的说明，Exadata上对Flash cache的使用对应用透明，且使用了比普通LRU更为优化的算法，他会自动判断哪些都像会被缓存，当然，也可以手工指定：

ALTER TABLE <表名> STORAGE (CELL_FLASH_CACHE KEEP)
ALTER INDEX <索引名> STORAGE (CELL_FLASH_CACHE KEEP)

Exadata Smart Flash Cache不能缓存的IO有如下类型：
 ASM的镜像的I/O
 Rman Backup、
 DataPump I/O
 表空间的格式化
 全表扫描（表没有被Keep的情况下）

Exadata Smart Flash Cache自动缓存的IO有如下类型：
 控制文件的Read/Write I/O
 文件头的Read/Write I/O
 Data Block与Index Block

Exadata Smart Flash Cache的读写流程(Write Through模式)：
1.缓存对象时的Write动作

2.不缓存对象时的Write动作
1)写入Disk
2)写完后返回ACK
（比如全表扫描的大表等等）

3.缓存数据的Read动作

4.已缓存到Flash的数据Read动作

通过上述流程，不难发现，Flash的使用特点：
1）Flash Cache不需要考虑数据的冗余，因为所有数据都在磁盘上保留了一份
2）Flash Cache可以结合磁盘扫表一起完成查询
3）考虑一下，如果压缩率为10：1，那么X3-2的Exadata 1/4Rack包含5.6T的闪存容量，可以总共可以缓存50T的数据

待续……

发表在 FAQ, 体系架构, 硬件配置 | 标签为 Exadata, flash cache | 一条评论

Smart Flash Cache on Exadata(1)—介绍

发表于 2013 年 5 月 28 日由 Lunar

Exadata上的闪存卡是做什么用的呢？
引用一段白皮书的内容： Exadata智能闪存（Smart Flash Card）自动将频繁访问的数据在PCI闪存中缓存，同时不常访问的数据仍保持在磁盘中。这种方式兼具闪存的高性能与磁盘的大容量和低成本。Exadata智能闪存了解数据库的工作负载，并知道何时避免缓存不被重复利用或不适合缓存的数据。Oracle数据库和Exadata存储允许用户任意指定数据库表、索引和段级别的数据驻留在闪存上。和传统的存储和闪存盘不同的是，表可以驻留在闪存中，且不必将表移动到另外的表空间，文件或卷中。

Exadata系统采用的是最新的 PCI闪存技术而不是闪存盘技术。PCI闪存是将闪存直接部署在高速PCI总线上，代替了部署在缓慢的磁盘控制器和磁盘管理器之后的方式，大大提升了性能。

Exadata上Smart Flash Cache的发展
2009年， Exadata V2开始配置Flash Card(F20)，每个cell配置了384G的闪存，详见V2的白皮书：

具体的硬件信息请参考： http://pts-storage.us.oracle.com
相应的V2的文档请参考： http://docs.oracle.com/cd/E18476_01/index.htm

V2上Exadata的Flash Card的配置如下:
• Full rack => (14 hosts x 4 F20s) + 2 replacement F20s = 58 F20s
• Half rack => (7 hosts x 4 F20s) + 1 replacement F20s = 29 F20s
• Quarter rack => (3 hosts x 4 F20s) + 1 replacement F20s = 13 F20s

硬件支持：

其限制条件如下:
• x4275 hosts are supported
• Linux is suppored. Exadata runs on Oracle’s version of RedHat Linux.

2010年，Oracle推出了X2系列，一个满配可以达到5.3TB的Exadata Smart Flash Cache（96G*4块卡*14个cell）：

从X3开始，每个cell节点配置了4个400G的Flash Card：
每个Exadata存储服务器包含4个PCI闪存卡，共计1.6TB闪存空间（4 x 400 GB）。满配机架的Exadata Database Machine包含56个闪存卡，共计22.4TB闪存空间。

待续…………

发表在 FAQ, 体系架构, 硬件配置 | 标签为 Exadata, flash cache | 留下评论

Exadata 升级过程（二）—catbundle.sql

发表于 2013 年 5 月 26 日由 Lunar

select capture_name from dba_capture where capture_name not like 'OGG$%';

select capture_name from dba_capture;

SQL> exec dbms_capture_adm.stop_capture('capture_name');
// where capture_name is a value returned from select statement above 

SQL> select apply_name from dba_apply;

SQL> exec dbms_apply_adm.stop_apply('apply_name');
// where apply_name is a value returned from select statement above


Reload the packages into the database, run the following command (make sure you have run step 1 which is mandatory for this step run):
cd $ORACLE_HOME
SQL> connect / as sysdba
SQL> @rdbms/admin/catbundle.sql exa apply

The catbundle.sql execution is reflected in the dba_registry_history view by a row associated with bundle series EXA.
For information about the catbundle.sql script, see My Oracle Support Note 605795.1 Introduction to Oracle Database catbundle.sql.


检查是否有报错：
cd $ORACLE_HOME/cfgtoollogs/catbundle
grep ^ORA * | sort -u 

the format of the <TIMESTAMP> is YYYYMMMDD_HH_MM_SS：
catbundle_EXA_<database SID>_APPLY_<TIMESTAMP>.log
catbundle_EXA_<database SID>_GENERATE_<TIMESTAMP>.log

发表在安装和升级 | 标签为 catbundle, Exadata, 升级 | 留下评论

Exadata 升级过程（三）–升级后的检查

发表于 2013 年 5 月 26 日由 Lunar

10、升级后的检查：
检查系统的image版本：

[root@dm02db01 onecommand]# dcli -g all_group -l root "imagehistory"
dm02db01: Version                              : 11.2.3.1.1.120607
dm02db01: Image activation date                : 2012-08-08 15:03:41 +0800
dm02db01: Imaging mode                         : fresh   刷机的（有时候出厂介质也是这个状态）
dm02db01: Imaging status                       : success
dm02db01: 
dm02db01: Version                              : 11.2.3.2.1.130302
dm02db01: Image activation date                : 2013-05-23 13:55:40 +0800
dm02db01: Imaging mode                         : patch   打patch的
dm02db01: Imaging status                       : success
dm02db01: 
....
dm02cel13: 
dm02cel14: Version                              : 11.2.3.1.1.120607
dm02cel14: Image activation date                : 2012-08-08 14:13:24 +0800
dm02cel14: Imaging mode                         : fresh
dm02cel14: Imaging status                       : success
dm02cel14: 
dm02cel14: Version                              : 11.2.3.2.1.130109
dm02cel14: Image activation date                : 2013-05-23 13:20:51 +0800
dm02cel14: Imaging mode                         : out of partition upgrade
dm02cel14: Imaging status                       : success
dm02cel14:

可以看见操作系统已经是UEK内核，版本是2.6.32

[root@dm02db01 onecommand]# dcli -g all_group -l root "uname -a"
dm02db01: Linux dm02db01.hrss 2.6.32-400.21.1.el5uek #1 SMP Wed Feb 20 01:35:01 PST 2013 x86_64 x86_64 x86_64 GNU/Linux
dm02db02: Linux dm02db02.hrss 2.6.32-400.21.1.el5uek #1 SMP Wed Feb 20 01:35:01 PST 2013 x86_64 x86_64 x86_64 GNU/Linux
...
dm02cel13: Linux dm02cel13.hrss 2.6.32-400.11.1.el5uek #1 SMP Thu Nov 22 03:29:09 PST 2012 x86_64 x86_64 x86_64 GNU/Linux
dm02cel14: Linux dm02cel14.hrss 2.6.32-400.11.1.el5uek #1 SMP Thu Nov 22 03:29:09 PST 2012 x86_64 x86_64 x86_64 GNU/Linux
[root@dm02db01 onecommand]#

Oracle Linux Server版本升级为 5.8(以前是5.7)：

[root@dm02db01 onecommand]# dcli -g all_group -l root "cat /etc/oracle-release"
dm02db01: Oracle Linux Server release 5.8
dm02db02: Oracle Linux Server release 5.8
。。。
dm02cel13: Oracle Linux Server release 5.8
dm02cel14: Oracle Linux Server release 5.8
[root@dm02db01 onecommand]#

检查 Infiniband：

[root@dm02db01 oracle.SupportTools]# ./CheckSWProfile.sh -I IS_SPINEdm02sw-ib1,dm02sw-ib2,dm02sw-ib3
Checking if switch dm02sw-ib1 is pingable...
Checking if switch dm02sw-ib2 is pingable...
Checking if switch dm02sw-ib3 is pingable...
Use the default password for all switches? (y/n) [n]:   
Use same password for all switches? (y/n) [n]: y    
Enter admin or root password for All_Switches:      
Confirm password:																		
[INFO] SUCCESS Switch dm02sw-ib1 has correct software and firmware version:
           SWVer: 1.3.3-2
[INFO] SUCCESS Switch dm02sw-ib1 has correct opensm configuration:
           controlled_handover=TRUE polling_retry_number=5 routing_engine=ftree sminfo_polling_timeout=1000 sm_priority=8 

[INFO] SUCCESS Switch dm02sw-ib2 has correct software and firmware version:
           SWVer: 1.3.3-2
[INFO] SUCCESS Switch dm02sw-ib2 has correct opensm configuration:
           controlled_handover=TRUE polling_retry_number=5 routing_engine=ftree sminfo_polling_timeout=1000 sm_priority=5 

[INFO] SUCCESS Switch dm02sw-ib3 has correct software and firmware version:
           SWVer: 1.3.3-2
[INFO] SUCCESS Switch dm02sw-ib3 has correct opensm configuration:
           controlled_handover=TRUE polling_retry_number=5 routing_engine=ftree sminfo_polling_timeout=1000 sm_priority=5 

[INFO] SUCCESS All switches have correct software and firmware version:
           SWVer: 1.3.3-2
[INFO] SUCCESS All switches have correct opensm configuration:
           controlled_handover=TRUE polling_retry_number=5 routing_engine=ftree sminfo_polling_timeout=1000 sm_priority=5 for non spine and 8 for spine switch5 
[root@dm02db01 oracle.SupportTools]#

检查数据库的版本信息：

SQL> select * from dba_registry_history;

ACTION_TIME                    ACTION NAMESPACE  VERSION               ID BUNDLE_SERIES   COMMENTS
------------------------------ ------ ---------- ------------- ---------- --------------- -------------------------
2012-09-04 18:41:36.254667     APPLY  SERVER     11.2.0.3               0 PSU             Patchset 11.2.0.2.0
2013-05-24 20:55:49.220036     APPLY  SERVER     11.2.0.3              16 EXA             BP16
2013-05-24 20:58:59.272651     APPLY  SERVER     11.2.0.3              16 EXA             BP16
2013-05-24 21:28:07.498157     APPLY  SERVER     11.2.0.3              16 EXA             BP16

检查opatch的信息：

[grid@dm02db01 ~]$ $ORACLE_HOME/OPatch/opatch lsinventory -detail -oh $ORACLE_HOME
Oracle Interim Patch Installer version 11.2.0.3.3
Copyright (c) 2012, Oracle Corporation.  All rights reserved.


Oracle Home       : /u01/app/11.2.0.3/grid
Central Inventory : /u01/app/oraInventory
   from           : /u01/app/11.2.0.3/grid/oraInst.loc
OPatch version    : 11.2.0.3.3
OUI version       : 11.2.0.3.0
Log file location : /u01/app/11.2.0.3/grid/cfgtoollogs/opatch/opatch2013-05-26_12-12-39PM_1.log

Lsinventory Output file location : /u01/app/11.2.0.3/grid/cfgtoollogs/opatch/lsinv/lsinventory2013-05-26_12-12-39PM.txt

--------------------------------------------------------------------------------
Installed Top-level Products (1): 

Oracle Grid Infrastructure                                           11.2.0.3.0
There are 1 products installed in this Oracle Home.


Installed Products (88): 

Agent Required Support Files                                         10.2.0.4.3
Assistant Common Files                                               11.2.0.3.0
Automatic Storage Management Assistant                               11.2.0.3.0
Bali Share                                                           1.1.18.0.0
Buildtools Common Files                                              11.2.0.3.0
Character Set Migration Utility                                      11.2.0.3.0
Cluster Ready Services Files                                         11.2.0.3.0
Cluster Verification Utility Common Files                            11.2.0.3.0
Cluster Verification Utility Files                                   11.2.0.3.0
Database SQL Scripts                                                 11.2.0.3.0
Deinstallation Tool                                                  11.2.0.3.0
Enterprise Manager Common Core Files                                 10.2.0.4.4
Enterprise Manager Common Files                                      10.2.0.4.3
Enterprise Manager plugin Common Files                               11.2.0.3.0
Expat libraries                                                       2.0.1.0.1
HAS Common Files                                                     11.2.0.3.0
HAS Files for DB                                                     11.2.0.3.0
Installation Common Files                                            11.2.0.3.0
Installation Plugin Files                                            11.2.0.3.0
Installer SDK Component                                              11.2.0.3.0
LDAP Required Support Files                                          11.2.0.3.0
OLAP SQL Scripts                                                     11.2.0.3.0
Oracle Clusterware RDBMS Files                                       11.2.0.3.0
Oracle Configuration Manager Deconfiguration                         10.3.1.0.0
Oracle Containers for Java                                           11.2.0.3.0
Oracle Core Required Support Files                                   11.2.0.3.0
Oracle Database 11g                                                  11.2.0.3.0
Oracle Database 11g Multimedia Files                                 11.2.0.3.0
Oracle Database Deconfiguration                                      11.2.0.3.0
Oracle Database User Interface                                       2.2.13.0.0
Oracle Database Utilities                                            11.2.0.3.0
Oracle DBCA Deconfiguration                                          11.2.0.3.0
Oracle Extended Windowing Toolkit                                    3.4.47.0.0
Oracle Globalization Support                                         11.2.0.3.0
Oracle Globalization Support                                         11.2.0.3.0
Oracle Grid Infrastructure                                           11.2.0.3.0
Oracle Help For Java                                                  4.2.9.0.0
Oracle Ice Browser                                                    5.2.3.6.0
Oracle Internet Directory Client                                     11.2.0.3.0
Oracle Java Client                                                   11.2.0.3.0
Oracle JDBC/OCI Instant Client                                       11.2.0.3.0
Oracle JDBC/THIN Interfaces                                          11.2.0.3.0
Oracle JFC Extended Windowing Toolkit                                4.2.36.0.0
Oracle JVM                                                           11.2.0.3.0
Oracle LDAP administration                                           11.2.0.3.0
Oracle Locale Builder                                                11.2.0.3.0
Oracle Multimedia                                                    11.2.0.3.0
Oracle Multimedia Client Option                                      11.2.0.3.0
Oracle Multimedia Java Advanced Imaging                              11.2.0.3.0
Oracle Multimedia Locator                                            11.2.0.3.0
Oracle Multimedia Locator RDBMS Files                                11.2.0.3.0
Oracle Net                                                           11.2.0.3.0
Oracle Net Listener                                                  11.2.0.3.0
Oracle Net Required Support Files                                    11.2.0.3.0
Oracle Netca Client                                                  11.2.0.3.0
Oracle Notification Service                                          11.2.0.3.0
Oracle Notification Service (eONS)                                   11.2.0.3.0
Oracle One-Off Patch Installer                                       11.2.0.1.7
Oracle Quality of Service Management (Client)                        11.2.0.3.0
Oracle Quality of Service Management (Server)                        11.2.0.3.0
Oracle RAC Deconfiguration                                           11.2.0.3.0
Oracle RAC Required Support Files-HAS                                11.2.0.3.0
Oracle Recovery Manager                                              11.2.0.3.0
Oracle Security Developer Tools                                      11.2.0.3.0
Oracle Text Required Support Files                                   11.2.0.3.0
Oracle Universal Installer                                           11.2.0.3.0
Oracle USM Deconfiguration                                           11.2.0.3.0
Oracle Wallet Manager                                                11.2.0.3.0
Parser Generator Required Support Files                              11.2.0.3.0
Perl Interpreter                                                     5.10.0.0.2
Perl Modules                                                         5.10.0.0.1
PL/SQL                                                               11.2.0.3.0
PL/SQL Embedded Gateway                                              11.2.0.3.0
Platform Required Support Files                                      11.2.0.3.0
Precompiler Required Support Files                                   11.2.0.3.0
RDBMS Required Support Files                                         11.2.0.3.0
RDBMS Required Support Files for Instant Client                      11.2.0.3.0
RDBMS Required Support Files Runtime                                 11.2.0.3.0
Required Support Files                                               11.2.0.3.0
Secure Socket Layer                                                  11.2.0.3.0
SQL*Plus                                                             11.2.0.3.0
SQL*Plus Files for Instant Client                                    11.2.0.3.0
SQL*Plus Required Support Files                                      11.2.0.3.0
SSL Required Support Files for InstantClient                         11.2.0.3.0
Sun JDK                                                             1.5.0.30.03
Universal Storage Manager Files                                      11.2.0.3.0
XDK Required Support Files                                           11.2.0.3.0
XML Parser for Java                                                  11.2.0.3.0
There are 88 products installed in this Oracle Home.


Interim patches (3) :

Patch  16401300     : applied on Thu May 23 20:45:22 CST 2013
Unique Patch ID:  15963835
Patch description:  "DISKMON PATCH FOR EXADATA (MAR 2013 - 11.2.0.3.16) : (16401300)"
   Created on 26 Feb 2013, 03:30:34 hrs PST8PDT
   Bugs fixed:
     13513982, 13688022, 14050993, 14307915, 13350234, 16401300, 14184077
   Files Touched:
     diskmon.bin --> ORACLE_HOME/bin/diskmon.bin
   Patch Location in Inventory:
     /u01/app/11.2.0.3/grid/inventory/oneoffs/16401300
   Patch Location in Storage area:
     /u01/app/11.2.0.3/grid/.patch_storage/16401300_Feb_26_2013_03_30_34

Patch  16355082     : applied on Thu May 23 20:44:29 CST 2013
Unique Patch ID:  15963835
Patch description:  "CRS PATCH FOR EXADATA (MAR 2013 - 11.2.0.3.16) : (16355082)"
   Created on 12 Mar 2013, 19:07:31 hrs PST8PDT
......
Rac system comprising of multiple nodes
  Local node = dm02db01
  Remote node = dm02db02
  Remote node = dm02db03
  Remote node = dm02db04
  Remote node = dm02db05
  Remote node = dm02db06
  Remote node = dm02db07
  Remote node = dm02db08

--------------------------------------------------------------------------------

OPatch succeeded.
[grid@dm02db01 ~]$ exit
logout

[oracle@dm02db01 ~]$ $ORACLE_HOME/OPatch/opatch lsinventory -detail -oh $ORACLE_HOME
Oracle Interim Patch Installer version 11.2.0.3.3
Copyright (c) 2012, Oracle Corporation.  All rights reserved.


Oracle Home       : /u01/app/oracle/product/11.2.0.3/dbhome_1
Central Inventory : /u01/app/oraInventory
   from           : /u01/app/oracle/product/11.2.0.3/dbhome_1/oraInst.loc
OPatch version    : 11.2.0.3.3
OUI version       : 11.2.0.3.0
Log file location : /u01/app/oracle/product/11.2.0.3/dbhome_1/cfgtoollogs/opatch/opatch2013-05-26_12-13-17PM_1.log

Lsinventory Output file location : /u01/app/oracle/product/11.2.0.3/dbhome_1/cfgtoollogs/opatch/lsinv/lsinventory2013-05-26_12-13-17PM.txt

--------------------------------------------------------------------------------
Installed Top-level Products (1): 

Oracle Database 11g                                                  11.2.0.3.0
There are 1 products installed in this Oracle Home.


Installed Products (136): 

Agent Required Support Files                                         10.2.0.4.3
Assistant Common Files                                               11.2.0.3.0
Bali Share                                                           1.1.18.0.0
Buildtools Common Files                                              11.2.0.3.0
Character Set Migration Utility                                      11.2.0.3.0
Cluster Verification Utility Common Files                            11.2.0.3.0
Database Configuration and Upgrade Assistants                        11.2.0.3.0
Database SQL Scripts                                                 11.2.0.3.0
Database Workspace Manager                                           11.2.0.3.0
Deinstallation Tool                                                  11.2.0.3.0
Enterprise Edition Options                                           11.2.0.3.0
Enterprise Manager Agent                                             10.2.0.4.3
Enterprise Manager Agent Core Files                                  10.2.0.4.4
Enterprise Manager Common Core Files                                 10.2.0.4.4
Enterprise Manager Common Files                                      10.2.0.4.3
Enterprise Manager Database Plugin -- Agent Support                  11.2.0.3.0
Enterprise Manager Database Plugin -- Repository Support             11.2.0.3.0
Enterprise Manager Grid Control Core Files                           10.2.0.4.4
Enterprise Manager plugin Common Files                               11.2.0.3.0
Enterprise Manager Repository Core Files                             10.2.0.4.4
Exadata Storage Server                                               11.2.0.1.0
Expat libraries                                                       2.0.1.0.1
Generic Connectivity Common Files                                    11.2.0.3.0
HAS Common Files                                                     11.2.0.3.0
HAS Files for DB                                                     11.2.0.3.0
Installation Common Files                                            11.2.0.3.0
Installation Plugin Files                                            11.2.0.3.0
Installer SDK Component                                              11.2.0.3.0
JAccelerator (COMPANION)                                             11.2.0.3.0
LDAP Required Support Files                                          11.2.0.3.0
OLAP SQL Scripts                                                     11.2.0.3.0
Oracle 11g Warehouse Builder Required Files                          11.2.0.3.0
Oracle Advanced Security                                             11.2.0.3.0
Oracle Application Express                                           11.2.0.3.0
Oracle Call Interface (OCI)                                          11.2.0.3.0
Oracle Clusterware RDBMS Files                                       11.2.0.3.0
Oracle Code Editor                                                   1.2.1.0.0I
Oracle Configuration Manager                                         10.3.5.0.1
Oracle Configuration Manager Client                                  10.3.2.1.0
Oracle Configuration Manager Deconfiguration                         10.3.1.0.0
Oracle Containers for Java                                           11.2.0.3.0
Oracle Core Required Support Files                                   11.2.0.3.0
Oracle Data Mining RDBMS Files                                       11.2.0.3.0
Oracle Database 11g                                                  11.2.0.3.0
Oracle Database 11g                                                  11.2.0.3.0
Oracle Database 11g Multimedia Files                                 11.2.0.3.0
Oracle Database Deconfiguration                                      11.2.0.3.0
Oracle Database Gateway for ODBC                                     11.2.0.3.0
Oracle Database User Interface                                       2.2.13.0.0
Oracle Database Utilities                                            11.2.0.3.0
Oracle Database Vault J2EE Application                               11.2.0.3.0
Oracle Database Vault option                                         11.2.0.3.0
Oracle DBCA Deconfiguration                                          11.2.0.3.0
Oracle Display Fonts                                                  9.0.2.0.0
Oracle Enterprise Manager Console DB                                 11.2.0.3.0
Oracle Extended Windowing Toolkit                                    3.4.47.0.0
Oracle Globalization Support                                         11.2.0.3.0
Oracle Globalization Support                                         11.2.0.3.0
Oracle Help For Java                                                  4.2.9.0.0
Oracle Help for the  Web                                             2.0.14.0.0
Oracle Ice Browser                                                    5.2.3.6.0
Oracle Internet Directory Client                                     11.2.0.3.0
Oracle Java Client                                                   11.2.0.3.0
Oracle JDBC Server Support Package                                   11.2.0.3.0
Oracle JDBC/OCI Instant Client                                       11.2.0.3.0
Oracle JDBC/THIN Interfaces                                          11.2.0.3.0
Oracle JFC Extended Windowing Toolkit                                4.2.36.0.0
Oracle JVM                                                           11.2.0.3.0
Oracle Label Security                                                11.2.0.3.0
Oracle LDAP administration                                           11.2.0.3.0
Oracle Locale Builder                                                11.2.0.3.0
Oracle Message Gateway Common Files                                  11.2.0.3.0
Oracle Multimedia                                                    11.2.0.3.0
Oracle Multimedia Annotator                                          11.2.0.3.0
Oracle Multimedia Client Option                                      11.2.0.3.0
Oracle Multimedia Java Advanced Imaging                              11.2.0.3.0
Oracle Multimedia Locator                                            11.2.0.3.0
Oracle Multimedia Locator RDBMS Files                                11.2.0.3.0
Oracle Net                                                           11.2.0.3.0
Oracle Net Listener                                                  11.2.0.3.0
Oracle Net Required Support Files                                    11.2.0.3.0
Oracle Net Services                                                  11.2.0.3.0
Oracle Netca Client                                                  11.2.0.3.0
Oracle Notification Service                                          11.2.0.3.0
Oracle Notification Service (eONS)                                   11.2.0.3.0
Oracle ODBC Driver                                                   11.2.0.3.0
Oracle ODBC Driverfor Instant Client                                 11.2.0.3.0
Oracle OLAP                                                          11.2.0.3.0
Oracle OLAP API                                                      11.2.0.3.0
Oracle OLAP RDBMS Files                                              11.2.0.3.0
Oracle One-Off Patch Installer                                       11.2.0.1.7
Oracle Partitioning                                                  11.2.0.3.0
Oracle Programmer                                                    11.2.0.3.0
Oracle Quality of Service Management (Client)                        11.2.0.3.0
Oracle RAC Deconfiguration                                           11.2.0.3.0
Oracle RAC Required Support Files-HAS                                11.2.0.3.0
Oracle Real Application Testing                                      11.2.0.3.0
Oracle Recovery Manager                                              11.2.0.3.0
Oracle Security Developer Tools                                      11.2.0.3.0
Oracle Spatial                                                       11.2.0.3.0
Oracle SQL Developer                                                 11.2.0.3.0
Oracle Starter Database                                              11.2.0.3.0
Oracle Text                                                          11.2.0.3.0
Oracle Text Required Support Files                                   11.2.0.3.0
Oracle UIX                                                           2.2.24.6.0
Oracle Universal Connection Pool                                     11.2.0.3.0
Oracle Universal Installer                                           11.2.0.3.0
Oracle USM Deconfiguration                                           11.2.0.3.0
Oracle Wallet Manager                                                11.2.0.3.0
Oracle XML Development Kit                                           11.2.0.3.0
Oracle XML Query                                                     11.2.0.3.0
Parser Generator Required Support Files                              11.2.0.3.0
Perl Interpreter                                                     5.10.0.0.2
Perl Modules                                                         5.10.0.0.1
PL/SQL                                                               11.2.0.3.0
PL/SQL Embedded Gateway                                              11.2.0.3.0
Platform Required Support Files                                      11.2.0.3.0
Precompiler Common Files                                             11.2.0.3.0
Precompiler Required Support Files                                   11.2.0.3.0
Provisioning Advisor Framework                                       10.2.0.4.3
RDBMS Required Support Files                                         11.2.0.3.0
RDBMS Required Support Files for Instant Client                      11.2.0.3.0
RDBMS Required Support Files Runtime                                 11.2.0.3.0
regexp                                                                2.1.9.0.0
Required Support Files                                               11.2.0.3.0
Sample Schema Data                                                   11.2.0.3.0
Secure Socket Layer                                                  11.2.0.3.0
SQL*Plus                                                             11.2.0.3.0
SQL*Plus Files for Instant Client                                    11.2.0.3.0
SQL*Plus Required Support Files                                      11.2.0.3.0
SQLJ Runtime                                                         11.2.0.3.0
SSL Required Support Files for InstantClient                         11.2.0.3.0
Sun JDK                                                             1.5.0.30.03
XDK Required Support Files                                           11.2.0.3.0
XML Parser for Java                                                  11.2.0.3.0
XML Parser for Oracle JVM                                            11.2.0.3.0
There are 136 products installed in this Oracle Home.


Interim patches (3) :

Patch  16401300     : applied on Thu May 23 20:37:10 CST 2013
Unique Patch ID:  15963835
Patch description:  "DISKMON PATCH FOR EXADATA (MAR 2013 - 11.2.0.3.16) : (16401300)"
   Created on 26 Feb 2013, 03:30:34 hrs PST8PDT
   Bugs fixed:
     13513982, 13688022, 14050993, 14307915, 13350234, 16401300, 14184077
   Files Touched:
     diskmon.bin --> ORACLE_HOME/bin/diskmon.bin
   Patch Location in Inventory:
     /u01/app/oracle/product/11.2.0.3/dbhome_1/inventory/oneoffs/16401300
   Patch Location in Storage area:
     /u01/app/oracle/product/11.2.0.3/dbhome_1/.patch_storage/16401300_Feb_26_2013_03_30_34

Patch  16355082     : applied on Thu May 23 20:36:52 CST 2013
Unique Patch ID:  15963835
Patch description:  "CRS PATCH FOR EXADATA (MAR 2013 - 11.2.0.3.16) : (16355082)"
   Created on 12 Mar 2013, 19:07:31 hrs PST8PDT
   Bugs fixed:
     15876003, 14275572, 13919095, 13696251, 13348650, 12659561, 13039908
     14277586, 13987807, 14625969, 13938166, 13825231, 13036424, 12794268
........
     /u01/app/oracle/product/11.2.0.3/dbhome_1/inventory/oneoffs/16233552
   Patch Location in Storage area:
     /u01/app/oracle/product/11.2.0.3/dbhome_1/.patch_storage/16233552_Mar_8_2013_02_05_30



Rac system comprising of multiple nodes
  Local node = dm02db01
  Remote node = dm02db02
  Remote node = dm02db03
  Remote node = dm02db04
  Remote node = dm02db05
  Remote node = dm02db06
  Remote node = dm02db07
  Remote node = dm02db08

--------------------------------------------------------------------------------

OPatch succeeded.
[oracle@dm02db01 ~]$ exit

发表在安装和升级 | 标签为 Exadata, patchmgr, yum | 留下评论

Exadata 之 CISCO Ethernet Switch安装配置

发表于 2013 年 5 月 26 日由 Lunar

通过串口线连接连接到CISCO Ethernet Switch的Console：

登录超级终端：

随便起个会话名称，例如Exadata：

COM 16就可以，这页不用修改，缺省就可以：

红色部分修改为9600，蓝色部分不用改（缺省就是8 bit）：

Serial连接参数如下：
Protocol: Serial
Baud rate: 9600
Data bits: 8
Parity: None
Stop bits: 1
no handshake
按回车键开始

切换至enable模式：
Switch>enable
Password: welcome1
Switch#

配置VLAN：
Switch#configure terminal
Switch(config)#interface vlan 1
Switch(config-if)#ip address 192.168.100.4 255.255.255.0
Switch(config-if)#end
Switch#write memory

禁用路由转发功能和配置默认路由：
Switch#configure terminal
Switch(config)#no ip routing
Switch(config)#ip default-gateway 192.168.100.254
Switch(config)#end
Switch#write memory

设置hostname：
Switch# configure terminal
Switch(config)#hostname dm01sw-adm
dm01sw-adm (config)#end
dm01sw-adm #write memory

设置密码：
dm01sw-adm #configure terminal
dm01sw-adm (config)#enable password welcome1
dm01sw-adm (config)#enable secret welcome1
dm01sw-adm (config)#end
dm01sw-adm #write memory

设置SSH接入的密码:
dm01sw-adm #configure terminal
dm01sw-adm(config)#line vty 0 15
dm01sw-adm(config-line)#password welcome1
dm01sw-adm(config-line)#login
dm01sw-adm(config-line)#end
dm01sw-adm#write memory

DNS服务器配置：
dm01sw-adm #configure terminal
dm01sw-adm(config-line)#ip domain-name cangpin.com
dm01sw-adm(config)#ip name-server 192.168.100.150
dm01sw-adm(config)#ip name-server 192.168.100.151
dm01sw-adm(config)#end
dm01sw-adm#write memory

设置时区和ntp服务器：
dm01sw-adm #configure terminal
szxdm02sw-ip(config)#clock timezone GMT +8
szxdm02sw-ip(config)#ntp server 192.168.100.155 prefer
szxdm02sw-ip(config)#ntp server 192.168.100.156
szxdm02sw-ip(config)#end
dm01sw-adm#write memory
dm01sw-adm#show ntp status
dm01sw-adm#show clock

检查配置：
dm01sw-adm#show running-config

发表在安装和升级 | 标签为 cisco, Exadata | 留下评论

Exadata升级过程(一)

发表于 2013 年 5 月 25 日由 Lunar

DB节点和CELL节点的升级都可以分为滚动个非滚动方式，为了节省时间，本次采用非滚动并行升级的方式。

本次升级从Exadata Image 11.2.3.1.1 升级到 Exadata 11.2.3.2.1。
数据库从Oracle 11.2.0.3 BP 8升级到 BP 16。

具体的升级步骤和过程还是要通读Oracle官方文档。

一、升级image前的准备工作

升级CELL节点需要使用patchmgr工具，首先要为patchmgr配置合适的SSH数据加密算法：

1、打开SSH的debug模式：

[root@dm02db01 ~]# ssh -v -v patchmgr_launch_node 2>ssh_client_debuglog.txt
[root@dm02db01 ~]#

这样就可以在ssh_client_debuglog.txt中查看debug信息：

[root@dm02db01 patch_11.2.3.2.1]# cat ssh_client_debuglog.txt 
OpenSSH_4.3p2, OpenSSL 0.9.8e-fips-rhel5 01 Jul 2008
debug1: Reading configuration data /root/.ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Applying options for *
debug2: ssh_connect: needpriv 0
debug1: Connecting to dm02db01.hrss [172.20.6.11] port 22.
debug1: Connection established.
debug1: permanently_set_uid: 0/0
debug1: identity file /root/.ssh/identity type -1
debug2: key_type_from_name: unknown key type '-----BEGIN'
debug2: key_type_from_name: unknown key type '-----END'
………………….

2、如果下面的命令输出结果为空，则表示需要为patchmgr设置数据加密算法：

[root@dm02db01 ~]# sed -e '/SSH2_MSG_KEXINIT received/,/first_kex_follows/!d'  \
> ssh_client_debuglog.txt | grep                              \
> 'aes128-ctr\|aes192-ctr\|aes256-ctr\|arcfour'
[root@dm02db01 ~]#

Exadata缺省没有配置特别的数据加密算法（只配置了一些数据传输的加密算法），因此，需要在/etc/ssh/ssh_config添加“Ciphers aes128-ctr,aes192-ctr,aes256-ctr,arcfour”，因为服务器需要至少其中一种数据加密算法：

[root@dm02db01 patch_11.2.3.2.1]# cat  /etc/ssh/ssh_config
#       $OpenBSD: ssh_config,v 1.21 2005/12/06 22:38:27 reyk Exp $

# This is the ssh client system-wide configuration file.  See
# ssh_config(5) for more information.  This file provides defaults for
# users, and the values can be changed in per-user configuration files
# or on the command line.

# Configuration data is parsed as follows:
#  1. command line options
#  2. user-specific file
#  3. system-wide file
# Any configuration value is only changed the first time it is set.
# Thus, host-specific definitions should be at the beginning of the
# configuration file, and defaults at the end.

# Site-wide defaults for some commonly used options.  For a comprehensive
# list of available options, their meanings and defaults, please see the
# ssh_config(5) man page.

# Host *
#   ForwardAgent no
#   ForwardX11 no
#   RhostsRSAAuthentication no
#   RSAAuthentication yes
#   PasswordAuthentication yes
#   HostbasedAuthentication no
#   BatchMode no
#   CheckHostIP yes
#   AddressFamily any
#   ConnectTimeout 0
#   StrictHostKeyChecking ask
#   IdentityFile ~/.ssh/identity
#   IdentityFile ~/.ssh/id_rsa
#   IdentityFile ~/.ssh/id_dsa
#   Port 22
#   Protocol 2,1
#   Cipher 3des
#   Ciphers aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,arcfour,aes192-cbc,aes256-cbc
#   EscapeChar ~
#   Tunnel no
#   TunnelDevice any:any
#   PermitLocalCommand no
Host *
        GSSAPIAuthentication yes
# If this option is set to yes then remote X11 clients will have full access
# to the original X11 display. As virtually no X11 client supports the untrusted
# mode correctly we set this to yes.
        ForwardX11Trusted yes
# Send locale-related environment variables
        SendEnv LANG LC_CTYPE LC_NUMERIC LC_TIME LC_COLLATE LC_MONETARY LC_MESSAGES 
        SendEnv LC_PAPER LC_NAME LC_ADDRESS LC_TELEPHONE LC_MEASUREMENT 
        SendEnv LC_IDENTIFICATION LC_ALL

Ciphers aes128-ctr,aes192-ctr,aes256-ctr,arcfour
[root@dm02db01 patch_11.2.3.2.1]#

3、检查各个cell节点之间root用户安全的信任关系（ssh User Equivalency）：
我这里检查的“all_group”，这个文件包含了全部的结算节点和cell节点：

[root@dm02db01 patch_11.2.3.2.1]# dcli -g all_group -l root date
dm02db01: Thu May 23 10:28:31 CST 2013
dm02db02: Thu May 23 10:28:40 CST 2013
…………….
dm02cel13: Thu May 23 10:26:37 CST 2013
dm02cel14: Thu May 23 10:26:25 CST 2013
[root@dm02db01 patch_11.2.3.2.1]#

如果root的安全信任关系没有建立，可以使用如下方法建立，否则，可以直接进入“4、检测磁盘组属性disk_repair_time配置”：
创建ssh秘钥：

[root@dm01db01 tmp]# ssh-keygen -t rsa
Generating public/private rsa key pair.

Enter file in which to save the key (/root/.ssh/id_rsa): Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
07:0f:1c:6e:78:42:6d:d0:15:42:1b:ab:ea:39:23:17 root@dm01db01.scq.com
[root@dm01db01 tmp]#

使用dcli的 -k 参数将秘钥添加到其他cell节点：

usage: dcli [options] [command]

options:
  --version            show program's version number and exit
  -c CELLS             comma-separated list of cells
  -d DESTFILE          destination directory or file
  -f FILE              files to be copied
  -g GROUPFILE         file containing list of cells
  -h, --help           show help message and exit
  -k                   push ssh key to cell's authorized_keys file
  -l USERID            user to login as on remote cells (default: celladmin)
  --maxlines=MAXLINES  limit output lines from a cell when in parallel
                       execution over multiple cells (default: 100000)
  -n                   abbreviate non-error output
  -r REGEXP            abbreviate output lines matching a regular expression
  -s SSHOPTIONS        string of options passed through to ssh
  --scp=SCPOPTIONS     string of options passed through to scp if different
                       from sshoptions
  --serial             serialize execution over the cells
  -t                   list target cells
  --unkey              drop keys from target cells' authorized_keys file
  -v                   print extra messages to stdout
  --vmstat=VMSTATOPS   vmstat command options
  -x EXECFILE          file to be copied and executed
[root@dm01db01 tmp]

[root@dm01db01 tmp]# cp /opt/oracle.SupportTools/onecommand/cell_group .
[root@dm01db01 tmp]# dcli -g cell_group -l root -k 
The authenticity of host 'dm01cel01 (9.9.10.3)' can't be established.
RSA key fingerprint is bf:1e:72:a9:fc:23:9a:2c:e8:0d:5f:2a:ce:02:ea:a7.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'dm01cel01,9.9.10.3' (RSA) to the list of known hosts.
root@dm01cel01's password:  这里输入root的口令
The authenticity of host 'dm01cel03 (9.9.10.5)' can't be established.
RSA key fingerprint is f7:b8:42:33:c4:9e:65:a6:69:fe:f4:b7:f7:dc:33:44.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'dm01cel03,9.9.10.5' (RSA) to the list of known hosts.
root@dm01cel03's password: 这里输入root的口令
The authenticity of host 'dm01cel02 (9.9.10.4)' can't be established.
RSA key fingerprint is 09:b7:58:8b:d5:75:71:d5:8c:bc:f0:36:b5:9b:58:fd.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'dm01cel02,9.9.10.4' (RSA) to the list of known hosts.
root@dm01cel02's password: 这里输入root的口令
dm01cel01: ssh key added
dm01cel02: ssh key added
dm01cel03: ssh key added
[root@dm01db01 tmp]#

检查个cell节点安全的信任关系是否正确配置：

[root@dm01db01 tmp]# dcli -g cell_group -l root 'hostname -i'
dm01cel01: 9.9.10.3
dm01cel02: 9.9.10.4
dm01cel03: 9.9.10.5
[root@dm01db01 tmp]#

4、检测磁盘组属性disk_repair_time配置（这是11g ASM的一个新特性，这里不单独赘述）：

sqlplus> select dg.name,a.value from v$asm_diskgroup dg, v$asm_attribute a 
         where dg.group_number=a.group_number and a.name='disk_repair_time';
NAME			       VALUE
------------------------------ ----------------------------------------
DATA_DG		       3.6h
DBFS_DG 		       3.6h
RECO_DG		       3.6h

修改这个主要是为了避免升级过程中达到缺省的3.6小时后在cell节点执行删除griddisk的操作。如果发生删除了griddisk的情况，那么，需要升级完成后手工添加这些磁盘组.

这个缺省值一般来说可以满足需求了。本次考虑升级满配的时间可能会超过3.6小时（这个是缺省值），因此将该值修改为10小时，等到升级完成后，可以讲该参数再改回3.6小时

SQL> alter diskgroup DATA_DM02 set attribute 'disk_repair_time'='10h';
Diskgroup altered.
SQL> alter diskgroup DBFS_DG set attribute 'disk_repair_time'='10h';
Diskgroup altered.
SQL> alter diskgroup RECO_DM02 set attribute 'disk_repair_time'='10h';
Diskgroup altered.
SQL>
SQL> select dg.name,a.value from v$asm_diskgroup dg, v$asm_attribute a 
  2  where dg.group_number=a.group_number and a.name='disk_repair_time';

NAME                           VALUE
------------------------------ ------------------------------
DATA_DM02                      10h
DBFS_DG                        10h
RECO_DM02                      10h

SQL>

5、检查操作系统内核版本
因为Exadata当前的image是11.2.3.1.1，从image 11.2.3.2.1开始，Oracle推荐使用ORACLE UEK（Unbreakable Enterprise Kernel）内核。UEK内核根据Oracle databae和中间件等软件的特点优化后内核，是从Linux 2.6.32开始的。

具体信息请参考：
http://www.oracle.com/us/technologies/linux/uek-r2-features-and-benefits-1555063.pdf
当然，db节点依然可以使用非UEK的内核（兼容RedHat Linux），但是这不是推荐的方式：
当前的内核版本：

[root@dm02db01 patch_11.2.3.2.1]# dcli -g all_group -l root 'uname -a'
dm02db01: Linux dm02db01.hrss 2.6.18-274.18.1.0.1.el5 #1 SMP Thu Feb 9 19:07:16 EST 2012 x86_64 x86_64 x86_64 GNU/Linux
dm02db02: Linux dm02db02.hrss 2.6.18-274.18.1.0.1.el5 #1 SMP Thu Feb 9 19:07:16 EST 2012 x86_64 x86_64 x86_64 GNU/Linux
dm02db03: Linux dm02db03.hrss 2.6.18-274.18.1.0.1.el5 #1 SMP Thu Feb 9 19:07:16 EST 2012 x86_64 x86_64 x86_64 GNU/Linux
dm02db04: Linux dm02db04.hrss 2.6.18-274.18.1.0.1.el5 #1 SMP Thu Feb 9 19:07:16 EST 2012 x86_64 x86_64 x86_64 GNU/Linux
dm02db05: Linux dm02db05.hrss 2.6.18-274.18.1.0.1.el5 #1 SMP Thu Feb 9 19:07:16 EST 2012 x86_64 x86_64 x86_64 GNU/Linux
dm02db06: Linux dm02db06.hrss 2.6.18-274.18.1.0.1.el5 #1 SMP Thu Feb 9 19:07:16 EST 2012 x86_64 x86_64 x86_64 GNU/Linux
dm02db07: Linux dm02db07.hrss 2.6.18-274.18.1.0.1.el5 #1 SMP Thu Feb 9 19:07:16 EST 2012 x86_64 x86_64 x86_64 GNU/Linux
dm02db08: Linux dm02db08.hrss 2.6.18-274.18.1.0.1.el5 #1 SMP Thu Feb 9 19:07:16 EST 2012 x86_64 x86_64 x86_64 GNU/Linux
dm02cel01: Linux dm02cel01.hrss 2.6.18-274.18.1.0.1.el5 #1 SMP Thu Feb 9 19:07:16 EST 2012 x86_64 x86_64 x86_64 GNU/Linux
dm02cel02: Linux dm02cel02.hrss 2.6.18-274.18.1.0.1.el5 #1 SMP Thu Feb 9 19:07:16 EST 2012 x86_64 x86_64 x86_64 GNU/Linux
dm02cel03: Linux dm02cel03.hrss 2.6.18-274.18.1.0.1.el5 #1 SMP Thu Feb 9 19:07:16 EST 2012 x86_64 x86_64 x86_64 GNU/Linux
dm02cel04: Linux dm02cel04.hrss 2.6.18-274.18.1.0.1.el5 #1 SMP Thu Feb 9 19:07:16 EST 2012 x86_64 x86_64 x86_64 GNU/Linux
dm02cel05: Linux dm02cel05.hrss 2.6.18-274.18.1.0.1.el5 #1 SMP Thu Feb 9 19:07:16 EST 2012 x86_64 x86_64 x86_64 GNU/Linux
dm02cel06: Linux dm02cel06.hrss 2.6.18-274.18.1.0.1.el5 #1 SMP Thu Feb 9 19:07:16 EST 2012 x86_64 x86_64 x86_64 GNU/Linux
dm02cel07: Linux dm02cel07.hrss 2.6.18-274.18.1.0.1.el5 #1 SMP Thu Feb 9 19:07:16 EST 2012 x86_64 x86_64 x86_64 GNU/Linux
dm02cel08: Linux dm02cel08.hrss 2.6.18-274.18.1.0.1.el5 #1 SMP Thu Feb 9 19:07:16 EST 2012 x86_64 x86_64 x86_64 GNU/Linux
dm02cel09: Linux dm02cel09.hrss 2.6.18-274.18.1.0.1.el5 #1 SMP Thu Feb 9 19:07:16 EST 2012 x86_64 x86_64 x86_64 GNU/Linux
dm02cel10: Linux dm02cel10.hrss 2.6.18-274.18.1.0.1.el5 #1 SMP Thu Feb 9 19:07:16 EST 2012 x86_64 x86_64 x86_64 GNU/Linux
dm02cel11: Linux dm02cel11.hrss 2.6.18-274.18.1.0.1.el5 #1 SMP Thu Feb 9 19:07:16 EST 2012 x86_64 x86_64 x86_64 GNU/Linux
dm02cel12: Linux dm02cel12.hrss 2.6.18-274.18.1.0.1.el5 #1 SMP Thu Feb 9 19:07:16 EST 2012 x86_64 x86_64 x86_64 GNU/Linux
dm02cel13: Linux dm02cel13.hrss 2.6.18-274.18.1.0.1.el5 #1 SMP Thu Feb 9 19:07:16 EST 2012 x86_64 x86_64 x86_64 GNU/Linux
dm02cel14: Linux dm02cel14.hrss 2.6.18-274.18.1.0.1.el5 #1 SMP Thu Feb 9 19:07:16 EST 2012 x86_64 x86_64 x86_64 GNU/Linux
[root@dm02db01 patch_11.2.3.2.1]#

6、检查操作系统版本

[root@dm02db01 patch_11.2.3.2.1]# dcli -g all_group -l root 'cat /etc/oracle-release'
dm02db01: Oracle Linux Server release 5.7
dm02db02: Oracle Linux Server release 5.7
。。。。。。。。。
dm02cel01: Oracle Linux Server release 5.7
dm02cel02: Oracle Linux Server release 5.7
。。。。。。。。。。。。。。
dm02cel14: Oracle Linux Server release 5.7
[root@dm02db01 patch_11.2.3.2.1]#

7、检查image版本

[root@dm02db01 patch_11.2.3.2.1]# dcli -g all_group -l root 'imageinfo'
dm02db01: 
dm02db01: Kernel version: 2.6.18-274.18.1.0.1.el5 #1 SMP Thu Feb 9 19:07:16 EST 2012 x86_64
dm02db01: Image version: 11.2.3.1.1.120607
dm02db01: Image activated: 2012-08-08 15:03:41 +0800
dm02db01: Image status: success
dm02db01: System partition on device: /dev/mapper/VGExaDb-LVDbSys1
dm02db01: 
dm02db02: 
dm02db02: Kernel version: 2.6.18-274.18.1.0.1.el5 #1 SMP Thu Feb 9 19:07:16 EST 2012 x86_64
dm02db02: Image version: 11.2.3.1.1.120607
dm02db02: Image activated: 2012-08-08 13:04:27 +0800
dm02db02: Image status: success
dm02db02: System partition on device: /dev/mapper/VGExaDb-LVDbSys1
dm02db02: 
dm02db03: 
………. 
dm02cel01: 
dm02cel01: Kernel version: 2.6.18-274.18.1.0.1.el5 #1 SMP Thu Feb 9 19:07:16 EST 2012 x86_64
dm02cel01: Cell rpm version: cell-11.2.3.1.1_LINUX.X64_120607-1
dm02cel01: 
dm02cel01: Active image version: 11.2.3.1.1.120607
dm02cel01: Active image activated: 2012-08-08 18:40:45 +0800
dm02cel01: Active image status: success
dm02cel01: Active system partition on device: /dev/md5
dm02cel01: Active software partition on device: /dev/md7
dm02cel01: 
dm02cel01: In partition rollback: Impossible
dm02cel01: 
dm02cel01: Cell boot usb partition: /dev/sdm1
dm02cel01: Cell boot usb version: 11.2.3.1.1.120607
dm02cel01: 
dm02cel01: Inactive image version: undefined
dm02cel01: Rollback to the inactive partitions: Impossible
dm02cel02: 
…….
dm02cel14: 
dm02cel14: Kernel version: 2.6.18-274.18.1.0.1.el5 #1 SMP Thu Feb 9 19:07:16 EST 2012 x86_64
dm02cel14: Cell rpm version: cell-11.2.3.1.1_LINUX.X64_120607-1
dm02cel14: 
dm02cel14: Active image version: 11.2.3.1.1.120607
dm02cel14: Active image activated: 2012-08-08 14:13:24 +0800
dm02cel14: Active image status: success
dm02cel14: Active system partition on device: /dev/md5
dm02cel14: Active software partition on device: /dev/md7
dm02cel14: 
dm02cel14: In partition rollback: Impossible
dm02cel14: 
dm02cel14: Cell boot usb partition: /dev/sdm1
dm02cel14: Cell boot usb version: 11.2.3.1.1.120607
dm02cel14: 
dm02cel14: Inactive image version: undefined
dm02cel14: Rollback to the inactive partitions: Impossible
[root@dm02db01 patch_11.2.3.2.1]#

[root@dm02db01 patch_11.2.3.2.1]# dcli -g all_group -l root 'imagehistory'
dm02db01: Version                              : 11.2.3.1.1.120607
dm02db01: Image activation date                : 2012-08-08 15:03:41 +0800
dm02db01: Imaging mode                         : fresh
dm02db01: Imaging status                       : success
dm02db01: 
dm02db02: Version                              : 11.2.3.1.1.120607
dm02db02: Image activation date                : 2012-08-08 13:04:27 +0800
dm02db02: Imaging mode                         : fresh
dm02db02: Imaging status                       : success
…………..
dm02cel01: Version                              : 11.2.3.1.1.120607
dm02cel01: Image activation date                : 2012-08-08 18:40:45 +0800
dm02cel01: Imaging mode                         : fresh
dm02cel01: Imaging status                       : success
dm02cel01: 
dm02cel02: Version                              : 11.2.3.1.1.120607
dm02cel02: Image activation date                : 2012-08-08 18:40:44 +0800
dm02cel02: Imaging mode                         : fresh
dm02cel02: Imaging status                       : success
dm02cel02:
。。。。。。。。。。。

8、检查ofa版本

[root@dm02db01 patch_11.2.3.2.1]# dcli -g all_group -l root 'rpm -qa | grep ofa'
dm02db01: ofa-2.6.18-274.18.1.0.1.el5-1.5.1-4.0.58
dm02db02: ofa-2.6.18-274.18.1.0.1.el5-1.5.1-4.0.58
。。。。。。。。。。。。
dm02cel01: ofa-2.6.18-274.18.1.0.1.el5-1.5.1-4.0.58
dm02cel02: ofa-2.6.18-274.18.1.0.1.el5-1.5.1-4.0.58
。。。。。。。。。。。。。。。。。。
dm02cel14: ofa-2.6.18-274.18.1.0.1.el5-1.5.1-4.0.58
[root@dm02db01 patch_11.2.3.2.1]#

9、检测硬件设备类型：

[root@dm02db01 patch_11.2.3.2.1]# dcli -g all_group -l root 'dmidecode -s system-product-name'
dm02db01: SUN FIRE X4170 M2 SERVER
dm02db02: SUN FIRE X4170 M2 SERVER
。。。。。
dm02cel01: SUN FIRE X4270 M2 SERVER
dm02cel02: SUN FIRE X4270 M2 SERVER
……
dm02cel14: SUN FIRE X4270 M2 SERVER
[root@dm02db01 patch_11.2.3.2.1]#

10、检查cell节点日志信息

[root@dm02db01 patch_11.2.3.2.1]# dcli -g cell_group -l root "cellcli -e list alerthistory"
。。。。。。。。。

如果有严重告警等信息，需要解决问题后才能升级。

11、检测是否存在offline状态的grid盘

[root@dm02db01 patch_11.2.3.2.1]# dcli -g cell_group -l root "cellcli -e "LIST GRIDDISK ATTRIBUTES name WHERE asmdeactivationoutcome != 'Yes'" "
[root@dm02db01 patch_11.2.3.2.1]#

12、确认infiniband 交换机firmware已经是最新的1.3.3.2
参考MOS NOTES 888828.1

13、验证cell节点网络配置信息与cell.conf保持一致

[root@dm02db01 ~]# dcli -g cell_group -l root /opt/oracle.cellos/ipconf -verify
dm02cel01: Verifying of Exadata configuration file /opt/oracle.cellos/cell.conf
dm02cel01: Done. Configuration file /opt/oracle.cellos/cell.conf passed all verification checks
dm02cel02: Verifying of Exadata configuration file /opt/oracle.cellos/cell.conf
dm02cel02: Done. Configuration file /opt/oracle.cellos/cell.conf passed all verification checks
。。。。。。。。。。。
dm02cel14: Verifying of Exadata configuration file /opt/oracle.cellos/cell.conf
dm02cel14: Done. Configuration file /opt/oracle.cellos/cell.conf passed all verification checks
[root@dm02db01 ~]#

二、升级CELL节点的image
升级和回滚CELL节点的image可以使用滚动和滚动的方式：

滚动升级可以实现零宕机时间，如果出问题也只是影响出问题的一个cell节点。缺点是升级时间比较长，要一台一台做，比如一个CELL升级大概需要2小时，那么一个满配的Exadata就需要2*14=28小时，比较熬人…………

如果客户可以配置一个应急库（比如使用ADG），那么我一般都推荐客户使用非滚动方式，这样，一个满配的Exadata升级一遍所有的CELL也就是两个多小时（含前期准备和检查的时间）。

为了节省时间，本次推荐客户采用非滚动方式（配置ADG的过程这里就不赘述了）。

1、停止所有的数据库

srvctl stop database -d your_database_name

2、停止所有节点的crs：

dcli -g dbs_group -l root "/u01/app/11.2.0/grid/bin/crsctl stop crs -f"

3、检查是否还有grid用户进程：

dcli -g dbs_group -l root "ps -ef | grep grid"

4、停止所有cell服务(cellsrv, ms, rs)：

dcli -g cell_group -l root "cellcli -e alter cell shutdown services all"

5、用户root以ssh方式登陆db01节点,不能通过KVM和串口会话进行升级

# echo $consoletype

6、解压cell image的介质

# unzip p14522699_112321_Linux-x86-64.zip

7、清理之前patchmgr运行后的环境，类似AIX环境打patch时执行的命令（具体叫啥我忘记了，readme上有……）

[root@dm02db01 patch_11.2.3.2.1.130109]# ./patchmgr -cells cell_group -cleanup
Linux dm02db01.hrss 2.6.18-274.18.1.0.1.el5 #1 SMP Thu Feb 9 19:07:16 EST 2012 x86_64 x86_64 x86_64 GNU/Linux

2013-05-23 11:40:18 :DONE: Cleanup

[root@dm02db01 patch_11.2.3.2.1.130109]#

8、预安装检查：

[root@dm02db01 patch_11.2.3.2.1.130109]# ./patchmgr -cells cell_group -patch_check_prereq
Linux dm02db01.hrss 2.6.18-274.18.1.0.1.el5 #1 SMP Thu Feb 9 19:07:16 EST 2012 x86_64 x86_64 x86_64 GNU/Linux

2013-05-23 11:40:44        :Working: DO: Check cells have ssh equivalence for root user. Up to 10 seconds per cell ...
2013-05-23 11:41:00        :SUCCESS: DONE: Check cells have ssh equivalence for root user.
2013-05-23 11:41:00        :Working: DO: Check space and state of Cell services on target cells. Up to 1 minute ...
2013-05-23 11:41:46        :SUCCESS: DONE: Check space and state of Cell services on target cells.
2013-05-23 11:41:46        :Working: DO: Copy, extract prerequisite check archive to cells. If required start md11 mismatched partner size correction. Up to 40 minutes ...
2013-05-23 11:42:09 Wait correction of degraded md11 due to md partner size mismatch. Up to 30 minutes.

                                          
2013-05-23 11:42:25        :SUCCESS: DONE: Copy, extract prerequisite check archive to cells. If required start md11 mismatched partner size correction.
2013-05-23 11:42:25        :Working: DO: Check prerequisites on all cells. Up to 2 minutes ...
2013-05-23 11:43:05        :SUCCESS: DONE: Check prerequisites on all cells.

[root@dm02db01 patch_11.2.3.2.1.130109]#

9、非滚动升级
我一般是开启一个vncserver在后台执行，如果网络稳定的话，可以开一个直接在crt中运行：
本次没看日志，闭眼升级不回滚，o(∩_∩)o 哈哈 *^_^*

下面的是以前的执行过程中的记录信息：

[root@dm01db01 patch_11.2.3.2.1.130109]# ./patchmgr -cells cell_group -patch
Linux dm01db01.scq.com 2.6.32-400.1.1.el5uek #1 SMP Mon Jun 25 20:25:08 EDT 2012 x86_64 x86_64 x86_64 GNU/Linux
NOTE Cells will reboot during the patch or rollback process.
NOTE For non-rolling patch or rollback, ensure all ASM instances using
NOTE the cells are shut down for the duration of the patch or rollback.
NOTE For rolling patch or rollback, ensure all ASM instances using
。。。。。。。。。。。。。

下面的是以前的升级过程中的信息：

[root@dm01db01 patch_11.2.3.2.1.130109]# less -rf patchmgr.stdout 

================PatchMgr run started Tue Apr  2 12:42:39 CST 2013 ===========
2013-04-02 12:42:40 :DONE: Cleanup
================PatchMgr run ended Tue Apr  2 12:42:40 CST 2013 ===========


================PatchMgr run started Tue Apr  2 12:42:51 CST 2013 ===========
2013-04-02 12:42:52 :DONE: Cleanup
================PatchMgr run ended Tue Apr  2 12:42:52 CST 2013 ===========


================PatchMgr run started Tue Apr  2 12:43:01 CST 2013 ===========
2013-04-02 12:43:01        :Working: DO: Check cells have ssh equivalence for root user. Up to 10 seconds per cell ...
2013-04-02 12:43:01 ++++++++++++++++++ Logs so far follow ++++++++++
2013-04-02 12:43:02        :SUCCESS: DONE: Check cells have ssh equivalence for root user.
2013-04-02 12:43:02        :Working: DO: Check space and state of Cell services on target cells. Up to 1 minute ...
2013-04-02 12:43:23 ++++++++++++++++++ Logs so far follow ++++++++++
dm01cel01: [INFO] 2013-04-02 12:43:05 patchmgr launch attempt from dm01db01.scq.com_9.9.10.1_tmp_patch_11.2.3.2.1.130109.
dm01cel01: [INFO] 2013-04-02 12:43:05 dostep called: prechk:1457 patch_prereq -no-auto-rollback non_rolling 600 3600 900 3600 default noforce
dm01cel01: [INFO] 2013-04-02 12:43:05 patchmgr launched from dm01db01.scq.com_9.9.10.1_tmp_patch_11.2.3.2.1.130109
dm01cel01: [DEBUG] BEGIN Various markers 2013-04-02 12:43:23
dm01cel01: -rw-r--r-- 1 root root 17 Feb 13 16:37 /.hwfwchk_save.post.imaging
dm01cel01: [DEBUG] End Various markers
dm01cel01: _EXIT_PASS_Cell dm01cel01 9.9.10.3 2013-04-02 12:43:23:
dm01cel02: [INFO] 2013-04-02 12:43:05 patchmgr launch attempt from dm01db01.scq.com_9.9.10.1_tmp_patch_11.2.3.2.1.130109.
。。。。。。。。。
2013-04-02 12:48:29 2 of 5 :Working: DO: Waiting to finish pre-reboot patch actions. Cells will remain up. Up to 45 minutes ...
2013-04-02 12:49:29 Wait for patch pre-reboot procedures

||||| Minutes left 045///// Minutes left 045----- Minutes left 045\\\\\ Minutes left 045||||| Minutes left 045||||| Minutes left 045///// Minutes left 045----- Minutes left 045\\\\\ Minutes left 045||||| Minutes left 045||||| Minutes left 045///// Minutes left 045----- Minutes left 045\\\\\ Minutes left 045||||| Minutes left 045||||| Minutes left 045///// Minutes left 045----- Minutes left 045\\\\\ Minutes left 045||||| Minutes left 045||||| Minutes left 045///// Minutes left 045----- Minutes left 045\\\\\ Minutes left 045||||| Minutes left 045||||| Minutes left 045///// Minutes left 045----- Minutes left 045\\\\\ Minutes left 045||||| Minutes left 045||||| Minutes left 045///// Minutes left 045----- Minutes left 045\\\\\ Minutes left 045||||| Minutes left 045||||| Minutes left 045///// Minutes left 045----- Minutes left 045\\\\\ Minutes left 045||||| Minutes left 045||||| Minutes left 045///// Minutes left 045----- Minutes left 045\\\\\ Minutes left 045||||| Minutes left 045||||| Minutes left 045……….

这个界面如果使用vncserver来执行，就会不会有刷屏的效果，这个看下升级的脚本就会知道。
升级过程中，每个节点都会有相应的log：

[root@dm02db01 patch_11.2.3.2.1.130109]# more dm02cel07.log
dm02cel07: [INFO] 2013-05-23 11:41:23 patchmgr launch attempt from dm02db01.lunar_152.26.6.11_tmp_patch_11.2.3.2.1_cell_ima
ge_patch_11.2.3.2.1.130109.
dm02cel07: [INFO] 2013-05-23 11:41:23 dostep called: prechk:1457 patch_prereq -no-auto-rollback non_rolling 600 3600 900 3
600 default noforce
dm02cel07: [INFO] 2013-05-23 11:41:23 patchmgr launched from dm02db01.lunar_152.26.6.11_tmp_patch_11.2.3.2.1_cell_image_pat
ch_11.2.3.2.1.130109
dm02cel07: [DEBUG] BEGIN Various markers 2013-05-23 11:41:37
dm02cel07: -rw-r--r-- 1 root root 20 Aug  8  2012 /.hwfwchk_save.post.imaging
dm02cel07: [DEBUG] End Various markers
dm02cel07: _EXIT_PASS_Cell dm02cel07 172.20.6.25 2013-05-23 11:41:37:
dm02cel07: 
dm02cel07: _EXIT_PASS_Cell dm02cel07 172.20.6.25
。。。。。。。。。。。。

升级过程中，可以使用 less -rf patchmgr.stdout 命令进行后台监控，还可以在cell上使用 tail -f /root/_patch_hctap_/_p_/wait_out 监控每一个cell的patch情况。

三、升级计算节点image
1、备份每个节点的GI和DB软件（也可以使用MOS上的dbserver_backup.sh，该脚本使用使用LVM快照进行系统备份）

2、检查磁盘空间

[root@dm02db01 db_image]# dcli -g dbs_group -l root 'df -h /boot'
dm02db01: Filesystem            Size  Used Avail Use% Mounted on
dm02db01: /dev/sda1             502M   30M  447M   7% /boot
dm02db02: Filesystem            Size  Used Avail Use% Mounted on
dm02db02: /dev/sda1             502M   30M  447M   7% /boot
dm02db03: Filesystem            Size  Used Avail Use% Mounted on
dm02db03: /dev/sda1             502M   30M  447M   7% /boot
dm02db04: Filesystem            Size  Used Avail Use% Mounted on
dm02db04: /dev/sda1             502M   30M  447M   7% /boot
dm02db05: Filesystem            Size  Used Avail Use% Mounted on
dm02db05: /dev/sda1             502M   30M  447M   7% /boot
dm02db06: Filesystem            Size  Used Avail Use% Mounted on
dm02db06: /dev/sda1             502M   30M  447M   7% /boot
dm02db07: Filesystem            Size  Used Avail Use% Mounted on
dm02db07: /dev/sda1             502M   30M  447M   7% /boot
dm02db08: Filesystem            Size  Used Avail Use% Mounted on
dm02db08: /dev/sda1             502M   30M  447M   7% /boot
[root@dm02db01 db_image]#

3、清除yum缓存：

[root@dm02db01 db_image]# dcli -g dbs_group -l root 'yum clean all'
dm02db01: Cleaning up Everything
dm02db02: Cleaning up Everything
dm02db03: Cleaning up Everything
dm02db04: Cleaning up Everything
dm02db05: Cleaning up Everything
dm02db06: Cleaning up Everything
dm02db07: Cleaning up Everything
dm02db08: Cleaning up Everything
[root@dm02db01 db_image]#

4、解压介质

[root@dm02db01 db_image]# unzip p16432033_112321_Linux-x86-64.zip

5、配置yum源

[root@dm02db01 patch_11.2.3.2.1]# dcli -g dbs_group -l root "mkdir -p /mnt/iso/yum/unknown/EXADATA/dbserver/11.2/latest"

在所有db节点的/etc/yum.repos.d/Exadata-computenode.repo中添加：

[exadata_dbserver_11.2_x86_64_latest]
name=Oracle Exadata DB server 11.2 Linux $releasever - $basearch - latest
baseurl=file:///mnt/iso/yum/unknown/EXADATA/dbserver/11.2/latest/x86_64
gpgcheck=1
enabled=0

挂载yum源：

[root@dm02db01 db_image]# dcli -g dbs_group -l root "mount -o loop /tmp/patch_11.2.3.2.1/db_image/112_latest_repo_130302.iso /mnt/iso/yum/unknown/EXADATA/dbserver/11.2/latest"


[root@dm02db01 db_image]# dcli -g dbs_group -l root 'ls -lrt /mnt/iso/yum/unknown/EXADATA/dbserver/11.2/latest'
dm02db01: total 78
dm02db01: dr-xr-xr-x 5 root root 79872 Mar  5 22:32 x86_64
dm02db02: total 78
dm02db02: dr-xr-xr-x 5 root root 79872 Mar  5 22:32 x86_64
dm02db03: total 78
dm02db03: dr-xr-xr-x 5 root root 79872 Mar  5 22:32 x86_64
dm02db04: total 78
dm02db04: dr-xr-xr-x 5 root root 79872 Mar  5 22:32 x86_64
dm02db05: total 78
dm02db05: dr-xr-xr-x 5 root root 79872 Mar  5 22:32 x86_64
dm02db06: total 78
dm02db06: dr-xr-xr-x 5 root root 79872 Mar  5 22:32 x86_64
dm02db07: total 78
dm02db07: dr-xr-xr-x 5 root root 79872 Mar  5 22:32 x86_64
dm02db08: total 78
dm02db08: dr-xr-xr-x 5 root root 79872 Mar  5 22:32 x86_64
[root@dm02db01 db_image]#

升级计算节点：
这个过程会线进行一些列检查，然后更新相关的动态库：

[root@dm02db01 ~]# yum --enablerepo=exadata_dbserver_11.2_x86_64_latest update
exadata_dbserver_11.2_x86_64_latest                                                                | 1.9 kB     00:00     
Excluding Packages in global exclude list
Finished
Setting up Update Process
Resolving Dependencies
--> Running transaction check
---> Package OpenIPMI.x86_64 0:2.0.16-13.el5_8 set to be updated
---> Package OpenIPMI-libs.x86_64 0:2.0.16-13.el5_8 set to be updated
---> Package acl.x86_64 0:2.2.39-8.el5 set to be updated
---> Package audit.x86_64 0:1.8-2.el5 set to be updated
---> Package audit-libs.x86_64 0:1.8-2.el5 set to be updated
---> Package audit-libs-python.x86_64 0:1.8-2.el5 set to be updated
---> Package bind-libs.x86_64 30:9.3.6-20.P1.el5_8.5 set to be updated
---> Package bind-utils.x86_64 30:9.3.6-20.P1.el5_8.5 set to be updated
---> Package binutils.x86_64 0:2.17.50.0.6-20.el5_8.3 set to be updated
---> Package busybox.x86_64 1:1.2.0-13.el5 set to be updated

…………………………
Replaced:
  kernel-headers.x86_64 0:2.6.18-274.18.1.0.1.el5                                                                         

Complete!
[root@dm02db01 ~]# 
Remote broadcast message (Thu May 23 13:55:11 2013):

Exadata post install steps started.
It may take up to 2 minutes.
The db node will be rebooted upon successful completion.

Update的过程大概不足10分钟，之后，系统会自动重启N次，大概30~40分钟后，升级完成，系统最后一次启动会正常启动（重启N次的过程中，可以通过ILOM观察到系统的N次重启活动）。

所有的节点可以并行做。

四、给GI 和 DB 打 BP 16
1，使用最新的或者相应的OPatch：
本次使用 p6880880_112000_Linux-x86-64.zip

2、解压OPatch(grid和oracle都要)：

[grid@dm02db01 bp16]$ unzip p6880880_112000_Linux-x86-64.zip -d /u01/app/11.2.0.3/grid/
Archive:  p6880880_112000_Linux-x86-64.zip
   creating: /u01/app/11.2.0.3/grid/OPatch/oplan/
  inflating: /u01/app/11.2.0.3/grid/OPatch/oplan/README.html  
  inflating: /u01/app/11.2.0.3/grid/OPatch/oplan/README.txt  
   creating: /u01/app/11.2.0.3/grid/OPatch/oplan/jlib/
  inflating: /u01/app/11.2.0.3/grid/OPatch/oplan/jlib/oplan.jar  
  inflating: /u01/app/11.2.0.3/grid/OPatch/oplan/jlib/oracle.oplan.classpath.jar  
  inflating: /u01/app/11.2.0.3/grid/OPatch/oplan/jlib/automation.jar  
  inflating: /u01/app/11.2.0.3/grid/OPatch/oplan/jlib/OsysModel.jar  
  inflating: /u01/app/11.2.0.3/grid/OPatch/oplan/jlib/EMrepoDrivers.jar  
  inflating: /u01/app/11.2.0.3/grid/OPatch/oplan/jlib/Validation.jar  
  inflating: /u01/app/11.2.0.3/grid/OPatch/oplan/jlib/ValidationRules.jar  
  inflating: /u01/app/11.2.0.3/grid/OPatch/oplan/jlib/osysmodel-utils.jar  
  inflating: /u01/app/11.2.0.3/grid/OPatch/oplan/jlib/CRSProductDriver.jar  
   creating: /u01/app/11.2.0.3/grid/OPatch/oplan/jlib/apache-commons/
  inflating: /u01/app/11.2.0.3/grid/OPatch/oplan/jlib/apache-commons/commons-cli-1.0.jar  
   creating: /u01/app/11.2.0.3/grid/OPatch/oplan/jlib/jaxb/
  inflating: /u01/app/11.2.0.3/grid/OPatch/oplan/jlib/jaxb/activation.jar  
  inflating: /u01/app/11.2.0.3/grid/OPatch/oplan/jlib/jaxb/jaxb-api.jar  
  inflating: /u01/app/11.2.0.3/grid/OPatch/oplan/jlib/jaxb/jaxb-impl.jar  
  inflating: /u01/app/11.2.0.3/grid/OPatch/oplan/jlib/jaxb/jsr173_1.0_api.jar  
  inflating: /u01/app/11.2.0.3/grid/OPatch/oplan/oplan  
   creating: /u01/app/11.2.0.3/grid/OPatch/docs/
  inflating: /u01/app/11.2.0.3/grid/OPatch/docs/FAQ  
  inflating: /u01/app/11.2.0.3/grid/OPatch/docs/Users_Guide.txt  
  inflating: /u01/app/11.2.0.3/grid/OPatch/docs/Prereq_Users_Guide.txt  
   creating: /u01/app/11.2.0.3/grid/OPatch/jlib/
   creating: /u01/app/11.2.0.3/grid/OPatch/jlib/fa/
  inflating: /u01/app/11.2.0.3/grid/OPatch/jlib/fa/oracle.opatch.fa.classpath.jar  
  inflating: /u01/app/11.2.0.3/grid/OPatch/jlib/fa/oracle.opatch.fa.classpath.unix.jar  
  inflating: /u01/app/11.2.0.3/grid/OPatch/jlib/fa/oracle.opatch.fa.classpath.windows.jar  
  inflating: /u01/app/11.2.0.3/grid/OPatch/jlib/opatch.jar  
  inflating: /u01/app/11.2.0.3/grid/OPatch/jlib/opatchsdk.jar  
  inflating: /u01/app/11.2.0.3/grid/OPatch/jlib/oracle.opatch.classpath.jar  
  inflating: /u01/app/11.2.0.3/grid/OPatch/jlib/oracle.opatch.classpath.unix.jar  
  inflating: /u01/app/11.2.0.3/grid/OPatch/jlib/oracle.opatch.classpath.windows.jar  
   creating: /u01/app/11.2.0.3/grid/OPatch/opatchprereqs/
   creating: /u01/app/11.2.0.3/grid/OPatch/opatchprereqs/opatch/
  inflating: /u01/app/11.2.0.3/grid/OPatch/opatchprereqs/opatch/opatch_prereq.xml  
  inflating: /u01/app/11.2.0.3/grid/OPatch/opatchprereqs/opatch/rulemap.xml  
  inflating: /u01/app/11.2.0.3/grid/OPatch/opatchprereqs/opatch/runtime_prereq.xml  
   creating: /u01/app/11.2.0.3/grid/OPatch/opatchprereqs/oui/
  inflating: /u01/app/11.2.0.3/grid/OPatch/opatchprereqs/oui/knowledgesrc.xml  
  inflating: /u01/app/11.2.0.3/grid/OPatch/opatchprereqs/prerequisite.properties  
   creating: /u01/app/11.2.0.3/grid/OPatch/crs/
   creating: /u01/app/11.2.0.3/grid/OPatch/crs/log/
  inflating: /u01/app/11.2.0.3/grid/OPatch/crs/auto_patch.pl  
  inflating: /u01/app/11.2.0.3/grid/OPatch/crs/installPatch.excl  
  inflating: /u01/app/11.2.0.3/grid/OPatch/crs/patch112.pl  
  inflating: /u01/app/11.2.0.3/grid/OPatch/crs/patch11202.pl  
  inflating: /u01/app/11.2.0.3/grid/OPatch/crs/patch11203.pl  
  inflating: /u01/app/11.2.0.3/grid/OPatch/opatch  
  inflating: /u01/app/11.2.0.3/grid/OPatch/opatch.bat  
  inflating: /u01/app/11.2.0.3/grid/OPatch/opatch.pl  
  inflating: /u01/app/11.2.0.3/grid/OPatch/opatch.ini  
  inflating: /u01/app/11.2.0.3/grid/OPatch/opatchdiag  
  inflating: /u01/app/11.2.0.3/grid/OPatch/opatchdiag.bat  
  inflating: /u01/app/11.2.0.3/grid/OPatch/emdpatch.pl  
  inflating: /u01/app/11.2.0.3/grid/OPatch/README.txt  
   creating: /u01/app/11.2.0.3/grid/OPatch/ocm/
   creating: /u01/app/11.2.0.3/grid/OPatch/ocm/bin/
  inflating: /u01/app/11.2.0.3/grid/OPatch/ocm/bin/emocmrsp  
   creating: /u01/app/11.2.0.3/grid/OPatch/ocm/doc/
   creating: /u01/app/11.2.0.3/grid/OPatch/ocm/lib/
  inflating: /u01/app/11.2.0.3/grid/OPatch/ocm/lib/emocmclnt-14.jar  
  inflating: /u01/app/11.2.0.3/grid/OPatch/ocm/lib/emocmclnt.jar  
  inflating: /u01/app/11.2.0.3/grid/OPatch/ocm/lib/emocmcommon.jar  
  inflating: /u01/app/11.2.0.3/grid/OPatch/ocm/lib/http_client.jar  
  inflating: /u01/app/11.2.0.3/grid/OPatch/ocm/lib/jcert.jar  
  inflating: /u01/app/11.2.0.3/grid/OPatch/ocm/lib/jnet.jar  
  inflating: /u01/app/11.2.0.3/grid/OPatch/ocm/lib/jsse.jar  
  inflating: /u01/app/11.2.0.3/grid/OPatch/ocm/lib/log4j-core.jar  
  inflating: /u01/app/11.2.0.3/grid/OPatch/ocm/lib/osdt_core3.jar  
  inflating: /u01/app/11.2.0.3/grid/OPatch/ocm/lib/osdt_jce.jar  
  inflating: /u01/app/11.2.0.3/grid/OPatch/ocm/lib/regexp.jar  
  inflating: /u01/app/11.2.0.3/grid/OPatch/ocm/lib/xmlparserv2.jar  
 extracting: /u01/app/11.2.0.3/grid/OPatch/ocm/ocm.zip  
  inflating: /u01/app/11.2.0.3/grid/OPatch/ocm/ocm_platforms.txt  
[grid@dm02db01 bp16]$

2、配置OCM

[grid@dm02db01 bp16]$ORACLE_HOME/OPatch/ocm/bin/emocmrsp

3、分别使用oracle和grid都检查一下OPatch的版本是否正确：

$ORACLE_HOME/OPatch/opatch version

4、分别使用oracle和grid都检查一下当前patch信息

$ORACLE_HOME/OPatch/opatch lsinventory -detail -oh $ORACLE_HOME

5、For Grid Infrastructure Home, as home user:

$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /tmp/bp16/16233552/16233552
$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /tmp/bp16/16233552/16355082
$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /tmp/bp16/16233552/16401300

6、For Database home, as home user:

$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /tmp/bp16/16233552/16233552
$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /tmp/bp16/16233552/16355082/custom/server/16355082

7、For Grid Infrastructure Home, as home user:

$ORACLE_HOME/OPatch/opatch prereq CheckSystemSpace -phBaseDir /tmp/bp16/16233552/16233552
$ORACLE_HOME/OPatch/opatch prereq CheckSystemSpace -phBaseDir /tmp/bp16/16233552/16355082
$ORACLE_HOME/OPatch/opatch prereq CheckSystemSpace -phBaseDir /tmp/bp16/16233552/16401300

8、For Database home, as home user:

$ORACLE_HOME/OPatch/opatch prereq CheckSystemSpace -phBaseDir /tmp/bp16/16233552/16233552
$ORACLE_HOME/OPatch/opatch prereq CheckSystemSpace -phBaseDir /tmp/bp16/16233552/16355082/custom/server/16355082

9、使用auto打patch

[root@dm02db01 bp16]#ORACLE_HOME/OPatch/opatch auto /tmp/bp16/16233552

本次打patch的时候，遇到了3个节点的crs不能正常shutdown，导apply patch失败（是因为之前手工配置的network2服务使用的eth3网卡引起的……），采用auto对这3个节点进行了回滚，然后重新使用auto 打patch，最终完美解决，o(∩_∩)o 哈哈。

su - root
ifdown eth3
/u01/app/11.2.0.3/grid/OPatch/opatch auto /tmp/bp16/16233552 –rollback

发表在安装和升级 | 标签为 Exadata, 升级 | 2 条评论

站内搜索

Search for:
Oracle证书
分类目录
- ASM (30)
- Database (86)
  - backup&recovery (21)
  - expdp/impdp (5)
  - Installation and Deinstall (31)
  - network (7)
  - ORA-600 or ORA-7445 (6)
  - Performence Tuning (13)
  - troubleshoooting (2)
- Dataguard (7)
- EBS (3)
  - EBS系统管理 (1)
  - 安装、克隆、迁移 (2)
- Exadata (120)
  - FAQ (19)
  - POC和性能调整 (11)
  - 体系架构 (19)
  - 内部机制 (22)
  - 安装和升级 (14)
  - 性能指标 (8)
    - Exadata V1 (1)
    - Exadata V2 (1)
    - Exadata X2-2 (2)
    - Exadata X3-2 (1)
    - Exadata X4-2 (1)
    - FAQ (1)
  - 故障诊断 (3)
  - 日常运维 (15)
  - 硬件配置 (43)
    - Exadata V1 (6)
    - Exadata V2 (6)
    - Exadata X2-2 (6)
    - Exadata X3-2 (8)
    - Exadata X4-2 (8)
    - FAQ (1)
- FAQ (16)
- Internal (21)
  - bbed (2)
  - DUL ODU (4)
- Linux (20)
- MYSQL (8)
  - FAQ (7)
  - 复制 (1)
  - 安装配置 (1)
- OGG (1)
- ORA-600/7445 (2)
- ORA-XXXXX (5)
- Oracle 11.1 & Oracle11.2 (6)
- ORACLE 12C (21)
- Oracle 8 & Oracle 8i (1)
- RAC (47)
- SAP (2)
- Scripts (6)
- 未分类 (1)
- 虚拟化 (1)
2024 年五月

S M T W T F S

« Nov

1 2 3 4

5 6 7 8 9 10 11

12 13 14 15 16 17 18

19 20 21 22 23 24 25

26 27 28 29 30 31
文章归档
文章归档
近期文章
近期评论
- tom 发表在《exadata巡检报告的模板》
- cyx 发表在《关于我》
- 李科胜发表在《EBS克隆–db和app分开在两个服务器上》
- xiao 发表在《exadata巡检报告的模板》
- Chris Sun 发表在《使用Oracle 11.2的DBMS_RESOURCE_MANAGER.CALIBRATE_IO对Exadata X5（HC）进行测试》

站内搜索

Oracle证书

分类目录

文章归档

近期文章

近期评论