使用Exadata的calibrate命令测试X5-2的IOPS和MBPS

联系:QQ(5163721)

标题:使用Exadata的calibrate命令测试X5-2的IOPS和MBPS

作者:Lunar©版权所有[文章允许转载,但必须以链接方式注明源地址,否则追究法律责任.]

之前测试的X3的flashcard的IOPS大概是满配200w左右(具体参见 《Exadata上的IOPS和MBPS》)
使用Oracle 11.2的DBMS_RESOURCE_MANAGER.CALIBRATE_IO对Exadata X5(HC)进行测试

使用Exadata的calibrate命令测试时需要注意:
1,这个不能再业务高峰期操作,否则可能造成应用hang等。
2,一般测试的时候,是把cellsrv进程宕掉,否则就要带-force参数。
3,在老板上上,如果这个命令报错,还有一种可能的原因,就是它需要生成一些临时的lun配置文件,然后自己格式化。
如果报错,可以手工自己删除这些临时的lun文件。


5


新版本目前没有发发现这类问题,因为Oracle已经修改了脚本,所有的文件生成目录都是带日期的(启动测试的日期)。
因此,不会因为不能格式化lun等信息报错,如图


1


4,Exadata的calibrate命令底层采用orion的方法,根据自己预定义的lun组合,然后压测N轮
(不同的lun组合,压测好多次,自动的过程,无需手工干预)
5,Exadata的calibrate采用只读方式(保护数据安全)
6,不能再一个cell上并发两个calibrate,否则会报错的。
.
本次测试的机器是Exadata X5-2:
CPU型号:

[root@dm01cel02 cellos]# grep -E '^model name|^cpu MHz' /proc/cpuinfo
cpu MHz         : 2401.000
model name      : Intel(R) Xeon(R) CPU E5-2630 v3 @ 2.40GHz
[root@dm01cel02 cellos]# 

[root@dm01db01 ibdiagtools]# grep -E '^model name|^cpu MHz' /proc/cpuinfo
cpu MHz         : 2301.000
model name      : Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz
[root@dm01db01 ibdiagtools]# 

主机型号:

[root@dm01db01 ibdiagtools]# /opt/oracle.SupportTools/CheckHWnFWProfile -c loose
[SUCCESS] The hardware and firmware matches supported profile for server=ORACLE_SERVER_X5-2
[root@dm01db01 ibdiagtools]# 

[root@dm01cel01 ~]# /opt/oracle.SupportTools/CheckHWnFWProfile -c loose
[SUCCESS] The hardware and firmware matches supported profile for server=ORACLE_SERVER_X5-2L
[root@dm01cel01 ~]# 

估算flash card的IOPS:

[root@dm01db01 lunar]# cat a.txt |grep FLASH
dm01cel01: LUN 1_1  on drive [FLASH_1_1] random read throughput: 3,227.00 MBPS, and 386910 IOPS
dm01cel01: LUN 2_1  on drive [FLASH_2_1] random read throughput: 3,226.00 MBPS, and 357090 IOPS
dm01cel01: LUN 4_1  on drive [FLASH_4_1] random read throughput: 3,227.00 MBPS, and 372007 IOPS
dm01cel01: LUN 5_1  on drive [FLASH_5_1] random read throughput: 3,226.00 MBPS, and 399469 IOPS
dm01cel02: LUN 1_1  on drive [FLASH_1_1] random read throughput: 3,226.00 MBPS, and 366062 IOPS
dm01cel02: LUN 2_1  on drive [FLASH_2_1] random read throughput: 3,226.00 MBPS, and 401361 IOPS
dm01cel02: LUN 4_1  on drive [FLASH_4_1] random read throughput: 3,226.00 MBPS, and 403004 IOPS
dm01cel02: LUN 5_1  on drive [FLASH_5_1] random read throughput: 3,226.00 MBPS, and 408830 IOPS
dm01cel03: LUN 1_1  on drive [FLASH_1_1] random read throughput: 3,226.00 MBPS, and 340622 IOPS
dm01cel03: LUN 2_1  on drive [FLASH_2_1] random read throughput: 3,226.00 MBPS, and 339174 IOPS
dm01cel03: LUN 4_1  on drive [FLASH_4_1] random read throughput: 3,227.00 MBPS, and 405781 IOPS
dm01cel03: LUN 5_1  on drive [FLASH_5_1] random read throughput: 3,226.00 MBPS, and 408817 IOPS
[root@dm01db01 lunar]#                                     

因此,理论上,这台X5-2 quarter rack的flash lun的IOPS总和超过 450万
.
而在之前测试的X3的flashcard的IOPS大概是满配200w左右(具体参见 《Exadata上的IOPS和MBPS》):
满配机器(14个cell)hard disk LUNs的MBPS推算: 1962*14个cell=27468
满配机器(14个cell)flash disk LUNs的MBPS推算: 4184*14个cell=58576
满配机器(14个cell)hard disk LUNs的IOPS推算: 4832*14个cell=67648
满配机器(14个cell)flash disk LUNs的IOPS推算:146413*14个cell=2049782
.
下面图是Oracle内部员工(凯耀)写的mon工具(底层基于开源的collect),在当前较新的版本上,貌似脚本还有点问题,我还没有调试。
因此这里只能看到磁盘的吞吐量,没有flash的。
磁盘的吞吐量最高大概3.5GB,CPU平均使用率在17%以下。


2


这个是执行过程中的一个瞬间的截图,可以看到这组LUN带的参数执行时,基本上是MBPS的较高值了:


3


这个图就是另外一组lun的orion,可以看到每个节点每次运行测试都是使用6个orion来压测的,这组lun显然MBPS较低:


4


具体测试如下:
测试时间,大概7分钟
测试方法,使用Exadata自带的calibrate命令:

[root@dm01db01 monitor]# dcli -g /opt/oracle.SupportTools/onecommand/cell_group -l root "cellcli -e calibrate force"
dm01cel01: Calibration will take a few minutes...
dm01cel01: Aggregate random read throughput across all hard disk LUNs: 1220 MBPS
dm01cel01: Aggregate random read throughput across all flash disk LUNs: 12568 MBPS
dm01cel01: Aggregate random read IOs per second (IOPS) across all hard disk LUNs: 2889
dm01cel01: Calibrating hard disks (read only) ...
dm01cel01: LUN 0_0  on drive [8:0      ] random read throughput: 147.00 MBPS, and 256 IOPS
dm01cel01: LUN 0_1  on drive [8:1      ] random read throughput: 142.00 MBPS, and 255 IOPS
dm01cel01: LUN 0_10 on drive [8:10     ] random read throughput: 148.00 MBPS, and 244 IOPS
dm01cel01: LUN 0_11 on drive [8:11     ] random read throughput: 145.00 MBPS, and 255 IOPS
dm01cel01: LUN 0_2  on drive [8:2      ] random read throughput: 145.00 MBPS, and 255 IOPS
dm01cel01: LUN 0_3  on drive [8:3      ] random read throughput: 146.00 MBPS, and 238 IOPS
dm01cel01: LUN 0_4  on drive [8:4      ] random read throughput: 146.00 MBPS, and 252 IOPS
dm01cel01: LUN 0_5  on drive [8:5      ] random read throughput: 146.00 MBPS, and 251 IOPS
dm01cel01: LUN 0_6  on drive [8:6      ] random read throughput: 143.00 MBPS, and 255 IOPS
dm01cel01: LUN 0_7  on drive [8:7      ] random read throughput: 150.00 MBPS, and 251 IOPS
dm01cel01: LUN 0_8  on drive [8:8      ] random read throughput: 151.00 MBPS, and 256 IOPS
dm01cel01: LUN 0_9  on drive [8:9      ] random read throughput: 145.00 MBPS, and 255 IOPS
dm01cel01: Calibrating flash disks (read only, note that writes will be significantly slower) ...
dm01cel01: LUN 1_1  on drive [FLASH_1_1] random read throughput: 3,227.00 MBPS, and 386910 IOPS
dm01cel01: LUN 2_1  on drive [FLASH_2_1] random read throughput: 3,226.00 MBPS, and 357090 IOPS
dm01cel01: LUN 4_1  on drive [FLASH_4_1] random read throughput: 3,227.00 MBPS, and 372007 IOPS
dm01cel01: LUN 5_1  on drive [FLASH_5_1] random read throughput: 3,226.00 MBPS, and 399469 IOPS
dm01cel01: CALIBRATE results are within an acceptable range.
dm01cel01: Calibration has finished.
dm01cel02: Calibration will take a few minutes...
dm01cel02: Aggregate random read throughput across all hard disk LUNs: 1195 MBPS
dm01cel02: Aggregate random read throughput across all flash disk LUNs: 12827 MBPS
dm01cel02: Aggregate random read IOs per second (IOPS) across all hard disk LUNs: 2990
dm01cel02: Calibrating hard disks (read only) ...
dm01cel02: LUN 0_0  on drive [8:0      ] random read throughput: 149.00 MBPS, and 254 IOPS
dm01cel02: LUN 0_1  on drive [8:1      ] random read throughput: 138.00 MBPS, and 253 IOPS
dm01cel02: LUN 0_10 on drive [8:10     ] random read throughput: 148.00 MBPS, and 255 IOPS
dm01cel02: LUN 0_11 on drive [8:11     ] random read throughput: 148.00 MBPS, and 248 IOPS
dm01cel02: LUN 0_2  on drive [8:2      ] random read throughput: 149.00 MBPS, and 250 IOPS
dm01cel02: LUN 0_3  on drive [8:3      ] random read throughput: 150.00 MBPS, and 252 IOPS
dm01cel02: LUN 0_4  on drive [8:4      ] random read throughput: 143.00 MBPS, and 252 IOPS
dm01cel02: LUN 0_5  on drive [8:5      ] random read throughput: 148.00 MBPS, and 241 IOPS
dm01cel02: LUN 0_6  on drive [8:6      ] random read throughput: 148.00 MBPS, and 255 IOPS
dm01cel02: LUN 0_7  on drive [8:7      ] random read throughput: 152.00 MBPS, and 256 IOPS
dm01cel02: LUN 0_8  on drive [8:8      ] random read throughput: 150.00 MBPS, and 255 IOPS
dm01cel02: LUN 0_9  on drive [8:9      ] random read throughput: 150.00 MBPS, and 242 IOPS
dm01cel02: Calibrating flash disks (read only, note that writes will be significantly slower) ...
dm01cel02: LUN 1_1  on drive [FLASH_1_1] random read throughput: 3,226.00 MBPS, and 366062 IOPS
dm01cel02: LUN 2_1  on drive [FLASH_2_1] random read throughput: 3,226.00 MBPS, and 401361 IOPS
dm01cel02: LUN 4_1  on drive [FLASH_4_1] random read throughput: 3,226.00 MBPS, and 403004 IOPS
dm01cel02: LUN 5_1  on drive [FLASH_5_1] random read throughput: 3,226.00 MBPS, and 408830 IOPS
dm01cel02: CALIBRATE results are within an acceptable range.
dm01cel02: Calibration has finished.
dm01cel03: Calibration will take a few minutes...
dm01cel03: Aggregate random read throughput across all hard disk LUNs: 1233 MBPS
dm01cel03: Aggregate random read throughput across all flash disk LUNs: 11763 MBPS
dm01cel03: Aggregate random read IOs per second (IOPS) across all hard disk LUNs: 3028
dm01cel03: Calibrating hard disks (read only) ...
dm01cel03: LUN 0_0  on drive [8:0      ] random read throughput: 148.00 MBPS, and 253 IOPS
dm01cel03: LUN 0_1  on drive [8:1      ] random read throughput: 152.00 MBPS, and 255 IOPS
dm01cel03: LUN 0_10 on drive [8:10     ] random read throughput: 148.00 MBPS, and 251 IOPS
dm01cel03: LUN 0_11 on drive [8:11     ] random read throughput: 154.00 MBPS, and 253 IOPS
dm01cel03: LUN 0_2  on drive [8:2      ] random read throughput: 150.00 MBPS, and 255 IOPS
dm01cel03: LUN 0_3  on drive [8:3      ] random read throughput: 152.00 MBPS, and 256 IOPS
dm01cel03: LUN 0_4  on drive [8:4      ] random read throughput: 148.00 MBPS, and 253 IOPS
dm01cel03: LUN 0_5  on drive [8:5      ] random read throughput: 147.00 MBPS, and 255 IOPS
dm01cel03: LUN 0_6  on drive [8:6      ] random read throughput: 144.00 MBPS, and 252 IOPS
dm01cel03: LUN 0_7  on drive [8:7      ] random read throughput: 153.00 MBPS, and 252 IOPS
dm01cel03: LUN 0_8  on drive [8:8      ] random read throughput: 143.00 MBPS, and 256 IOPS
dm01cel03: LUN 0_9  on drive [8:9      ] random read throughput: 142.00 MBPS, and 250 IOPS
dm01cel03: Calibrating flash disks (read only, note that writes will be significantly slower) ...
dm01cel03: LUN 1_1  on drive [FLASH_1_1] random read throughput: 3,226.00 MBPS, and 340622 IOPS
dm01cel03: LUN 2_1  on drive [FLASH_2_1] random read throughput: 3,226.00 MBPS, and 339174 IOPS
dm01cel03: LUN 4_1  on drive [FLASH_4_1] random read throughput: 3,227.00 MBPS, and 405781 IOPS
dm01cel03: LUN 5_1  on drive [FLASH_5_1] random read throughput: 3,226.00 MBPS, and 408817 IOPS
dm01cel03: CALIBRATE results are within an acceptable range.
dm01cel03: Calibration has finished.
[root@dm01db01 monitor]# 
此条目发表在 POC和性能调整 分类目录,贴了 , , , , 标签。将固定链接加入收藏夹。

发表评论

电子邮件地址不会被公开。 必填项已用 * 标注

您可以使用这些 HTML 标签和属性: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>