The Geek Diary

HowTos | Basics | Concepts

How to change the ASM rebalance power of an ongoing operation

By

ASM rebalance Power

Whenever there is a disk added/removed/resized, ASM starts to rebalance the data on the disks in the disk group. The power with which it does the rebalancing is the rebalancing power. The rebalance power defaults to the value of the ASM_POWER_LIMIT initialization parameter (default value = 1). The higher the limit, the faster a rebalance operation may complete. Lower values cause rebalancing to take longer but consume fewer processing and I/O resources. This leaves these resources available for other applications, such as the database. Beginning with Oracle Database 11g Release 2 (11.2.0.2), if the COMPATIBLE.ASM disk group attribute is set to 11.2.0.2 or higher, then the range of ASM_POWER_LIMIT parameter is 0 to 1024. All previous versions have the parameter range of 0 to 11.

The disk group rebalance operation has three phases:
1. Planning
2. File extents relocation
3. Disk compacting

It is possible to adjust this parameter dynamically, however adjusting ASM_POWER_LIMIT only affects future rebalances. It does not affect an in progress rebalance. The asm_power_limit can be found in the v$asm_operation view.

SQL> show parameter limit

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------
asm_power_limit                      integer     1

Changing the rebalancing power of an ongoing operation

Let us drop a disk from the diskgroup DATA :

SQL> alter diskgroup DATA drop disk data_disk01;

Diskgroup altered.

Whenever there is a rebalancing in the disk group, a rebalancing process ARBn will start :

SQL> select * from v$asm_operation;

GROUP_NUMBER OPERA STAT      POWER     ACTUAL      SOFAR   EST_WORK   EST_RATE
------------ ----- ---- ---------- ---------- ---------- ---------- ----------
           1 REBAL WAIT          1          0          0          0

ASM alert log :

Fri Dec 26 16:55:08 2008
NOTE: starting rebalance of group 1/0x8c6014a7 (DATA) at power 1
Starting background process ARB0
ARB0 started with pid=14, OS id=316
Fri Dec 26 16:55:08 2008
NOTE: assigning ARB0 to group 1/0x8c6014a7 (DATA)

As we can see, the power of the rebalancing is 1, as it defaults to the parameter ASM_POWER_LIMIT. After running a rebalance command, the running rebalance process will be stopped and new rebalance slave processes will be spawned depending on POWER used.

SQL> alter diskgroup dg1 rebalance power 8;

Diskgroup altered.

Check for the 8 new rebalance slave processes spawned (ARB0,….,ARB7)

SQL> select * from v$asm_operation;

GROUP_NUMBER OPERA STAT      POWER     ACTUAL      SOFAR   EST_WORK   EST_RATE
------------ ----- ---- ---------- ---------- ---------- ---------- ----------
           1 REBAL RUN           8          8          0        407          0

ASM alert.log :

Fri Dec 26 16:55:11 2008
SQL> alter diskgroup dg1 rebalance power 8 

NOTE: stopping process ARB0
Fri Dec 26 16:55:11 2008
NOTE: rebalance interrupted for group 1/0x8c6014a7 (DG1)
Fri Dec 26 16:55:11 2008
NOTE: PST update: grp = 1
NOTE: requesting all-instance PST refresh for group=1
Fri Dec 26 16:55:11 2008
NOTE: PST refresh pending for group 1/0x8c6014a7 (DG1)
SUCCESS: refreshed PST for 1/0x8c6014a7 (DG1)
Fri Dec 26 16:55:14 2008
NOTE: starting rebalance of group 1/0x8c6014a7 (DG1) at power 8
Starting background process ARB0
Starting background process ARB1
ARB0 started with pid=14, OS id=1628
Fri Dec 26 16:55:15 2008
Starting background process ARB2
ARB1 started with pid=15, OS id=1920
Fri Dec 26 16:55:15 2008
Starting background process ARB3
ARB2 started with pid=16, OS id=1964
Fri Dec 26 16:55:15 2008
Starting background process ARB4
ARB3 started with pid=17, OS id=1664
Fri Dec 26 16:55:15 2008
Starting background process ARB5
ARB4 started with pid=18, OS id=1836
Fri Dec 26 16:55:15 2008
Starting background process ARB6
ARB5 started with pid=19, OS id=1972
Fri Dec 26 16:55:15 2008
Starting background process ARB7
ARB6 started with pid=20, OS id=1800

How much should I increase the power ?

The rebalance power can be adjusted dynamically, i.e. during the rebalance. If the rebalance with the default power is ‘too slow’, the power can be increased. How much? To answer that questions, we need to understand the I/O load, the I/O throughput and most importantly the I/O limits the system can take. If we don’t know that, the power can be increased to 5 (with ‘ALTER DISKGROUP … REBALANCE POWER 5;’). We can then check if that makes a difference. Should we go any higher with the rebalance power? Again, as long as we are not adversely impacting the database I/O performance, we can keep increasing the power. But I haven’t seen much improvement beyond power 30. Note that the power can go up to 11 with disk groups with COMPATIBLE.ASM &lt 11.2.0.2 and up to 1024 for disk groups with COMPATIBLE.ASM=&gt11.2.0.2.

The testing is the key here. We really need to test the rebalance under the regular production load, with different values for the power. There is no point testing with no databases running or on a system with different storage characteristics

You May Also Like