The Basics
Oratcptest is a command line tool that runs on two hosts, one in client mode and the other in server mode, to measure network bandwidth and latency between client and server. Oratcptest establishes a TCP connection between the client host and server host and then sends network messages to Oratcptest on the server host. Oratcptest running on the server host writes, if desired, the received messages to disk before replying with the ACK message.
Oratcptest could be used as a general purpose tool for measuring network bandwidth and latency. However, oratcptest has been written specifically to help users assess network resources prior to performing Data Guard configuration, RMAN backup, migration, DG instantiation, etc.
Users can control the test behavior by specifying various options. For example,
- Network message size
- Delay time in between messages
- Parallel sending streams
- Whether Oratcptest-server should write messages on disk, or not.
- Whether Oratcptest-client should wait for ACK before it sends next message, or not, thus simulating the ASYNC and SYNC transport used by Data Guard.
Installation
1. Copy the jar file attached to this MOS note onto both client and server hosts:
2. Verify that host where oratcptest is being installed has JRE 6 or later.
% java -version java version "1.7.0_09-icedtea" OpenJDK Runtime Environment (rhel-2.3.4.1.0.1.el6_3-x86_64) OpenJDK 64-Bit Server VM (build 23.2-b09, mixed mode)
3. Verify that the JVM can run the jar file with this command on both hosts.
% java -jar oratcptest.jar -help
This command should display the help.
4. Start the test server on the receiving side (note that you can supply any available port for the server).
% java -jar oratcptest.jar -server -port=5555
5. Run the test client (please change the server address and port number to match that of your server started in step 4).
% java -jar oratcptest.jar test.server.address.com -port=5555 -duration=10s -interval=2s
The test will display output similar to the following :
Message payload = 1 Mbyte
Payload content type = RANDOM
Delay between messages = NO
Number of connections = 1
Socket send buffer = (system default)
Transport mode = SYNC
Disk write = NO
Statistics interval = 2 seconds
Test duration = 10 seconds
Test frequency = NO
Network Timeout = NO
(1 Mbyte = 1024x1024 bytes)
(07:43:06) The server is ready.
Throughput
(07:43:08) 107.747 Mbytes/s
(07:43:10) 107.900 Mbytes/s
(07:43:12) 107.826 Mbytes/s
(07:43:14) 107.861 Mbytes/s
(07:43:16) 107.914 Mbytes/s
(07:43:16) Test finished.
Socket send buffer = 526476 bytes
Avg. throughput = 107.819 Mbytes/s
6. To see a complete list of options issue the following command:
$ java -jar oratcptest.jar -help
Assessing Network Bandwidth for Data Guard Transport
The goal for all Data Guard configurations is to ship redo data to the remote disaster recovery site fast enough to meet recovery time and recovery point objectives. No amount of tuning can achieve this goal if there is insufficient bandwidth available to handle the required volume. To answer question of how much network bandwidth is needed to configure Data Guard for this application, start by collecting the redo volume of your production database and then determine if there’s sufficient network bandwidth using the oratcptest tool.
The simplest way to determine application throughput in terms of redo volume is to collect AWR reports during normal and peak workload and determine the number of bytes per second of redo data the production database is producing. However, if you use peak redo rates as obtained from AWR reports to size your network you might not truly have enough bandwidth. If you take AWR snapshot every 30 minutes it could be that during that 30 minutes period you had really high redo rates for 10 minutes and moderate to low rates for the remainder of the time. This would average down the redo rate as seen in the AWR report. To determine peak redo rate for each log use the following query:
sql> select thread#,sequence#,blocks*block_size/1024/1024 MB,(next_time-first_time)*86400 sec, blocks*block_size/1024/1024/((next_time-first_time)*86400) "MB/s" from v$archived_log where ((next_time-first_time)*86400<>0) and first_time between to_date('2015/01/15 08:00:00','YYYY/MM/DD HH24:MI:SS') and to_date('2015/01/15 11:00:00','YYYY/MM/DD HH24:MI:SS') and dest_id=2 order by first_time;
You should see output similar to the following:
THREAD# SEQUENCE# MB SEC MB/s
---------- ---------- ---------- ---------- ----------
2 2291 29366.1963 831 35.338383
1 2565 29365.6553 781 37.6000708
2 2292 29359.3403 537 54.672887
1 2566 29407.8296 813 36.1719921
2 2293 29389.7012 678 43.3476418
2 2294 29325.2217 1236 23.7259075
1 2567 11407.3379 2658 4.29169973
2 2295 29452.4648 477 61.7452093
2 2296 29359.4458 954 30.7751004
2 2297 29311.3638 586 50.0193921
1 2568 3867.44092 5510 .701894903
The above query output tells us that we must accommodate for a peak redo of 61MB/sec. In general we recommend adding an additional 30% on top of the peak rate.
Now that we know our peak redo rate we can measure the network bandwidth with the oratcptest tool to see if it can sustain that peak rate. For example, on one of the standby hosts start the oratcptest server process:
% java -jar oratcptest.jar -server -port=5555
On one of the primary hosts use oratcptest to connect to the server process on the standby host and transfer a fixed amount of data. For example, if the Data Guard transport will be using ASYNC run a command similar to the following to determine the maximum throughput the network will provide:
$ java -jar oratcptest.jar slcc32adm07 -port=5555 -mode=async -duration=10s -interval=2s
[Requesting a test]
Message payload = 1 Mbyte
Payload content type = RANDOM
Delay between messages = NO
Number of connections = 1
Socket send buffer = (system default)
Transport mode = ASYNC
Disk write = NO
Statistics interval = 2 seconds
Test duration = 10 seconds
Test frequency = NO
Network Timeout = NO
(1 Mbyte = 1024x1024 bytes)
(08:08:03) The server is ready.
Throughput
(08:08:05) 113.052 Mbytes/s
(08:08:07) 113.232 Mbytes/s
(08:08:09) 113.198 Mbytes/s
(08:08:11) 113.205 Mbytes/s
(08:08:13) 113.189 Mbytes/s
(08:08:13) Test finished.
Socket send buffer = 528880 bytes
Avg. throughput = 113.140 Mbytes/s
The above example shows that the avg. network throughput of 113MB/sec is sufficient to keep up with the database peak redo rate. If the network throughput is not enough try adjusting the socket send and receive buffer sizes similar to the following:
Server:
$ java -jar oratcptest.jar -server -port=5555 -sockbuf=2097152
Client:
$ java -jar oratcptest.jar slcc32adm07 -port=5555 -mode=async -duration=60s -sockbuf=2097152
Note that the value for sockbuf size cannot exceed the maximum allowed by the Operating System. For Linux that is controlled by the /proc/sys/net/core/rmem_max and /proc/sys/net/core/wmem_max values. Once you have determined the optimal tcp socket buffer size it is recommended to configure Oracle Net recv_buf_size and send_buf_size as described in the MAA best practice guide to those values.
Assessing Network Bandwidth for Data Guard SYNC Transport
Synchronous redo transport requires a primary database to wait for confirmation from the standby that redo has been received and written to disk (a standby redo log file) before commit success is signaled to the application. When assessing the performance of the remote write it is important to consider the following items:
- How large is the redo write size that LGWR is submitting to the network
- How long does it take the network to transfer that redo write
- How long does it take for the I/O to complete to the standby redo log
Using oratcptest we can simulate SYNC writes over the network by first finding out the average redo write size that LGWR performs. This is done by using an AWR report on the primary database and dividing the “redo size” statistic by the “redo writes” statistic and setting the oratcptest tool -length option to this value. Note that redo write sizes typically increase when using SYNC transport due to batching of commits so you may need to test with larger redo write sizes. You can specify that the oratcptest server process write network message to the same disk location where the standby redo logs will be placed (note that ASM is currently not supported). For example, from the AWR report:
total redo size - 3,418,477,080 total redo writes - 426,201
Average redo write size (redo size / redo writes) = 8,020 or 8k. If the standby redo logs will be placed on /u01/oraredo the server command to issue would be:
$ java -jar oratcptest.jar -server -port=5555 -file=/u01/oraredo/oratcp.tmp
On the sending side we would issue the following client command to send over 8k messages with SYNC writes:
$ java -jar oratcptest.jar slcc32adm07 -port=5555 -write -mode=sync -length=8020 -duration=10s -interval=2s
[Requesting a test]
Message payload = 8020 bytes
Payload content type = RANDOM
Delay between messages = NO
Number of connections = 1
Socket send buffer = (system default)
Transport mode = SYNC
Disk write = YES
Statistics interval = 2 seconds
Test duration = 10 seconds
Test frequency = NO
Network Timeout = NO
(1 Mbyte = 1024x1024 bytes)
(07:24:04) The server is ready.
Throughput Latency
(07:24:06) 21.832 Mbytes/s 0.351 ms
(07:24:08) 25.967 Mbytes/s 0.295 ms
(07:24:10) 27.846 Mbytes/s 0.275 ms
(07:24:12) 28.155 Mbytes/s 0.272 ms
(07:24:14) 27.917 Mbytes/s 0.274 ms
(07:24:14) Test finished.
Socket send buffer = 8 kbytes
Avg. throughput = 26.335 Mbytes/s
Avg. latency = 0.291 ms
In the above output, latency includes the time to send the message to the server host, the optional disk write at the server host and the acknowledgment back to the client process.
Command Options
The following table lists the available server options:
| Options | Description |
|---|---|
| -port | Listening TCP port number. Must be specified. |
| -file=[name] | File name for disk-write test. Default value is oratcp.tmp. |
| -sockbuf=[bytes] | Server socket receive buffer size. Default value is zero, which means system default receive buffer size. |
| -help | Displays help |
The following table lists the available client options:
| Options | Description |
|---|---|
| -port | Listening TCP port number. Must be specified. |
| -write | server writes network message to disk before server replies with ACK. |
| -mode=[SYNC|ASYNC] | In SYNC mode, client waits for server’s ACK before it sends next message. In ASYNC mode, it doesn’t wait. Default value is SYNC |
| -num_conn=[number] | Number of TCP connections. Default value is 1. |
| -sockbuf=[bytes] | Client socket receive buffer size. Default value is zero, which means system default receive buffer size. |
| -length=[bytes] | Message payload length. Default value is 1 Mbyte. |
| -delay=[milliseconds] | Delay in milliseconds between network messages. Default value is zero, which means no delay between messages. |
| -rtt | Round-trip-time measurement mode. Equivalent to -mode=SYNC and -length=0. |
| -random_length | random payload length uniformly distributed between 512 bytes and -length option value. |
| -random_delay | random delay uniformly distributed between zero and -delay option value. |
| -payload=[RANDOM|ZERO|[filename]] | payload content type among random data, all zeroes, or the contents of a user-specified file. Default value is RANDOM. |
| -interval=[time] | Statistics reporting interval. Default value is 10 seconds. |
| -duration=[time] or [bytes] | Test duration in time or bytes. If not specified, test does not terminate. |
| -freq=[time]/[time] | Test repeat frequency. For example, -freq=1h/24h means the test will repeat every 1 hour for 24 hours. |
| -timeout=[time] | Network timeout. Default value is zero, which means no timeout. |
| -output=[name] | Output file name where client stores test result statistics. |
| -help | display help |