The Problem
On CentOS/RHEL 7 with Oracle database 12.1.0.2 version, when attempting to stop DB during server shutdown the automatic shutdown script getting below error:
# cat /u01/app/oracle/product/12.1.0.2/dbhome_1/shutdown.log Processing Database instance "XXXX1": log file /u01/app/oracle/product/12.1.0.2/dbhome_1/shutdown.log Info: Database instance "XXXX" already down (PMON process not there).
Which indicate that PMON is forcefully being closed before the script is called. This is the configuration used
# cat dbora.service [Unit] Description=The Oracle Database Service After=network.target [Service] Type=forking # Type=oneshot RemainAfterExit=yes KillMode=none # Set this to something larger if it has an impact TimeoutStopSec=0 ExecStart=/u01/app/oracle/product/12.1.0.2/dbhome_1/bin/dbora start ExecStop=/u01/app/oracle/product/12.1.0.2/dbhome_1/bin/dbora stop [Install] # Puts wants directive for the other units in the relationship WantedBy=default.target
systemctl enable dbora.service systemctl daemon-reload systemctl start dbora.service
The Solution
Issue with the dbora script which is doing su to oracle owner process and systemd is not able to keep track of that process. This is the sample dbora script
ORA_HOME=/u01/app/oracle/product/12.1.0.2/dbhome_1 ORA_OWNER=oracle case "$1" in 'start') # Start the Oracle databases: # The following command assumes that the oracle login # will not prompt the user for any values # Remove "&" if you don't want startup as a background process. su - $ORA_OWNER -c "$ORA_HOME/bin/dbstart $ORA_HOME" touch /var/lock/subsys/dbora ;; 'stop') # Stop the Oracle databases: # The following command assumes that the oracle login # will not prompt the user for any values su - $ORA_OWNER -c "$ORA_HOME/bin/dbshut $ORA_HOME" rm -f /var/lock/subsys/dbora ;; esac
With this arrangement the systemd is losing control over the service and not stopping it in time. The better option is to use dbstart and dbstop script directly under the directive with user and owner details:
[Unit] Description=The Oracle Database Service After=network.target [Service] Type=forking RemainAfterExit=yes KillMode=none TimeoutStopSec=5min User=oracle << Replace Oracle software owner details here Group=oinstall ExecStart=$ORACLE_HOME/bin/dbstart $ORACLE_HOME & ===> Please use absolute path here instead of the $ORACLE_HOME variable ExecStop=$ORACLE_HOME/bin/dbshut $ORACLE_HOME ===> Please use absolute path here instead of the $ORACLE_HOME variable Restart=no [Install] # Puts wants directive for the other units in the relationship WantedBy=default.target
Then systemd has control about the services
# systemctl status dbora.service ● dbora.service - The Oracle Database Service Loaded: loaded (/usr/lib/systemd/system/dbora.service; enabled; vendor preset: disabled) Active: active (running) since Wed 2017-02-01 12:19:53 GMT; 22s ago Process: 3905 ExecStop=/u01/app/oracle/product/12.1.0.2/dbhome_1/bin/dbshut /u01/app/oracle/product/12.1.0.2/dbhome_1 (code=exited, status=0/SUCCESS) Process: 4043 ExecStart=/u01/app/oracle/product/12.1.0.2/dbhome_1/bin/dbstart /u01/app/oracle/product/12.1.0.2/dbhome_1 & (code=exited, status=0/SUCCESS) CGroup: /system.slice/dbora.service ├─4051 /u01/app/oracle/product/12.1.0.2/dbhome_1/bin/tnslsnr LISTENER -inherit ├─4143 ora_pmon_XXXXX└─4477 ora_q003_XXXX