Disclaimer

These scripts come without warranty of any kind. Use them at your own risk. I assume no liability for the accuracy, correctness, completeness, or usefulness of any information provided by this site nor for any sort of damages using these scripts may cause.
Showing posts with label Data Guard. Show all posts
Showing posts with label Data Guard. Show all posts

Monday, October 29, 2012

Shell Script to Monitor Data Guard Status

Here is a shell script that can be scheduled to monitor the status of the data guard. The best method for monitoring data guard and databases is to user Oracle Enterprise Manager Grid Control and setup  and Manage the Data Guard.

Environment Used to test the script.

OS: Enterprise Linux Server release 5.3
Database: 11.2.0.2
No of Standby DB: 1
Data guard  Type: Physical Standby

Below script will connect to primary database and query V$ARCHIVE_DEST_STATUS and V$ARCHIVED_LOG to get the data guard status and un-applied log sequence count. If the data guard status is in error or count of un-applied logs are greater than 2, a subroutine can be called to alert the DBA. Assume that DEST_ID 2 is Standby Database.


#!/usr/bin/ksh
#Script to monitor Data guard Status and  alert if there is any issue.
#=======================================================================
DG_STATUS=`sqlplus -s "/ as sysdba" <<EOF
set heading off verify off feedback off serveroutput on
declare
l_stat varchar2(30);
l_gap_stat varchar2(100);
l_stat1 varchar2(150);
l_stat2 varchar2(20);
l_gap_cnt number;
l_appld_seq number;
begin
l_stat2 :='DG_NORMAL';
SELECT STATUS, GAP_STATUS into l_stat, l_gap_stat FROM "V\\$ARCHIVE_DEST_STATUS" WHERE DEST_ID = 2;
l_stat1 := l_stat||l_gap_stat;
dbms_output.put_line(l_stat1);
if l_stat1 <> 'VALIDNO GAP' then
--      dbms_output.put_line('Datagaurd Status is in Error');
        l_stat2 := 'DG_ERRORR';
end if;
select count(1) into l_gap_cnt from "V\\$ARCHIVE_GAP";
 dbms_output.put_line('Gap count: '||l_gap_cnt);

if l_gap_cnt>0 then
        dbms_output.put_line('Datagaurd Status is in Error');
        l_stat2 := 'DG_ERRORR';
end if;

SELECT COUNT(1) into l_appld_seq FROM "V\\$ARCHIVED_LOG" WHERE APPLIED='NO' and DEST_ID=2;
 dbms_output.put_line('Unapplied log count: '||l_appld_seq);

if l_appld_seq>2 then  -- Archived log may not apply immediatly so chk for more than 2
--      dbms_output.put_line('Datagaurd: Log apply is stopped');
        l_stat2 := 'DG_ERRORR';
end if;
dbms_output.put_line(l_stat2);
end;
/
quit
EOF`

rm -rf ./DG_PREV_STAT
cp ./DG_CURR_STAT ./DG_PREV_STAT
DG_PREV_STAT=`cat ./DG_PREV_STAT`


echo $DG_STATUS
l_stat=${DG_STATUS: -9}
echo $l_stat
if [ "$l_stat" =  "DG_NORMAL" ]; then
        echo "NORMAL" > ./DG_CURR_STAT
        if [ "$DG_PREV_STAT" = "NORMAL" ]; then
                echo "No Change in Dataguard Status"
         else
                echo "Status of Dataguard changed from ERROR to NORMAL"
        fi

else
        echo "ERROR" > ./DG_CURR_STAT
  if [ "$DG_PREV_STAT" = "ERROR" ]; then
    echo "No Change in Dataguard Status"
  else
    echo "Status of Dataguard changed from NORMAL to ERROR"
    #--- Call Subroutine to Alert DBA -------

    # For Example use wget function to call a SMS gateway URL
    # or mail command to send email.
    #-----------------------------------------

EOFT
 fi

fi

Monday, April 23, 2012

How to resolve MRP stuck issues on a physical standby database

How to resolve MRP stuck issues on a physical standby database

On my standby database, one day suddenly redo apply stopped and log files are showing the following error messages. On investigating, found that MRP process not running and doesn't apply more logs
ORA-16766: Redo Apply unexpectedly offline
ORA-00332: archived log is too small - may be incompletely archived
ORA-00334: archived log: <Archive log file name>
MRP stuck issues on a physical standby database
MRP0: Background Media Recovery terminated with error 332
Environment:
10g R2 Single  Instance Database with Dataguard Broker on Linux

I have performed the following activities and was able to restart the redo log apply
Solution:
1.       Verify the size & checksum of the reported archive log between primary and standby database. If they are different, then copy the archive log from primary to standby.  Can use md5sum to verify the checksum.

If the archive log not available you can restore from rman backup as follows
connect target /  catalog=username/pwd@catdb
run {
restore archivelog from logseq=340 until logseq=340 thread=1;
}


2.       Once the file is copied, restart the redo apply service.

Stop the managed recovery and start the manual recovery.

DGMGRL> EDIT DATABASE '<standby db_unique_name>' SET STATE='LOG-APPLY-OFF';
DGMGRL> EDIT DATABASE '<standby db_unique_name>' SET STATE='ONLINE';
3.       Verify the redolog apply started
ps –ef | grep mrp

Thursday, December 22, 2011

Adding Standby Redo Log and Resolving Data guard Error

ORA-16801: redo transport-related property is inconsistent with database Settings

Data Guard Setup: Primary with Two Standbys
DB Version: 10.2.0.4.0 64Bit
OS: Enterprise Linux 5.3 64Bit
Data Guard Broker: Yes
Redo Transport Mode: ASYNC

Second standby database created using rman duplicate command. Started the managed recovery process and logs are started applying. Data Guard broker enabled for this database.  While enabling & verifying the configuration for this database, I got the following error.

ORA-16801: redo transport-related property is inconsistent with database setting
ORA-16715: redo transport-related property LogXptMode of standby database "stby2" is inconsistent

Note: I tried to add Standby Redo Log (SRL) immediately after  creating standby but before starting the managed recovery, but I got the error ORA-301 signalled during: ALTER DATABASE ADD STANDBY LOGFILE GROUP  and not able to add the standby redo logs.

I performed the following steps to resolve this issue.

1.       Stop the redo apply on the standby database
ALTER DATABASE RECOVER MANAGED STANDBY DATABASE CANCEL
2.       Create the Standby Redo Log on standby

a) The size of the standby redo log files must match the size of primary database online redo log files. Find the size of the primary database online redo log files. Number of Standby Redo Logs should be at least one greater than number online redo logsSQL> select bytes from v$log;
   BYTES
   ----------
  52428800
  52428800
  52428800
Here the size of my online redo log files is 50M.

b) Use the following query to find the details of the log files in the primary database.
SQL> select group#,type,member from v$logfile;
    GROUP# TYPE    MEMBER
---------- ------- --------------------------------------------------
         1 ONLINE  /u03/oradata/prod_redo01a.log
         1 ONLINE  /u03/oradata/prod_redo01b.log
         2 ONLINE  /u03/oradata/prod_redo02a.log
         2 ONLINE  /u03/oradata/prod_redo02b.log
         3 ONLINE  /u03/oradata/prod_redo03a.log
         3 ONLINE  /u03/oradata/prod_redo03b.log
         4 STANDBY /u03/oradata/prod_redo04a.log
         4 STANDBY /u03/oradata/prod_redo04b.log
         5 STANDBY /u03/oradata/prod_redo05a.log
         5 STANDBY /u03/oradata/prod_redo05b.log
         6 STANDBY /u03/oradata/prod_redo06a.log

    GROUP# TYPE    MEMBER
---------- ------- --------------------------------------------------
         6 STANDBY /u03/oradata/prod_redo06b.log
         7 STANDBY /u03/oradata/prod_redo07a.log
         7 STANDBY /u03/oradata/prod_redo07b.log

From above output you can see that I already have standby redo logs on primary.

c) Now I have created the standby redo log files in the standby database.
alter database add standby logfile group 4 ('/u03/oradata/prod_redo04a.log’, '/u03/oradata/prod_redo04b.log’) size 50M ;
alter database add standby logfile group 5 ('/u03/oradata/prod_redo05a.log’, ‘/u03/oradata/oas2/prod_redo05b.log’) size 50M;
alter database add standby logfile group 6 ('/u03/oradata/oas2/prod_redo06a.log’, ‘/u03/oradata/oas2/prod_redo06b.log’) size 50M;
alter database add standby logfile group 7 ('/u03/oradata/oas2/prod_redo07a.log’, ‘/u03/oradata/oas2/prod_redo07b.log’) size 50M;

3.       Start the log apply

SQL> ALTER DATABASE RECOVER MANAGED STANDBY DATABASE DISCONNECT FROM SESSION;

4.     Enable second standby (SSTD)
DGMGRL> enable database stdby2;
5.       Verify the configuration
DGMGRL> show configuration;

Monday, July 2, 2007

Patching oracle apps with data guard

Lately we have applied a ptach on our apps instance with physical standby database.



Following are the steps used.



1. Stopped the recovery delay in standby db

2. Stopeed application tier services on production

3. Switch the log in prod db and get the last sequence#

4. verify on standby that last log applied

5. Stop recovery on standby

6. Take the apps prod instance to maintenance mode using adadmin

7. Perfoem patching

8. Re-start redo shipping and apply on standby

9. Run middle tier pre-clone on prod

10. Run DB tier pre-clone (Only if patching updates ORACLE_HOME)

11. Synchronize the application tier file system with standby (use rsync on linux)

12. When sync completes create new context file using adclonectx.pl

13. After new context file created configure file system using adconfig.pl

14. Optinally configure standby DB file system if you did step 10.

15. End.



Now the production and stnadby oracle applications instances are in sync.