Saturday, August 29, 2020

Verify readiness of Database switchover using DG Validate command

5:55 PM Posted by Dilli Raj Maharjan , , , 8 comments
    
    Oracle Dataguard ensures high availability, data protection, and disaster recovery for Oracle database. Data Guard provides a comprehensive set of services for DR solution. The services can be used to create, maintain, manage, monitor one or more standby databases, and validate the database for the readiness of role switch. Data Guard can switch any standby database manually or automatically in case the production database becomes unavailable due to a planned or unplanned outage. Data Guard minimizes the downtime associated with the outage.

    Oracle Data Guard is included with the Enterprise Edition and Personal Edition of the Oracle database software. We can manage an Oracle Data Guard configuration by using either SQL*Plus, the Oracle Data Guard broker's command-line interface (DGMGRL), or a compatible version of Oracle Enterprise Manager Cloud Control (Cloud Control).

    Validate command has a set of options used to check a comprehensive set of checks prior to role change. This command is used to verify the readiness of switchover and to validate the standby database is ready to failover and switchover anytime. Validate command was first introduced in Oracle database 12.1.0.0. New commands like validate database spfile, validate network configuration, validate static connect identifier was introduced in Oracle Database Release 18c, Version 18.1. For more details click here.

Environment:

Docker Image: dillimaharjan/oracle19c
Docker Container: ora19cp, ora19cs
Oracle Database version: 19.3.0.0
Oracle Database Edition: Oracle Database 19c Enterprise Edition
Container database: False

In my case, there are docker images stored in my repo for different versions of the database. I have executed docker image command to list all available images. oracle19c is a docker image with fresh Oracle 19.3.0.0 installed and ready to use.

docker image ls | sort




Start docker container using the docker run command. The following are a brief description of the option being used.
-i                     : Interactive mode
-t                     : Allocate a pseudo-TTY
-d                    : Detach once container is started
--privileged    :  Give an extended privileges to the container
-v             : Bind mount a volume. OS directory will be mounted to the specified mount point in the container.
-p                    : Publish a container port to the host
--hostname     : Host name of the container.
--name            : Name of the container.

docker run -i -t -d --privileged -v /Users/dilli/Docker/shared:/shared -p 15219:1521 --hostname ora19c_primary --name ora19cp dillimaharjan/oracle19c
docker run -i -t -d --privileged -v /Users/dilli/Docker/shared:/shared -p 15220:1521 --hostname ora19c_standby --name ora19cs dillimaharjan/oracle19c



List running docker containers with docker ps command.
docker ps



Run bash in a running container to get bash terminal of the container.
docker exec -it ora19cp /bin/bash
docker exec -it ora19cs /bin/bash




Physical standby and the DG have been configured for those docker containers. If you need any help to configure physical standby or DG. Please refer to the links below.


Once DG configuration is completed, the configuration looks like below. The command show configuration will list the primary and standby database along with the FSFO and configuration status.

show configuration.





1. Validate Database

The validate database command is used for a comprehensive set of database checks prior to a role change. Role change from Primary to Standby and from Standby to Primary. This command is used to verify the readiness prior to the switchover.

Syntax:
VALIDATE DATABASE [VERBOSE] <database-name>;

Validating primary database. 

The Ready for Switchover is Yes, which means the database has no issue and ready to switchover anytime.

validate database accdb;



Verbose option will reveal more details about the database. 
validate database verbose accdb;



Validating standby database.

The Ready for Switchover and the Ready for Failover both have value Yes, which means the database is ready for switchover and failover.

validate database accstdb;



validate database verbose accstdb;





2. Validate database datafile

The validate database datafile validates data files across the primary and standby database. Option ALL can be defined to validate all the datafiles on either of the database or all databases. OUTPUT=<output_file_name> is mandatory and command dumps output to the file. Output should not contain any directory. The output file will be store in the trace directory.


Syntax:
VALIDATE DATABASE [database-name | ALL] DATAFILE [datafile-name | datafile-number | ALL] OUTPUT="<output file name>";

List the available data files in the database.

select file# from v$database;



If you specify the directory in the output file it will return error: DGM-17138: OUTPUT should not contain a directory. The output file will be saved in the trace file location.


Validate datafile # 1 in database accdb and save all output to accdb_dbfile_1.txt file in the trace location.
Additionally, validate all datafiles in accdb database, validate datafile # 1 in accstdb, and validate all datafiles in all databases.

validate database accdb datafile 1 output='accdb_dbfile_1.txt'
validate database accdb datafile all output='accdb_all_dbfiles.txt'
validate database accstdb datafile 1 output='accstdb_dbfile_1.txt';
validate database all datafile all output='all_datafiles.txt'




Sample content of the output file.

Client is connected to database: accdb. Role: primary database.

Remote database accstdb.remote db role: physical standby


Slave Id  0

Summary:

*******************************************************************************

                       TOTAL: total no. of blocks found

                         |

   +--------+------------+-------+---------+---------+

   |        |                    |         |         |

   |      DIFFV:             LOST_WRITE    |       CORR: corrupted blocks

 SAMEV     diff ver              |       SKIPPED:

   |       block pairs        +--+--+      direct load, empty blocks,

+--+--+--+                    |     |      RMAN optimized blocks,

|  |  |  |                    |     |      flashback optimized blocks

|  |  | SAMEV&C:              |     |

|  |  |  same ver &           |   LWLOC: lost writes at local db

|  |  |  same checksum &     LWRMT: lost writes at remote db

|  |  |  same contents

|  |  |

|  | SAMEV_NO_CHKSUM: same ver & same contents but diff checksum

|  |                  (checksum can be diff but identical contents)

|  |

| DIFFPAIR: same ver but differrent contents (data inconsistency)

|

ENCERR: undecided block pairs due to encryption related issue

        (e.g. when Wallet is not open)



ID TOTAL   CORR SKIPPED DIFFV   SAMEV   SAMEV&C ENCERR  LWLOC  LWRMT DIFFPAIR

00 0039190 0000 0039189 0000001 0000000 0000000 0000000 000000 000000 0000000

02 0042599 0000 0000000 0000005 0042594 0042566 0000000 000000 000000 0000000

06 0126447 0000 0027918 0000286 0098243 0098224 0000000 000000 000000 0000000

14 0000001 0000 0000000 0000000 0000001 0000001 0000000 000000 000000 0000000


3. Validate database spfile

The validate spfile command validates the difference in spfile between the primary and the standby database. Executing this command prior to switchover lists out all the parameter differences. Setting parameters of primary and standby exactly the same may prevent performance issues after switchover. If all the parameter values are the same then "No parameter difference found." message will be displayed.

Syntax:
VALIDATE DATABASE [verbose] {standby database-name} spfile;

Note: You should log in as user sys to execute following command
If you log in with os authentication in dgmgrl then the following error will be displayed
Command requires a connection that uses database or external credentials.



Note: Validate spfile command cannot be executed for the Primary database. The following error will be encountered if you execute validate spfile for the primary database.
This command cannot be used for the primary database.



Validate database <standby database> spfile;



Verbose option will display matched and unmatched parameter values.



4. Validate FAR_SYNC


The validate far_sync validates far sync instance. It performs a set of checks for a far sync instance.


Syntax:
VALIDATE FAR_SYNC VERBOSE] far_sync_instance_name [WHEN PRIMARY IS <primary db_name>]


5. Validate Network configuration.

The validate network configuration command checks the network configuration between members. It performs network connectivity checks and reports for any issues.


Syntax:
VALIDATE NETWORK CONFIGURATION FOR <ALL | MEMBER NAME>;


In my case the hostname of standby is not resolvable from Primary and hostname of Primary is not resolvable from Standby because these settings are missing on my /etc/hosts file. So I have encountered the following error.


Once my error is resolved the validate command passed.

validate network configuration for all;
validate network configuration for accdb;
validate network configuration for accstdb;




6. Validate static connect identifier.

The validate static connect identifier validates the static connect identifier of a database. Static connect identifiers are required to start the database during the switchover.


Syntax:
VALIDATE STATIC CONNECT IDENTIFIER FOR <ALL | DATABASE NAME>;


In my case I have not set <server_name>_DGMGRL static server name in the listener configuration that is why I am getting error ORA-12514: TNS:listener does not currently know of service requested in connect descriptor.





Once I added static service name <service>_DGMGRL the validate command passed.

validate static connect identifier for all;
validate static connect identifier for accdb;
validate static connect identifier for accstdb;







Switchover

Once all my validate commands passed, I have executed switchover and it succeed without any issue.


Now accstdb is my primary database and accdb is standby database.












Wednesday, July 1, 2020

Install Oracle Database 19c in docker container

9:07 PM Posted by Dilli Raj Maharjan , No comments



Docker is a set of platform as a service (PaaS) product that uses OS-level virtualization to deliver software in packages called containers. Containers are isolated from one another and bundle their own software, libraries, and configuration files; they can communicate with each other through well-defined channels. All containers are run by a single operating system kernel and therefore use fewer resources than virtual machines. The software that hosts the containers is called Docker Engine.  It was first started in 2013 and is developed by Docker, Inc.

All versions of Oracle databases are now available as a container in Oracle official Github.

Before starting the installation, install the Docker on your computer. I have already downloaded and installed Docker for Mac. Make sure your Docker app is up and running before start installation.

Following is the step by step guide to install Oracle Database 19c in a docker container.

Open URL https://github.com/oracle/docker-images in the browser and click on Clone.



Click on copy icon to copy the web URL.



Open terminal and change directory to one where you want to clone this git URL. 
Type git clone https://github.com/oracle/docker-images to clone docker images from Oracle.



Once docker files are fully cloned it displayes 100% done.



Change directory to docker-images. There are a lot of directories for Oracle software docker image files.
cd docker-images



Change Directory to OracleDatabase and SingleInstance. There is a directory for RAC also but we are installing the Single Instance Oracle 19c database. 
cd OracleDatabase
cd SingleInstance



Change directory to Dockerfiles and you will notice the database versions there. Change directory to 19.3.0 for Oracle 19c installation.
cd dockerfiles
cd 19.3.0



Download Oracle database version 19c from Oracle official site. Copy Oracle 19c source file to the current directory. Do not extract file over here. Just copy the compressed source file.
cp ~/Documents/Software/Oracle/Oracle_19c/LINUX.X64_193000_db_home.zip .


List all the available files on the directory. Validate Oracle19c source file has been copied successfully.



Select the content of db_inst.rsp response file. Use the following command to exclude blank lines and comments from the response file. We can see that only Oracle software will be installed. Oracle Base location, Oracle Home location, and Oracle Edition will be set via environment variables.
grep -v ^# db_inst.rsp | grep -v ^$



List the content of dbca.rsp.tmpl file. Use the command below to exclude blank lines and comments from the dbca response file. A container database with one pluggable database will be installed using this dbca response file. Pluggable database name, Oracle sid, and sys password will be set via environment variables.
grep -v ^# dbca.rsp.tmpl | grep -v ^$



Change directory to parent directory and execute buildDockerImage.sh executable binary with -v and -e option. Specify the version number with v option and -e for enterprise edition image.



Building Image in progress #####



Building Image in progress ###############



Once Oracle Software Installation is complete it will display the message as Successfully setup Software. Few pre-requisites before Oracle software installation failed so there are warning messages with the successful setup.



Finally Oracle image build is complete and tagged as name oracle/database:19.3.0-ee



Execute docker run to start docker container with Oracle 19c image. During its first run, it will create a database using dbca so it may take 15-20 min to complete dbca task. The following are the options used and their description. 

docker run \
--name oracle19c \
-p 1521:1521 \
-p 5500:5500 \
-e ORACLE_PDB=orcl \
-e ORACLE_PWD=password \
-e ORACLE_MEM=2000 \
-v /opt/oracle/oradata \
-d \
oracle/database:19.3.0-ee

run   : Command docker to run specified Image.
-p.    : Publish port in the form of local port:docker container port.
-e     : Set environment variable
-v     : Bind mount a volume
-d     : Detach. Run container in background and print container ID
oracle/database:19.3.0-ee : Name of image.



If you list the currently running docker container you may notice the health of the container is starting. The database creation process is in progress now.



While checking the docker log we can notice oracle dbca is in progress. Docker log can be checked from the command line as well as from the Docker dashboard. We can see dbca outputs in the docker log file.

Checking the docker log from the command line. We can noticed listener status and dbca ongoing.




Checking log via Docker GUI. We can notice database configuration is on progress.

Oracle 19c database configuration in progress. #####



Oracle 19c database configuration in progress. ##########



# Oracle Database installation complete.



Logged in to the container with the command below and check the listener status. We can notice the listener is up and running.
docker exec -it oracle19c /bin/bash


lsnrctl status


Provide username, password, Service Name, and click on Test. It will return success if the installation completes successfully.



 Now we can execute Oracle SQL commands in the SQLDeveloper workspace.

 


List current running docker containers.








docker ps



Stopping Docker container.


Starting Docker container.


Execute /bin/bash for docker container.