Monday, May 25, 2015

Compare Oracle tables with linux md5

2:33 PM Posted by Dilli Raj Maharjan , No comments


Comparison between two table with same name in different schema 
Lets say MEMBERS be the table that need to be compared on schemas SCHEMA_A and SCHEMA_B.

Enabling timing to track down the time taken.

set timing on;

Dropping external table if exists from SCHEMA_A
drop table SCHEMA_A.MEMBERS_EXT_A;

Create external table from first schema, SCHEMA_A in our case  
CREATE TABLE SCHEMA_A.MEMBERS_EXT_A
ORGANIZATION EXTERNAL
(
TYPE ORACLE_DATAPUMP
DEFAULT DIRECTORY EXT_DIR
ACCESS PARAMETERS
(
nologfile
)
LOCATION ('MEMBERS_EXT_A.csv')
)
AS SELECT * FROM SCHEMA_A.MEMBERS; 




Dropping external table if exists from SCHEMA_B
drop table SCHEMA_B.MEMBERS_EXT_B; 

Create external table from second schema, SCHEMA_B in our case
CREATE TABLE SCHEMA_B.MEMBERS_EXT_B
ORGANIZATION EXTERNAL
(
TYPE ORACLE_DATAPUMP
DEFAULT DIRECTORY EXT_DIR
ACCESS PARAMETERS
(
nologfile
)
LOCATION ('MEMBERS_EXT_B.csv')
)
AS SELECT * FROM SCHEMA_B.MEMBERS;


Check the MD5 before removing header from each external csv file
time md5sum MEMBERS_EXT_A.csv
time md5sum MEMBERS_EXT_B.csv


Remove header from the external files
time sed -i '1,3d' MEMBERS_EXT_A.csv
time sed -i '1,3d' MEMBERS_EXT_A.csv


Compare md5 with following commands, If both tables contains exactly same data then md5 values will be same.
time md5sum MEMBERS_EXT_A.csv
time md5sum MEMBERS_EXT_B.csv



Here is the execution output

drop table SCHEMA_A.MEMBERS_EXT_A;

table MEMBERS_EXT_A dropped.
Elapsed: 00:00:00.385
CREATE TABLE SCHEMA_A.MEMBERS_EXT_A
ORGANIZATION EXTERNAL
(
TYPE ORACLE_DATAPUMP
DEFAULT DIRECTORY EXT_DIR

ACCESS PARAMETERS
(
nologfile
)
LOCATION ('MEMBERS_EXT_A.csv')
)
AS SELECT * FROM SCHEMA_A.MEMBERS;

table MEMBERS_EXT_A created.

Elapsed: 00:00:48.268


drop table SCHEMA_B.MEMBERS_EXT_B;

table MEMBERS_EXT dropped.

Elapsed: 00:00:00.386


CREATE TABLE SCHEMA_B.MEMBERS_EXT_B
ORGANIZATION EXTERNAL
(
TYPE ORACLE_DATAPUMP
DEFAULT DIRECTORY EXT_DIR
ACCESS PARAMETERS
(
nologfile
)
LOCATION ('MEMBERS_EXT_B.csv')
)
AS SELECT * FROM SCHEMA_B.MEMBERS;

table MEMBERS_EXT_B created.

Elapsed: 00:00:45.530


[dilli@dilliraj mydir]$ time md5sum MEMBERS_EXT_A.csv
a0d831508aca62dec56db1e1148afd06 MEMBERS_EXT_A.csv

real 0m8.005s
user 0m7.136s
sys 0m0.867s


[dilli@dilliraj mydir]$ time md5sum MEMBERS_EXT_B.csv
78d0d09b757e97b6d9609709c3193270 MEMBERS_EXT_B.csv

real 0m7.990s
user 0m7.019s
sys 0m0.971s


[dilli@dilliraj mydir]$ time sed -i '1,3d' MEMBERS_EXT_A.csv

real 0m25.588s
user 0m5.089s
sys 0m20.489s


[dilli@dilliraj mydir]$ time sed -i '1,3d' MEMBERS_EXT_B.csv

real 0m26.335s
user 0m5.110s
sys 0m20.528s


[dilli@dilliraj mydir]$ time md5sum MEMBERS_EXT_A.csv
10c7b4d9b8cb4506f97a542e16da5f0b MEMBERS_EXT_A.csv

real 0m8.276s
user 0m7.245s
sys 0m0.953s

[dilli@dilliraj mydir]$ time md5sum MEMBERS_EXT_B.csv
10c7b4d9b8cb4506f97a542e16da5f0b MEMBERS_EXT_B.csv

real 0m8.389s
user 0m7.331s
sys 0m0.983s

0 comments:

Post a Comment