Quantcast
Channel: SCN : Unanswered Discussions - Data Services and Data Quality
Viewing all articles
Browse latest Browse all 3719

Performace Issue- Need Suggesion

$
0
0

Hi Experts, I am implementing SCD2, My Source has 500 Columns, I have to Compare all Columns, When i did SCD2, When i include all the Columns in Table Comparison for compare , the job never runs, it fails so i splitted the Data flow to multiple data Flows, what i did is as below 1, In the First Data Flow , i took only 1 Column to compare because all the inserts happens in the Data flow so the job will be as below     Source-query transform-table comparision-history preserving-key generation-target 2, From the second and sub sequent data flows i took 50 fieldsin each data flow and before the table comparison i am setting the current flag as 'y' so only updates happens and the Data flow  is as below     source-Query transform(set current flag ='y')-table comparison-target The job runs fine for all the 500 columns comparing. The Problem is It is taking 7 Hours to complete for updating 250000 records Please let me know how to improve performance Thanks


Viewing all articles
Browse latest Browse all 3719

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>