Home > Software > Data-Warehouse > DataStage
Interview Questions   Tutorials   Discussions   Programs   Discussion   

DataStage - How to remove duplicates in transformer stage by using stage variables? One example?




1851
views
asked mar September 20, 2014 06:33 AM  

How to remove duplicates in transformer stage by using stage variables? One example?


           

1 Answers



 
answered By vishnoiprem   0  

First select input tab from tx stage and select any partitioning tech with out auto and select perform sort option on that same page and after this select unique option. this will remove the duplicates.

In that Time double click on transformer stage---> Go to Stage properties(its having in hedder line first icon) ---->double click on stage properties --->Go to inputs ---->go to partitioning---->select one partition technick(with out auto)--->now enable perform sort--->click on perfom sort----> now enable unique---->click on that and we can take required colum name. now out put will come unique values so here duplicats will be removed. Before going the data into the Transformer you sort the data on key column(on which key you are going to remove the duplicates)

In the Transformer Create two stage variables A and B

A=Key column B=A

In the constrainnt u have to mention A<>B

Its easy to do using transformer:-

create three stage veriables

Curr=input.col Val= if (Curr = Prev ) then 0 else 1 Prev=Curr(Initialize Prev to -99999)

In Constraint put

Val=0

flag   
   add comment

Your answer

Join with account you already have

FF

Preview


Ready to start your tutorial with us? That's great! Send us an email and we will get back to you as soon as possible!

Alert