First select input tab from tx stage and select any partitioning tech with out auto and select perform sort option on that same page and after this select unique option. this will remove the duplicates.
In that Time double click on transformer stage---> Go to Stage properties(its having in hedder line first icon) ---->double click on stage properties --->Go to inputs ---->go to partitioning---->select one partition technick(with out auto)--->now enable perform sort--->click on perfom sort----> now enable unique---->click on that and we can take required colum name. now out put will come unique values so here duplicats will be removed. Before going the data into the Transformer you sort the data on key column(on which key you are going to remove the duplicates)
In the Transformer Create two stage variables A and B
A=Key column B=A
In the constrainnt u have to mention A<>B
Its easy to do using transformer:-
create three stage veriables
Curr=input.col Val= if (Curr = Prev ) then 0 else 1 Prev=Curr(Initialize Prev to -99999)
In Constraint put
Ready to start your tutorial with us? That's great! Send us an email and we will get back to you as soon as possible!