Home > Software > Data-Warehouse > Informatica
Interview Questions   Tutorials   Discussions   Programs   

Informatica - How do we eliminate duplicate records in a flat file without using Sorter and Aggregator?




598
views
asked marvit September 20, 2014 08:01 AM  

How do we eliminate duplicate records in a flat file without using Sorter and Aggregator?


           

1 Answers



 
answered By vishnoiprem   0  
Method 1:

USing  unix command sort FileDir\FileName | uniq;
We can use this command in the session properties for the source file as command generating data.

For a flat file use the input type as command at session properties and give the following command in the command option:
sort FileDir\FileName | uniq;

 Method:2
You can achieve this using 2 pipelines.
 
In first pipeline Src>>SQ>>Exp>>Temp_tgt  ( In the expression using v_cnt=v_cnt+1) create sequence number and connect to target.
 
In temp target data will be some thing like below:
 
Col1  col2
a        1
b        2
c        3
a        4
a        5  
b        6
d        7
 
In the second pipeline looks like below:

Src>>SQ>>Exp>>lkp_temp_tgt>>Fil>>Tgt2
 
In the expression create same sequence number as in the first pipeline.
 
Lookup on Temp_target with condition col1 =temp_tgt.col1  and col2 >temp_tgt.col2 and get the col2 value from lookup.

In the filter mention the condition as lkp_temp_tgt.col2>0
 
That's it you will get only unique records.

flag   
   add comment

Your answer

Join with account you already have

FF

Preview


Ready to start your tutorial with us? That's great! Send us an email and we will get back to you as soon as possible!

Alert