Home > Software > Data-Warehouse > Informatica
Interview Questions   Tutorials   Discussions   Programs   

Informatica - How will you remove the duplicate records from flat file without using sorter?




621
views
asked marvit September 20, 2014 08:02 AM  

How will you remove the duplicate records from flat file without using sorter?


           

1 Answers



 
answered By vishnoiprem   0  
If your source is Oracle table, then we can use analytical function in source qualifier and filter the duplicate recods and only one record will be selected from the source based on the key columns and will be loaded into target.
Use the below query in SQ:
 
select * from Table
where rowid in
(select rowid from
   (select rowid,row_number()
    over (partition by Key1,Key2,Key3 order by Key1,Key2,Key3) dup
    from Table)
  where dup > 1);
 
For a relational source you can use the select distinct option from source qualifier properties.
For a flat file use the input type as command at session properties and give the following command in the command option:
sort FileDir\FileName | uniq;

At Source qualifier, you can enable 'Select Distinct' Option.
Or, you can also right your own query in SQL so only distinct rows would be selected.
However it works for only relational sources,
flag   
   add comment

Your answer

Join with account you already have

FF

Preview


Ready to start your tutorial with us? That's great! Send us an email and we will get back to you as soon as possible!

Alert