[Solved-1 Solution] Removing duplicates using PigLatin ?
What is pig latin ?
- Pig is a high level scripting language that is used with Apache Hadoop. Pig enables data workers to write complex data transformations without knowing Java.
- Pig's simple SQL-like scripting language is called Pig Latin, and appeals to developers already familiar with scripting languages and SQL.
- If you using PigLatin. And you may want to remove the duplicates from the bags and want to retain the last element of the particular key.
- Here the first filed is a key. And if you want the last record of that particular key to be retained in the output.
- we know how to retain the first element. It is as below. But not able to retain the last element.
- If order by one of the fields in descending order. Its possible to get the last record. In the below code, have ordered by second field of input