How to remove the repeated words in a line, like shown in file below
===============================
vlsispace = "bin/trail/world", "bin/drive/space", "usr/bin/vlsi", "bin/trail/world";
string1 = "bangalore", "chennai", "newyork", "bangalore";
fruit = "apple", "orange", "grape","apple";
==================================
In above file few words are repeated in each line and those extra words need to be removed from the lines
First using grep expression try to capture the line and then split the pattern using a split command. This step will convert the string to a array format .
@fields = split(/delimiter/, “pattern”);
Next step try to remove the repeated word in the array. Finally once all the words are unique, join the array to string with the delimiter that was used to split the string. Join command is used to join the words in array
$string=join(“delimiter”,@array)
There are many ways to write the logic and one of the way is given below
==========================================================
#!/usr/bin/perl
open(rh,"","new_file") or die "couldnt open the file, $!;
foreach $line() {
if($line =~/(\".*\");$){
@fields split(/, /,$1);
my %hashes;
my @unique;
foreach my $value(@fields){
if(!$hashes{$value}){
push @unique $value;
$hashes{$value}=1;
}}
$string = join(", ",@unique);
print wh "$string;\n";
}
else {
print wh $line;
}}
close(wh);
close(rh);
==========================================================
This perl scprit will remove the repeated word from the line and prints a new file with a unique word in a line.
