943,789 Members | Top Members by Rank

Software Development > Perl > Duplicate lines in a text file
Ad:
  • Perl Discussion Thread
  • Unsolved
  • Views: 1705
  • Perl RSS
Permalink
Jun 25th, 2009
0

Duplicate lines in a text file

Expand Post »
How to print the duplicates in a text file. Also the line number where the duplicate text has been found needs to be printed. pls help

KZ
Similar Threads
Reputation Points: 10
Solved Threads: 0
Newbie Poster
kummetha is offline Offline
4 posts
since Jun 2009
Permalink
Jun 25th, 2009
0

Re: Duplicate lines in a text file

Hi,

If you are working in Unix then, Use following command

<code>
cat -n file-name > file_name2
</code>

Then use
<code>
sort -u -k 2,4 file_name2
</code>
Reputation Points: 10
Solved Threads: 6
Light Poster
msvinaykumar is offline Offline
48 posts
since May 2006
Permalink
Jun 25th, 2009
0

Re: Duplicate lines in a text file

No, I am not working in Unix. I am working with Windows.
KZ
Reputation Points: 10
Solved Threads: 0
Newbie Poster
kummetha is offline Offline
4 posts
since Jun 2009
Permalink
Jun 25th, 2009
0

Re: Duplicate lines in a text file

Or use Uniq command

<code>
$ cat happybirthday.txt

Happy Birthday to You!

Happy Birthday to You!

Happy Birthday Dear vinay!

Happy Birthday to You!
</code>

<code>
$ sort happybirthday.txt | uniq -dc
3 Happy Birthday to You!
</code>
Reputation Points: 10
Solved Threads: 6
Light Poster
msvinaykumar is offline Offline
48 posts
since May 2006
Permalink
Jun 25th, 2009
0

Re: Duplicate lines in a text file

Click to Expand / Collapse  Quote originally posted by kummetha ...
How to print the duplicates in a text file. Also the line number where the duplicate text has been found needs to be printed. pls help

KZ
What have you tried? Where are you stuck?
Reputation Points: 246
Solved Threads: 67
Practically a Posting Shark
KevinADC is offline Offline
898 posts
since Mar 2006
Permalink
Jun 25th, 2009
0

Re: Duplicate lines in a text file

many a thanks msvinaykumar, i am looking for a perl script that works with windows.
KZ
Reputation Points: 10
Solved Threads: 0
Newbie Poster
kummetha is offline Offline
4 posts
since Jun 2009
Permalink
Jun 26th, 2009
0

Re: Duplicate lines in a text file

open(FILE,"1.txt") || die "$!"; ## 1.txt is a file name
%seen =();
$line=0 ;
while (<FILE>)
{
$seen{$_}++;
$line++;
print "$line : $_" if $seen{$_} > 1 ;
}
Reputation Points: 10
Solved Threads: 0
Newbie Poster
Prakash_8111 is offline Offline
21 posts
since Jun 2009
Permalink
Jun 26th, 2009
0

Re: Duplicate lines in a text file

thanks prakash, it did really work.
also if you could provide help for a script that reads a blank line in a txt file and prints the line above the blank line.
Eg
abcdef

william
marc
sumeet

nair

output:
abcdef
sumeet
thanks in advance
Reputation Points: 10
Solved Threads: 0
Newbie Poster
kummetha is offline Offline
4 posts
since Jun 2009
Permalink
Jun 26th, 2009
0

Re: Duplicate lines in a text file

At some point you should start doing your own school work and the rest of the people participating in this thread you stop doing it for you.
Reputation Points: 246
Solved Threads: 67
Practically a Posting Shark
KevinADC is offline Offline
898 posts
since Mar 2006

This thread is more than three months old

No one has posted to this discussion for at least three months. Please let old threads die and do not reply to them unless you feel you have something new and valuable to contribute that absolutely must be added to make the discussion complete. Otherwise, please start a new thread in this forum instead.
Message:
Previous Thread in Perl Forum Timeline: Assist with code.
Next Thread in Perl Forum Timeline: Compare 2 tables





About Us | Contact Us | Advertise | Acceptable Use Policy
Forum Index | Build Custom RSS Feed


Follow us on Twitter


© 2011 DaniWeb® LLC